Floating point accuracy

General
1

I am using this formula in my code.

z = (6.112*pow(2.71828,(17.67*x/(x+243.5)))*y*2.1674)/(273.15+x);

with values x=20.88 and y=88 (all are floats)

I get z=15.7

However, if I plug in the same values in an excel spreadsheet

=6.1122.71828^(17.67B39/(B39+243.5))C392.1674/(273.15+B39)

I get 16.006

Difference is almost 2%. Why the difference?

Any help appreciated

– Sunder

2

Excel formula got garbled in the cut&paste, here’s the right one
=6.112 * 2.71828 ^ (17.67*B39/(B39+243.5)) * C39 * 2.1674/(273.15+B39)

3

Check your excel formula… are you missing a left-parentheses “(” before the 2.71828 number? Also, I don’t like the lack of operators between the cell references and float numbers; might not affect anything though.

You beat me to it.

Should this be?:

=6.112 * ( 2.71828 ^ (17.67 * B39 / (B39+243.5) ) ) * C39 * 2.1674 / (273.15 + B39)

Also, not sure if Excel processes operands in the same order as C++… verify all the groupings by adding more parentheses so it is unambiguous how to sequence the math operations.

I’m leaning towards a rounding error because of the number of decimal places each system uses.

Also, check this out. https://www.microsoft.com/en-us/microsoft-365/blog/2008/04/10/understanding-floating-point-precision-aka-why-does-excel-give-me-seemingly-wrong-answers/

4

Thanks for the comments, I will do some more digging. I added additional parentheses (especially around the exponent)-no change. Error seems to vary, for other values of x&y, the difference is negligible.

5

I fixed the formatting of the C code in your original post by adding back-quotes around it. I have a couple of ideas:

  1. In C, float is 32-bit single precision. I am sure Excel uses more precision than that, either double which is 64-bit or better. You could try converting your code to use the C type double.
  2. You have to be careful with the Arduino print routines. They do not always print numbers at full precision. I would try using sprintf() with the %f format directive instead. You might find that your 20.88 and 88 are really different values that are rounded.

Looking at your function in single-precision arithmetic, there is a line from about x=20.36,y=88.95 to x=20.73,y=87.05 across the z surface where the value can be 15.7

surfSingle
surfSingle999×755 298 KB

1 Like
6

@bko thanks for your comments, the 3D graph looks pretty informative. I changed to double precision, I was using snprintf already, so didn’t change it. But changed to %.3f instead of %.1f. Still there is a difference between C calculation and Excel. For example:

x=19.39 y=95 z (excel) 15.8369313 z (from c) 15.476 delta % 2.27904823

There are other values that are spot on, for example
x= 24.13 y=60 z (excel) 13.15247847 z(from c) 13.152 delta % 0.003637904

For my application, this level of accuracy is acceptable, so from a practical point of view, I am not blocked.

–Sunder

1 Like
7

I would suggest you to first go with all “upper” multiplications and then dividing stuff per parts, it’s possible that you are hitting some underflow somewhere or making a number too small to be correct represented

8

I found the problem. It’s in my code. I have two external sensors and I read them both and do two identical calculations. so I cut and paste the second one. long story short- the second one had one variable incorrect. If I was the more observant type, I’d have noticed that the error is always in the second sensor.

btw float is plenty, I didn’t need double. Thanks again for all the helpful suggestions.

1 Like