Has anyone had trouble with 64-bit variables on the Core?
I’m working on a library for the MS5607 Barometric Pressure Sensor and getting weird results. Here’s my code:
offP = calData * 131072 + (calData * deltaT) / 64; sensP = calData * 65536 + (calData * deltaT) / 128; P = (rawP * sensP / 2097152 - offP) / 32768;
rawP is a
calData is an array of the type
Here’s a flowchart from the datasheet:
It’s returning completely invalid results. I’ve done the math by hand and know what it should be reporting (around 105000).
MS5607 Library by @TimothyBrown Version: 20130323 - Build: 0xA4 44527 40588 27507 25360 32823 27567 Temp C: 2035 Temp F: 6863 Pressure: -32072
The first six numbers are the six entries in the
calData array (C1 through C6 in the flowchart). The temperature results are correct (20.35C/68.63F), but to me it looks like the pressure variable is overflowing, since it should never return a negative value…