timb
March 27, 2014, 1:09am
1
Has anyone had trouble with 64-bit variables on the Core?
I’m working on a library for the MS5607 Barometric Pressure Sensor and getting weird results. Here’s my code:
offP = calData[1] * 131072 + (calData[3] * deltaT) / 64;
sensP = calData[0] * 65536 + (calData[2] * deltaT) / 128;
P = (rawP * sensP / 2097152 - offP) / 32768;
P
, offP
and sensP
are int64_t
types. rawP
is a uint32_t
and calData[]
is an array of the type uint16_t
.
Here’s a flowchart from the datasheet:
It’s returning completely invalid results. I’ve done the math by hand and know what it should be reporting (around 105000).
MS5607 Library by @TimothyBrown
Version: 20130323 - Build: 0xA4
44527
40588
27507
25360
32823
27567
Temp C: 2035
Temp F: 6863
Pressure: -32072
The first six numbers are the six entries in the calData[]
array (C1 through C6 in the flowchart). The temperature results are correct (20.35C/68.63F), but to me it looks like the pressure variable is overflowing, since it should never return a negative value…
BDub
March 27, 2014, 1:50am
2
The compiler might be multiplying 32bit x 64bit first which could be larger than 64bit (explaining the overflow).
Try this:
int64_t temp = sensP / 2097152; // widdle that 64bit number down a bit first
temp = rawP * temp; // then bloat it back up
P = (temp - offP) / 32768;
Raldus
March 27, 2014, 1:52am
3
What is the type of deltaT
? If it’s not a long long
, your expression is being computed as an int
(32-bit, which you’re overflowing) before being stored into a long long
.
Raldus
March 27, 2014, 1:54am
4
No, a 32-bit x 64-bit can't overflow, because the result is 64-bit.
BDub
March 27, 2014, 2:03am
5
Say what?
64bit x 2 = 65bit, which overflows back into 64bit space.
However since this is signed, it would be 7FFF FFFF FFFF FFFF x 2 which would not overflow to 65 bits, but multiply is by 3 and it will (i.e. a 2 bit number)
64bit x 32bit is definitely going to overflow.
32bit x 32bit is just going to be 64bit, but not overflow.
Raldus
March 27, 2014, 2:05am
6
BDub:
Say what?
Gah, you're right, and I need about a gallon more coffee ...
1 Like
BDub
March 27, 2014, 2:07am
7
No problem… I pour this shtuff in my daily!! so I’m surprised I can even read anymore…
Raldus
March 27, 2014, 2:20am
8
Still, the original question stands: if deltaT
is not a 64-bit datatype, everything’s being computed as 32-bit, which will overflow.
No, wait, it’s going to overflow anyway:
offP = calData[1] * 131072 + (calData[3] * deltaT) / 64;
The first two terms are 32-bit:
calData[1] * 131072
The result of this is going to be 32-bit, which will overflow (caldata[1]
is 40588, if I’m interpreting his data correctly, and that will overflow).
EDIT: To force the expected computation, we can go old-school:
offP = (long long)calData[1] * 131072 + ((long long)calData[3] * deltaT) / 64;
Er, to make it more readable, perhaps we should use:
offP = calData[1] * 131072LL + ((long long)calData[3] * deltaT) / 64;
timb
March 27, 2014, 2:25am
9
Here’s the whole sketch. @BDub ’s suggestion didn’t work. (Note, the only way I can read the data is by casting int64_t P
as an int32_t
with the Serial1.println
command, as it doesn’t support long longs.)
#define MS5607_ADDR 0x76
#define RESET_REG 0x1E
#define PROM_BASE_REG 0xA0
uint16_t calData[6] = {0, 0, 0, 0, 0, 0};
uint32_t rawT;
uint32_t rawP;
int32_t deltaT;
int32_t Tc;
int32_t Tf;
int64_t offP;
int64_t sensP;
int64_t P;
void setup() {
Wire.begin();
Serial1.begin(115200);
Serial1.println("MS5607 Library by @TimothyBrown");
Serial1.println("Version: 20130323 - Build: 0xA7");
sensorStart();
getData();
Serial1.print("Temp C: ");
Serial1.println(Tc);
Serial1.print("Temp F: ");
Serial1.println(Tf);
Serial1.print("Pressure: ");
Serial1.println((int)P);
}
void loop() {
}
void sensorStart() {
Wire.beginTransmission(MS5607_ADDR);
Wire.write(RESET_REG);
Wire.endTransmission();
delay(5);
uint8_t calReg = PROM_BASE_REG;
for(uint8_t i = 0; i < 6; i++) {
calReg = calReg + 0x02;
Wire.beginTransmission(MS5607_ADDR);
Wire.write(calReg);
Wire.endTransmission();
delay(1);
Wire.requestFrom(MS5607_ADDR, 2);
while(Wire.available()) {
calData[i] = Wire.read();
calData[i] = calData[i] << 8 | Wire.read();
}
Serial1.println(calData[i]);
}
}
void getData() {
/* Read Raw Temperature Data */
Wire.beginTransmission(MS5607_ADDR);
Wire.write(0x58);
Wire.endTransmission();
delay(10);
Wire.beginTransmission(MS5607_ADDR);
Wire.write(0x00);
Wire.endTransmission();
delay(1);
Wire.requestFrom(MS5607_ADDR, 3);
while(Wire.available()) {
rawT = Wire.read();
rawT = rawT << 8 | Wire.read();
rawT = rawT << 8 | Wire.read();
}
/* Read Raw Pressure Data */
Wire.beginTransmission(MS5607_ADDR);
Wire.write(0x48);
Wire.endTransmission();
delay(10);
Wire.beginTransmission(MS5607_ADDR);
Wire.write(0x00);
Wire.endTransmission();
delay(1);
Wire.requestFrom(MS5607_ADDR, 3);
while(Wire.available()) {
rawT = Wire.read();
rawT = rawP << 8 | Wire.read();
rawT = rawP << 8 | Wire.read();
}
/* Process Temperature */
deltaT = rawT - calData[4] * 256;
Tc = 2000 + deltaT * calData[5] / 8388608;
Tf = Tc * 1.8 + 3200;
/* Process Pressure */
offP = calData[1] * 131072 + (calData[3] * deltaT) / 64;
sensP = calData[0] * 65536 + (calData[2] * deltaT) / 128;
//P = (rawP * sensP / 2097152 - offP) / 32768;
int64_t x = sensP / 2097152; // widdle that 64bit number down a bit first
x = rawP * x; // then bloat it back up
P = (x - offP) / 32768;
}
timb
March 27, 2014, 2:26am
10
Thinking about what @Raldus said, can I just cast the 16 and 32-Bit variables as 64-Bit ones when doing math on them?
Raldus
March 27, 2014, 2:27am
11
Yes, calData[1] * 131072
will overflow if calData[1]
is 16K or larger.
EDIT: I’m not sure, but I think an easier way is to simply declare calData
as an unsigned long long
. The values in calData
appear to be only used in computations, and do not seem to be sent back to any hardware registers and the like.
Raldus
March 27, 2014, 2:33am
12
char buf[100];
long long x = 42;
sprintf(buf, "x = %lld", x);
Serial.println(buf);
or, in 64-bit hex:
char buf[100];
long long x = 42;
sprintf(buf, "x = 0x%016llx", x);
Serial.println(buf);
timb
March 27, 2014, 2:42am
13
Raldus:
EDIT: I'm not sure, but I think an easier way is to simply declare calData as an unsigned long long. The values in calData appear to be only used in computations, and do not seem to be sent back to any hardware registers and the like.
Unfortunately that screws up the temperature calculations:
MS5607 Library by @TimothyBrown
Version: 20130323 - Build: 0xA8
44527
40588
27507
25360
32823
27567
Temp C: -25613
Temp F: -42903
Pressure: -8796093082952
The same thing happens if I make deltaT
64-Bit. I'm not used to dealing with 64-Bit math in this fashion, so this is breaking my brain.
timb
March 27, 2014, 3:00am
15
Here we go, something helpful!
float MS561101BA::getPressure(uint8_t OSR) {
// see datasheet page 7 for formulas
int32_t dT = getDeltaTemp(OSR);
if(dT == NULL) {
return NULL;
}
uint32_t rawPress = rawPressure(OSR);
if(rawPress == NULL) {
return NULL;
}
int64_t off = ((uint32_t)_C[1] <<16) + (((int64_t)dT * _C[3]) >> 7);
int64_t sens = ((uint32_t)_C[0] <<15) + (((int64_t)dT * _C[2]) >> 8);
return ((( (rawPress * sens ) >> 21) - off) >> 15) / 100.0;
}
I’m studying the rest of this Arduino library I found, and it seems they store all their variables in a similar method to the way I’m doing it. I’m doing some bit twiddling now and making a bit of progress.
timb
March 27, 2014, 3:18am
16
Behold, progress!
MS5607 Library by @TimothyBrown
Version: 20130323 - Build: 0xB1
Temp C: 2034
Temp F: 6861
Pressure: 70330
offP = ((uint32_t)calData[1] * 131072) + ((calData[3] * (int64_t)deltaT) / 64);
sensP = ((uint32_t)calData[0] * 65536) + ((calData[2] * (int64_t)deltaT) / 128);
P = ((((rawP * sensP) / 2097152) - offP) / 32768);
Raldus
March 27, 2014, 3:20am
17
Ah,
int64_t off = ((uint32_t)_C[1] <<16) + (((int64_t)dT * _C[3]) >> 7);
This is a bit different than your code :
offP = calData[1] * 131072 + (calData[3] * deltaT) / 64;
This is the same as:
offP = calData[1] << 17 + (calData[3] * deltaT) >> 6;
Sorry.
Edit - OK, I’m really confused. If you’ve got things working, I’ll just leave you alone.
timb
March 27, 2014, 3:24am
18
Yeah, I know. I’m following a newer algorithm for the offset calculations then they used in their code.
BDub
March 27, 2014, 3:35am
19
Serial1.print("Pressure: 0x");
Serial1.print((int32_t)((P>>32)&0xFFFFFFFF),HEX);
Serial1.println((int32_t)(P&0xFFFFFFFF),HEX);
maybe start here and see what you really have?
timb
March 27, 2014, 3:45am
20
Yeah, I’m not using that method to print it anymore. I used @Raldus ’ helpful CPP snippet.
Serial1.print("Pressure: ");
char buf[100];
sprintf(buf, "%lld", P);
Serial1.println(buf);
Serial1.print("Pressure: 0x");
Serial1.print((int32_t)((P>>32)&0xFFFFFFFF),HEX);
Serial1.println((int32_t)(P&0xFFFFFFFF),HEX);
MS5607 Library by @TimothyBrown
Version: 20130323 - Build: 0xB1
Temp C: 2035
Temp F: 6863
Pressure: 70329
Pressure: 0x0112B9
0x0112B9 in dec => 70,329
So it matches up. That would be 703.29mb of pressure. I’m trying to find the altitude of the nearby airport so I can convert it to a value at sea-level and compare it to the current weather map.
BDub
March 27, 2014, 3:57am
21
FYI: sprintf() consumes 20KB of FLASH In case you end up running out of space…