I can’t get my head around timing in this sketch. I’d appreciate help getting unstuck. I have an tape liquid sensor and the output is erratic and I’d like to smooth it out by using a running average. I’d like to publish a value every 30 seconds that is an average of values taken every 3 seconds. The sketch will average 10 values. How would I make it take a value every 3 sec and publish the average of 10 readings?
Here’s the sketch:
// This #include statement was automatically added by the Particle IDE.
#include <Ubidots.h>
const int numReadings = 10;
int readings[numReadings]; // the readings from the analog input
int readIndex = 0; // the index of the current reading
int total = 0; // the running total
float average = 0; // the average
//token for ubidots
#define TOKEN "MYTOKEN"
Ubidots ubidots(TOKEN);
#define SENSORPIN A0
#define SERIESRESISTOR 547.0
void setup(void) {
Particle.variable("bilgeLevel", readings); //make bilge level avail via var
// initialize serial communication with computer:
Serial.begin(9600);
// initialize all the readings to 0:
for (int thisReading = 0; thisReading < numReadings; thisReading++) {
readings[thisReading] = 0;
}
}
void loop(void) {
// subtract the last reading:
total = total - readings[readIndex];
// read from the sensor:
readings[readIndex] = analogRead(SENSORPIN);
// add the reading to the total:
total = total + readings[readIndex];
// advance to the next position in the array:
readIndex = readIndex + 1;
// if we're at the end of the array...
if (readIndex >= numReadings) {
// ...wrap around to the beginning:
readIndex = 0;
}
// calculate the average:
average = total / numReadings;
// send it to the computer as ASCII digits
Serial.print("voltage count is ");
Serial.println(average);
delay(200);
// convert the value to resistance
average = (4096.0 / average) - 1;
average = SERIESRESISTOR / average;
Serial.print("Sensor resistance ");
Serial.println(average);
delay(200);
ubidots.add("bilgeLevel", average); //define bilge level as ubidot var
ubidots.sendAll(); //send vars to ubidot
// Publish temp var for reading remotely
Particle.publish("bilgeLevel", average);
}
Also, I’ve picked up that I should be using millis() instead of delay. Not sure of how to implement (or if delay is fine here)
Thanks in advance
Since you’re taking a reading every 3 seconds, you want 10 readings, and you want to publish every 30 seconds, you’re replacing your whole array for each publish. That’s fine (though I wouldn’t call that a running average), but you don’t need an array to do that. If there’s nothing else in loop that you need to do other than what you’re showing here, then it would be ok to use delay(). You just need something like this (not complete, or tested),
void loop() {
int total = 0;
for (int i=0; i<10; i++) {
total += analogRead(SENSORPIN);
delay(3000);
}
average = total/10;
// convert the value to resistance
// publish
}
It’s only a little more complicated if you want to use millis(). You need to create 3 global variables, and do something like this,
int total = 0;
int counter = 0;
unsigned long readNext; // set this equal to millis() in setup
void loop() {
if (millis() - readNext >= 3000) {
total += analogRead(SENSORPIN);
counter++;
readNext = millis();
}
if (counter == 10) {
counter = 0;
average = total/10;
// convert the data
// publish
total = 0;
}
// do other stuff here if you want
}
Simple averaging of data should work ok if your noisy data is scattered about the true value randomly without many outliers. If you have many outliers that are way low or high, you might want to consider something like a truncated mean filter. In this technique, you sort an array of values from low to high, throw out some number of the lowest and highest values, then average the remaining points.
A median filter (sort the points and return just the central data point) is also very useful in this situation. No outlier analysis is needed as no decisions about which samples to throw away are done.
The most powerful technique here would likely be k-means where in this case you might choose to partition your data into three clusters, a central or good cluster, a high outlier cluster and a low outlier cluster.
millis(), always millis() (not as good as a Timer but you can’t have everything), delay() is bad habit as far as I’m concerned.
There are probably arguments for letting the cloud sort out rolling averages and outlying sensor readings, however I don’t think this is something Ubidots can currently do for you. In addition to the averaging solutions suggested in this thread there are several others offered in c++ libraries like boost that might apply.
It could be worth getting an oscilloscope so you can see how unstable that voltage level is or at least shoving values out of the serial port a bit closer together than every 3 seconds and plotting the results (there are a bunch of programs for doing that or you can just use a spreadsheet). To me it does’t seem like an unstable reading is something you should be expecting form this not inexpensive sensor. This better fix might be circuit based…
Thanks everyone for the feedback. This has been an interesting adventure into sensors, output dynamics and how to best manage sub-optimal results either through hardware or software. I’m a noob to this area but have a background in science and engineering so I have just enough knowledge to be dangerous. I appreciate your responses but am struggling to implement solutions on my own. Specifically, you mention median filters. I have another sensor that is an ideal candidate for median filtering (a temperature sensor that produces spiky values every so often) but I dont see examples or lib’s to help get this done in the particle world. Specifically there’s a lib written for arduino by Phillip Schmidt that looks like it would work in my case but is not ported over to Particle (https://github.com/daPhoosa/MedianFilter).
I guess my question is this; how to best proceed when this is the case - no example code to adapt, and no libs to leverage, and no skill to write on your own?
I don’t think you really need a library to do a median filter. Here is how I implemented a truncated mean filter.
if (timeToRead) {
for (int i=0; i<numberOfReads; i++) {
temps[i] = getTemperature(); // fill the array with 10 readings
}
qsort(temps, 10, sizeof(float), compare); // sort to put any outliers on either end of the array
float avg = (temps[3] + temps[4] + temps[5] + temps[6])/ 4.0; // take the average of the center 4 values, discarding the 3 on either end
Serial.printf(" Average of center 4 is: %.1f", avg);
timeToRead = false;
}
if (Time.now() > (now + delayBetweenReadGroups)) {
timeToRead = true;
now = Time.now();
Serial.println();
}
}
int compare (const void * a, const void * b) {
float fa = *(const float*) a;
float fb = *(const float*) b;
return (fa > fb) - (fa < fb);
}
If you want a true median filter, you can just take the center value of the array after doing the qsort.
Thanks Ric. That’s what I’m looking for. My temp sensor throws out a spike pretty often that is typically a single value that is way out of line with the surrounding data and this is preventing me from doing any meaningful alerting on thresholds. As I am a self professed noob, can you explain the lower half of the code you posted above? What variables need to be defined as global, local, etc.
Much appreciated.
Ok, the arguments for the function, const void * a and const void * b are what qsort requires. Having the pointers typed as void makes the function very flexible, so you can compare all types of things. However, when it comes to implementing the comparison, you do need to know what kind of things you’re comparing. To do that, you cast the void pointers to a specific type of pointer; in this case, since we’re dealing with floats, we cast to float*. That’s what this part means: (const float) a*. So that part just casts from a void pointer to a float pointer. We still don’t have the value that’s pointed to by a. To get that value, you use the dereference operator, which is the asterisk before the parentheses: * (const float) a;*. So, with that, fa is equal to the value that the pointer a points to.
The return line is just a succinct way to return -1 if fa < fb, 0 if the two values are the same, and 1 if fa > fb. The expressions in the parentheses evaluate to true or false, which is equivalent to 1 or 0. So, that line is equivalent to this,
if (fa<fb) {
return -1;
}else if (fa>fb) {
return 1;
}else{
return 0;
)