I’ve been monitoring two DHT22 temp and humidity sensors remotely using a photon and the Ubidots dashboard. The setup has been stable and providing good readings with the exception of occasional spikes in the data (5-10 / day). The spikes are always positive and only one value (ex. 42, 42, 43, 95, 42, 43…). I would like to smooth these random spikes out so that I can utilize alerting based on pre-set thresholds. If I want an alert to trigger if the temps go over 45 deg F in the previous series, I’d get an false alert for the erroneous spike. I’ve spent hours combing the internet looking for a way of eliminating the spike through software. Most of the libraries or approaches are tailored to analog readings (not sure if that matters…) or are above my pay grade in terms of complexity. The software approaches seem to zero in on median filters or average filters as a solution.
My question is - is there a hardware solution for the spikes, or what is the best - simplest - way to code out these spikes?
@OBSteve, one quick way to filter out spikes is to ignore readings where the temperature for the current reading cannot exceed the previous reading’s value by more than X where X is based on time between readings and the maximum temperature increase you would expect in that time (the slope of the temperature curve). From your diagram, your typical range of readings 5 degrees over 30 minutes or so (blue curve). So a spike whose value exceeds 2 degrees from the previous reading is noise. You get the idea. 
@OBSteve there’s a cool Ubidots feature that allows you to do this; just set an “allowed range” for your variable:
Thanks for the feedback. With regards to the first approach - how would you do that? declare a variable for the last value and do a conditional if/then with some math to determine if it meets the required condition? And if it doesn’t and you have to discard the value what would you update your logs with, the last valid value or wait until a value that meets your condition comes in…
What would that look like in code?
As for the second approach. I poked around and discovered some alerting built into Ubidots. I’ll set the upper range (i already had set it for 100 F but will set it closer to my expected max for that sensor) and see if it cleans the data up for me.
Hi, this might not be the best solution, codewise, I had the same issue and used the idea of creating a float called tlast (as in temperature last time it was taken = every 30secs), then I used this in the conditional:
if(t >35 && t <= tlast + 5) //35 is my temp is 35 beyond which I want the fans to start
//I used peekay123’s idea of accepting a range of value that are within +5 range
{
digitalWrite(fan1, LOW); //my fans are controlled by a relay, they start on LOW
digitalWrite(fan2, LOW); /my fans are controlled by a relay, they start on LOW
Serial.print(“Fans ON”);
delay(100);
}
else if(t<=35)
{
digitalWrite(fan1, HIGH);
digitalWrite(fan2, HIGH);
Serial.print(“Fans OFF”);
delay(100);
}
Ive had some success setting a rolling average that smooths this out. The problem here is that if it happens too regularly it will still skew your average reading up by a couple of points (i.e. if averaging across 10 readings, a difference like you have of 40 would still pull that average up by 4…).
The other alternative mentioned here is to set a threshold for changes that remember the reading from the last loop before you take the next reading, then comparing that against the new reading to decide whether to throw out the new… something like:
int lastGoodreading;
int newReading = sensorFunction();
if (newReading >= lastReading*1.20 || newReading <= lastReading*0.80)
// change to suit % tolerance (20% here)
{
newReading = lastGoodreading // Bad Reading :-(
}
else
{
lastGoodreading = newReading // Good Reading!
}
Another option is to use the TemperatureHumidityValidatorRK library.
/**
* @brief Smooths out values from sensors like the DHT11 and DHT22
*
* On Gen 3 devices in particular, these sensors can return invalid (bitshifted)
* values but since the sensor doesn't send a CRC, there's no way to determine
* the value is bad.
*
* Create a TemperatureHumidityValidator as a global variable. Query the sensor
* periodically, preferably once a second to several seconds and call
* addSample with the temperature and humidity values. If you don't have
* humidity you can omit that argument.
*
* To get the temperature, use the getTemperatureC() or getTemperatureF()
* function.
*
* For humidity, use getHumidity(). This returns relative humidity in
* a percentage from 0.0 to 100.0.
*
* Note: These functions will return nan (non-a-number) until addSample
* is called 10 times with at least somewhat valid samples. Before that, it's
* hard to determine what's valid or not, and this is also why you should
* call it periodically.
*
* The returned temperature is calculated by calculating the mean of all
* temperatures within 1 standard deviation of the mean. This removes the
* outliers and makes the calculation more reliable.
*
* The specific error that is typically encountered is a bit shift to make
* the temperature or humidity twice as large as it normally should be.
* This algorithm works well for filtering that out.
*/
Oh wow, that looks great. Would like to repurpose this for the BME280!