Odd analog readings (part 2)

I did a bunch of testing today with my TMP36 temperature sensor to try and get to the bottom of why the readings are so wacky. This is a continuation of the Odd analog readings thread, but I felt you might want to find this information without reading 55 replies :wink:

How to get accurate analog readings:

:large_blue_circle: Hook your analog sensor to the 3V3* pin and GND. The 3V3* pin has a low pass filter on it and is the same node that is connected to the Analog to Digital convertorā€™s reference voltage (VDDA).

:large_blue_circle: Keep your wires as short as possible, and avoid crossing over the top of the Spark Core and the Wifi antenna if you can.

:red_circle: If your analog sensor has a high impedance output, you are going to need to place a 0.01uF (usually marked 103) capacitor from the analog input pin (A0-A7) to GND. This effectively lowers the impedance of the input, and allows the ADC to convert the voltage to a digital value properly. This is by far the BEST thing you can do to help your readings stabilize (thus the double bullet). More on this later.

:large_blue_circle: If you analog sensor has a high impedance output, AND you donā€™t want to install a capacitor to help lower the impedanceā€¦ you might try to delay your analogRead() calls by as much as once per second to help get readings closer to where they should be. You will still see values wildly swinging around, but this helps in a pinch.

:large_blue_circle: If you are using resistance or voltage values in your equations, take a multimeter and measure the value of these items and put the exact values in your equations where possible. Your 3V3* pin most likely does not output exactly 3.30V. This may affect your results slightly.

:large_blue_circle: If your sensor is a temperature sensor, be sure to mount it to a separate breadboard or on the end of a short cable away from the Core. The reason for this, is after a while runningā€¦ the Spark Core Wifi module and 3.3V regulator warm up to over 100Ā°F. This temperature is thermally coupled into all 24 of the pins that are pressed into the breadboard, which in turn heats up the metal pins in each rail. Even if you donā€™t plug your temp sensor directly into these pins, you can be sure the entire bread board is heating up. My TMP36 was running 11Ā°F higher than ambient because of this. As soon as I moved it off to another breadboard, the temperature immediately dropped down to the correct value.

:large_blue_circle: Many sensors benefit from adding a decoupling capacitor across their power and gnd inputs, to help reject common mode noise from entering the sensor and affecting the output. Typical values here are 0.1uF. Keep the leads short as possible.

:large_blue_circle: Code can easily go bad fast, so if you are having problemsā€¦ start by measuring the voltage at the analog input, and calculating what your analog reading should be. See if that is the value you are getting before it goes through your conversions. Break up each step of your conversion to find out where a problem may exist. Follow along with a calculator and see if you are getting numbers that are too big or too small for the types of variables are using.

:large_blue_circle: Have you done all of the above and you still get bouncy readings? You might just have a noisy sensor, or a noisy environmentā€¦ or maybe the thing you are sensing is just fluctuating. If you donā€™t want to set up a low pass filter in hardware, you can try to create a software filter. [I like to use this dilution filter]2 but there are a ton of different ways to do it, pros and cons for many of the ways.


Ok on with the hours of testing I did today with the TMP36:

So one long standing question is whether or not the ADC is setup correctly. I never tried changing the ADC_SAMPLING_TIME found in spark_wiring.h until today. Currently itā€™s set to ADC_SampleTime_1Cycles5 and you can see a good write up of why I think thatā€™s bad here.

So I tried 3 different Sample Times and 4 different levels of capacitance on the input pin. Keep in mind I have my TMP36 on a separate breadboard, and Iā€™m using the A7 input so I can have a GND pin close by for my caps. In addition to measuring the temp, I also measured how long the A to D conversion was taking (Conv. Time). Notes: 10000pF = 0.01uF, and my room thermostat was set to 70Ā°F, These are averages of 100 readings taken 100ms apart.

You can see that with zero capacitance at the sample time of 1Cycles5 (1.5), the situation is pretty bad. Temperature is averaging 21Ā°F higher than normal. And was regularly spiking up to 100Ā°F. Adding just 100pF of capacitance brings the average reading down to within 8Ā°F. Not bad. 470pF improves this even more, and 0.01uF is spot on. The readings barely fluctuated more than 0.3Ā°F with the 0.1uF cap. Conversion times are hella fast, 5us.

sample time of 41.5 and 239.5 effectively increase the allowable input impedance for our circuit, and as you can see even with zero capacitance the readings on average are pretty good! Adding capacitance doesnā€™t change the average much, but it does improve the variability between readings. Conversion time respectively increases to 8.5us and 25us. These are still hella fast.

To keep the conversion time fast, but also help to improve the readings for users that have no idea they should add a capacitor to the analog pin, Iā€™m recommending changing the ADC Sample Time to 41.5.

Check out the variability in the graphs! Be sure to pay attention to the change in temperature scale between the graphs.

Hereā€™s my Spark Core test code:

#include <application.h>

uint16_t temperature = 0;
float voltage = 0.0;
float t1 = 0.0;
bool s = 1;
char tempStr[20];
uint32_t start,end;
uint16_t sample = 0;

void setup()
{
  pinMode(D7, OUTPUT);
  
  Serial.begin(115200);
  RGB.control( true );
  RGB.brightness(255);
  while(!Serial.available()) {
    // Run some test code so we know the core is running!
    s = !s; // toggle the state
    if(s) {
      RGB.color(255,255,255);
      delay(10); // makes it blippy
    } else {
      RGB.color(0,0,0);
      delay(50);
    }
  }
  RGB.brightness(64);
  RGB.control( false );
  Serial.println("SAMPLE, CONV TIME, TEMP C, TEMP F");
}

void loop()
{
  start = micros();
  temperature = analogRead(A7);
  end = micros();
  voltage = (temperature * 3.3)/4095.0;
  t1 = (voltage - 0.5) * 100.0;
  sprintf(tempStr,"%d, %d, %.2f, %.2f",++sample,end-start,t1,(t1*1.8+32));
  Serial.println(tempStr);
  
  // Run some test code so we know the core is running!
  //digitalWrite(D7,s);
  //s = !s; // toggle the state
  delay(100); // makes it blinky
}

Comments? questions?

13 Likes

Thanks for the info! Its detailed and the graphs really help show the difference that the different caps make in the output readings.

One less thing to drive people crazy :smile:

1 Like

Awesome research! Science! Wow!

1 Like

Canā€™t wait for my cores and kits to come in and start testing this myself. Getting my feet wet for the first time in hardwareā€¦gonna be funā€¦
Thanks for your detailed research on this, temperature is a very critical part in the applications I want to use it.

Cheers,
Luc

Thanks @BDub, this is fantastic work! Would you be up for issuing a pull request to change the ADC_SAMPLING_TIME?

3 Likes

Thanks and no problem @zach ! Pull request submitted :spark:

2 Likes

Has the ADC sampling time been changed recently?
I updated my core files last night and my nice steady ADC values are no longer correct. Iā€™m sampling a couple of 50Hz signals every 50 microseconds. I havenā€™t changed my hardware which is in line with @BDub BDub 's recommendations and was working beautifully.
If thereā€™s been no change in the ADC part of the core firmware Iā€™ll dig out some more diagnostic information.

My pull request was merged with master but compile/server2 is still well behind master at this point:

Iā€™m not sure how ā€œsoftā€ updates to compile/server2 on Fridays workā€¦ but apparently there was one this past friday.

There was also a fairly big set of changes from @satishgn to the ADC that used the DMAā€¦ which I canā€™t seem to find now, so perhaps it was rolled back out temporarily.

I think you may be experiencing some other issueā€¦ especially if you are trying to read the ADC every 50us. This is quite fast! Currently the user loop() only runs once every 5-6ms, so you must be reading the ADC in a hard loop to achieve these speeds?

I hope it is some other issue! Nevertheless before the recent update it worked and now it doesnā€™t.
Yes it is a fast sample rate, but not wildly different from what the old Arduino Uno can manage.

Iā€™m using a port of Arduinoā€™s Open Energy Monitor. It does a tight loop data grab for 10 mains cycles - 50Hz in the UK, to establish the voltage, current and phase/power factor. The Arduino manages about 50 samples per cycle when all the surrounding code is included but since the Spark is faster I put in a delay (thank you Spark Team for the delaymicroseconds() function). With a 25 microsecond delay the Spark returns (ed) very clean and accurate results.

Iā€™m away on business for a while but on my return Iā€™ll remove the Spark from the monitoring circuit and start some breadboard testing and also check Iā€™ve not made a silly mistake. Nevertheless before the recent update it worked and now it doesnā€™t and Iā€™m pretty sure that I didnā€™t change anything other than do the rebuild with the newer core code.

This is the test code I used originally - a 500 sample record covered one and a bit mains cycles, consistent with a 50 microsecond sample time - Iā€™ll rerun it when I get back and that should clarify the issue.

void setup() {
    pinMode(D7, OUTPUT);     // Turn on the D7 led so we know it's time
    pinMode(A0, INPUT);
    digitalWrite(D7, HIGH); // to open the Serial Terminal.
    Serial.begin(9600);     // Open serial over USB.
    while (!Serial.available()); // Wait here until the user presses ENTER in the Serial Terminal

    digitalWrite(D7, LOW);
    Serial.println("OK");
    lastRead = millis();
    t1 = millis();
    t2 = millis();
    digitalWrite(D7, HIGH);
    for (int i = 0; i < 500; i++) {
        ADC0[i] = analogRead(A0);
        digitalWrite(D7, !digitalRead(D7));//delay if I forget to add one 
        delayMicroseconds(25);
    }
    t3 = millis();
    delay(500);
    digitalWrite(D7, HIGH);
    Serial.println(((t3-t2)-(t2-t1)));
    for (int i = 0; i < 500; i++) {
        Serial.println(ADC0[i]);
        delay(10);
    }

@phec 'm also interested in the OPEN ENERGY Monitor project. Can you post up and seperate thread on this and how your using it with the Spark Core when you get time?

Sure - the Spark seems made for the job with its built in WiFi. Previously I had wires trailing from an Arduino. Everything was running nicely till last night. I use it in conjunction with a Raspberry Pi which looks after the data, display and switches on the immersion heater when I have spare Solar PV.

@phec Cool, are you using their online interface to view the collected data?

Looking forward to your thread on this. Iā€™m sure others will like this also.

1 Like

Ok now I see what you are doingā€¦ I believe you are probably just reading the ADC too fast while it is still incorrectly set for too fast of a sample rate. Even if you are adding capacitance to your A0 input, you are probably not allowing enough time for the cap that you added to charge back up. Also there could be a bit of an issue with the fact that you are trying to measure an moving waveform on top of that.

Your best bet is to grab the core-firmware from compile/server2 branch, and make the same change I did to the sample timeā€¦ and test that with your code to see if it improves things.

Btw: whatā€™s this do? digitalWrite(D7, !digitalRead(D7));//delay if I forget to add one

Seems like it would be pretty unpredictableā€¦ since D7 is set to an OUTPUT, and a digitalRead of an output returns -1

int32_t digitalRead(uint16_t pin)
{
	if (pin >= TOTAL_PINS || PIN_MAP[pin].pin_mode == OUTPUT || PIN_MAP[pin].pin_mode == NONE)
	{
		return -1;
	}

and digitalWrite() only sets the output to 3.3V or 0V if the value is HIGH or LOW.

#define HIGH 0x1
#define LOW 0x0

Pretty sure !( (uint32_t) -1) is going to be 0 so youā€™re getting a false impression that you are reading D7 LOW and then setting LOW.

Thanks @BDub. The thing is that the ADC worked fine last week. I was getting measured mains voltage within a couple of % of the actual value.
Following the core software change the measured voltage is 1/5 of the real value - entirely consistent with your diagnosis.

The moving waveform is not an issue. At these sampling speeds the maximum rate of change of the voltage even at Arduino sampling rates is a couple of percent between samples.

I donā€™t use the Open Energy display. I had already constructed a 20x4 lcd display driven by the Arduino by the time I came across the open energy project. I use the open energy project software to obtain the power factor - before that I was correlating the two meter values to infer whether I was importing or exporting. Anyway Iā€™ll put much more about that in a new thread when Iā€™m back in a couple of weeks.

Iā€™ll also check what the state of the ADC software is by then and if I can understand how to use Git Iā€™ll take a look at the compile branch 2 you mention.

The digital read/write is a standard Arduino statement to toggle the state of a digital pin. I was just too lazy to remove it here. I can see that it is hardware dependent but for the Atmega a digital read returns the state set for an output pin.
The other hardware dependance I found was something we discussed in another thread. With the Arduino I could use the pull up resistors with a light dependent resistor to trigger an interrupt (many European electricity meters flash an LED every Joule measured). With the Spark the built in pull ups donā€™t seem to have consistent performance so Iā€™ve used external ones - my eyes are getting too old for too much soldering.

Thank you for your help and advice.

Incidentally once the Energy Monitor is working again my next project is to monitor my wifeā€™s bees. Iā€™ve checked the range of the Spark built in antenna and it is inadequate so Iā€™m waiting delivery of another Spark with the uFL and keeping my fingers crossed that this will have a range that reaches the apiary. Iā€™ve got several tiny 100kOhm thermistors that give very consistent and accurate temperature measurements, a small condenser microphone that I havenā€™t tried out yet and a light sensor - so that should be fun. I had a small neural network running on the Arduino so I will try one on the Spark to see whether I can classify the beesā€™ behaviuor. Again Iā€™m looking for a fast sample rate to pick up the buzz frequency and hopefully queen piping. For the hive monitor the Web interface and the sleep modes will be much more useful than they are for the energy monitor where I use UDP.
Kind Regards

The external antenna should help you out with the signal strength. Monitoring the Bees sounds pretty cool. I would like to see what kind of web interface you build to monitor the bee hives.

If you need more range then the uf antenna setup can provide then you could look at XBee radios also. I'm going to do a thread on how well the XBee's work in the next few days.

I'll be sure to catch you in the Open Energy Monitor Thread.

@BDub Iā€™m delighted to report that the ADC is working again following another git pull. The increase in sample time wasnā€™t the problem.
Iā€™ve not needed to change the timings in my own code and it now gives the correct answers again - weird.

I should be able to upload the Spark version of the open energy monitor emonlib and the Spark UDP server software that delivers the energy data to my network this evening.
Detail of the wiring and python client software will have to wait until Iā€™m back from my travels.

The reason for this post is to ask where and how to upload these relatively chunky bits of code. Embedded in a post or is there a better way?

1 Like

In addition to the sample time change, satishgn has re-written the entire ADC to use DMA in ā€œSlow Interleaved Modeā€ so youā€™re probably seeing the benefits of that. I should run my test above again and see how it performs vs. the old ADC configuration.

As far as a place to store/share your files, github is great!

or even

http://gist.github.com for small examples that you donā€™t want to create an entire repository for.

or http://www.pastebin.com if you are feeling lazy :smile:

or ā€¦

right here in the post for the super no-frills method of sharing like this:

1 Like

Looking forward to your Open Energy Monitor code.

1 Like

Here s the open energy monitor port. @BDub will recognise some bits of code. Iā€™ve stuck everything in pretty much as is. Spark/Pi/Arduino. Iā€™ll refine it when I have time. I particularly want to get to the bottom of the ADC behaviour and find out how fast I can sample ready for my next project as soon as I get my next Spark.
Before anyone mentions capacitors - that is what the little rectangular things are on all the inputs. :smile: They donā€™t just reduce the input impedance of the ADC they make debouncing the digital inputs much more robust.

1 Like

Things are really looking up! Thanks for your work guys. The blue line is the ADC output with the current 41_5 delay time when I sample 50 Hz with a 5uS delay in my measurement loop. The real sample time is 46uS per sample (i.e your 41_5 plus a bit).

Restoring the 1_5 delay time in spark_wiring.h gives me the green line with the time per sample down to 24uS. Not quite as fast as the original but nice and stable with virtually no jitter. While the peak is a bit lower than the 41_5 value, the range is within 1.5% so Iā€™m happy with that.