Retained variable not retained on Electron after SLEEP_MODE_DEEP

Greetings,

I am using an Electron to take readings from a Maxbotix sonar sensor to read the height of a stream every 15 minutes, and then upload that information to data.sparkfun.com. Each reading is identified by number, so I am trying to track this with a retained variable that will survive the deep sleep I put the unit in every 15 minutes. Unfortunately, on every upload the number remains 1.

Here is the code:

STARTUP(System.enableFeature(FEATURE_RETAINED_MEMORY));
SYSTEM_MODE(SEMI_AUTOMATIC);

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //

#define VERSION "0.5e-160323.1308"  //  Major version-build date.build time

const int HEIGHT = 184;		        	//	Height of top of octagonal gasket, in cm

#define SPARKFUN_PUBLIC_KEY "XXXXXXXXXXXXXXXXXX"
#define SPARKFUN_PRIVATE_KEY "XXXXXXXXXXXXXXXXXX"

#define UTCOFFSET -4			    //	Local time offset from UTC.
#define UPLOADINTERVAL 15	    //	Interval, in minutes, between uploads

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - //

#include "SparkFunPhant.h"
#include "SparkTime.h"

#define SERVER "data.sparkfun.com"
#define PULSEPIN D1			//  Pulse width data pin from sonar

retained int readingNumber;    //  Tracking the reading/upload number

int pulse;							//  Length of return pulse from the sonar
int pulseMode;          //  The mathematical mode of pulse set
const byte arraysize = 7; 		//  Array for sonar values (must be odd for mode)
int rangevalue[] = {0, 0, 0, 0, 0, 0, 0};	//  Initial array for sonar

int streamHeight = 0;		//  Height of the stream being monitored

int connectFailed = 0;      //  Tracking the number of failed upload attempts
int badReq = 0;
int unexpRes = 0;
bool uploaded = false;

const unsigned int timeoutInterval = 20000;  //  Period, in ms, to wait for connection

Phant phant (SERVER, SPARKFUN_PUBLIC_KEY, SPARKFUN_PRIVATE_KEY);
FuelGauge fuel;

UDP UDPClient;



void setup()
{
		Serial.begin(57600);		        //  Begin the Serial interface
		Serial.print ("Hidrosonico version ");
	        Serial.println (VERSION);
		Serial.print ("Particle firmware version ");
		Serial.println (System.version());

		pinMode (PULSEPIN, INPUT);	        //  Sonar pulse width pin

		Time.zone (UTCOFFSET);              //  Set the local time zone
}



void loop()
{
	//	If this is not a designated upload minute and the uploaded flag is up,
	//	put the flag down

	if(Time.minute() % UPLOADINTERVAL != 0 && uploaded == true)
	{
			uploaded = false;		//	Reset the flag to false in prep for next upload
	}

	//	If this is a designated upload minute, and we have not already uploaded
	//	this minute, take and upload a reading.

	if (Time.minute() % UPLOADINTERVAL == 0 && uploaded == false)
	{
		readingNumber++;			//  Increment the reading counter
		takeReading();				//  Take a reading before turning on GSM

		if (Particle.connected() == false)
		{
			Particle.connect();
		}

		Serial.println (Time.timeStr());			//  Print the time for confirmation

		Serial.printf("Stream height: %d", streamHeight);

		uploadData();
	}

	//  Usually we will leave the GSM module off to save power, but this means we
	//	will not be able to reflash the firmware OTA. We will turn the module on
	//	for two hours during the day to enable OTA reflash.

	bool otaWindow;

	if (Time.hour() == 10 || Time.hour() == 16)   //  These will be the hours of the OTA upload break
	{
			if (otaWindow == false)		//  If the window was closed...
			{
						Particle.publish ("OTA_window_open", PRIVATE);
			}
			otaWindow = true;
	}
	else
	{
			if (otaWindow == true)		//	If the window was open....
			{
					Particle.publish ("OTA_window_closed", PRIVATE);
			}
			otaWindow = false;
	}

	if (otaWindow == false && uploaded == true)           //  Provided we're not on an OTA upload break...
	{
			uploaded = false;

			Serial.printf ("Next upload in %d seconds. Sleeping...", sleepTime);
			Serial.flush();			//	For future implementation by Particle
			wait (10);					//	Because flush() is not yet implemented

			System.sleep(SLEEP_MODE_DEEP, ((((UPLOADINTERVAL * ((Time.minute()/UPLOADINTERVAL) + 1)) - Time.minute()) * 60) - Time.second()));
	}
}



void takeReading()
{
	Serial.print(F("Taking readings..."));

	for(int readingCount = 0; readingCount < arraysize; readingCount++)
	{
		/* The MaxSonar measures the distance of objects by bouncing a
	        superaudible pulse off the object and measuring the time of flight
		(TOF) between the emission of the pulse and its return. For the
		MB7369, 1 microsecond TOF = 1mm of distance. For more see:
		http://www.maxbotix.com/articles/085-pt5.htm#codes*/

		pulse = pulseIn(PULSEPIN, HIGH);		//	Returns length of pulse in us
		Serial.printf ("Reading %d: %d", readingCount, pulse);
		rangevalue[readingCount] = pulse;

		wait (10);				//	Short delay before next pulse reading
	}

	//	Take mode of readings to smooth out any errors or noise
	pulseMode = mode(rangevalue, arraysize);

	Serial.printf ("pulseMode: %d", pulseMode);

	streamHeight = HEIGHT - (pulseMode / 10);

	Serial.println ("done.");
}



int mode(int * x, int n)    //  Calculate the mode of an array of readings
{
	int i = 0;
	int count = 0;
	int maxCount = 0;
	int mode = 0;

	int bimodal;
	int prevCount = 0;
	while(i < (n - 1))
	{
		prevCount = count;
		count = 0;
		while(x[i] == x[i + 1])
		{
			count++;
			i++;
		}
		if(count > prevCount && count > maxCount)
		{
			mode = x[i];
			maxCount = count;
			bimodal = 0;
		}
		if(count == 0)
		{
			i++;
		}
		if(count == maxCount) 		//	If the dataset has 2 or more modes
		{
			bimodal = 1;
		}
		if(mode == 0 || bimodal == 1) 	//	Return the median if no mode
		{
			mode = x[(n / 2)];
		}
		return mode;
	}
}



int postToPhant()
{
	phant.add("1_reading", readingNumber);
	phant.add("2_streamheight", streamHeight);
	phant.add("3_voltage", fuel.getVCell());
	phant.add("4a_connectfailed", connectFailed);
	phant.add("4b_badreq", badReq);
	phant.add("4c_unexpres", unexpRes);

    TCPClient client;
    char response[512];
    int i = 0;
    int retVal = 0;

    if (client.connect(SERVER, 80)) // Connect to the server
    {
		// Post message to indicate connect success
        Serial.println (F("Posting..."));

		// phant.post() will return a string formatted as an HTTP POST.
		// It'll include all of the field/data values we added before.
		// Use client.print() to send that string to the server.
        client.print (phant.post());
        Serial.print (phant.post());
        wait (15000);
		// Now we'll do some simple checking to see what (if any) response
		// the server gives us.
        while (client.available())
        {
            char c = client.read();
            Serial.print(c);	// Print the response for debugging help.
            if (i < 512)
                response[i++] = c; // Add character to response string
        }
		// Search the response string for "200 OK", if that's found the post
		// succeeded.
        if (strstr(response, "200 OK"))
        {
            Serial.println (F("Post success!"));
            retVal = 1;
        }
        else if (strstr(response, "400 Bad Request"))
        {	// "400 Bad Request" means the Phant POST was formatted incorrectly.
			// This most commonly ocurrs because a field is either missing,
			// duplicated, or misspelled.
            Serial.println (F("Bad request"));
            retVal = -1;
						badReq++;
        }
        else
        {
			// Otherwise we got a response we weren't looking for.
						Serial.println (F("Unexpected response"));
            retVal = 1;
						unexpRes++;
        }
    }
    else
    {	// If the connection failed, print a message:
        Serial.println (F("Connection failed"));
        retVal = -3;
				connectFailed++;
    }

    client.stop();	// Close the connection to server.
    return retVal;	// Return error (or success) code.

	delay (500);
}



void uploadData()
{
	int attempts = 0;

	while (postToPhant() != 1 && attempts < 5)
	{
		attempts++;
		if (attempts > 1)
		{
		    wait (1000);
		    Particle.process();
		}
	}

	Particle.publish ("streamheight", (String)streamHeight);

	uploaded = true;
}



void wait(unsigned long period)
{
    /*  Interrupts cannot trigger during a delay(). This function serves the same purpose
    without locking up the clock. */
    unsigned long waitEnd = millis() + period;
    while (millis() < waitEnd){};
}



long sleepTime()
{
    return ((((UPLOADINTERVAL * ((Time.minute()/UPLOADINTERVAL) + 1)) - Time.minute()) * 60) - Time.second());
}

The output looks like:

1_reading	2_streamheight	3_voltage
    1	             184	  4.0109
    1	             184	  4.0122	
    1	             184	  4.0085	

Any advice appreciated.

I believe this is a known problem in the current version of the electron firmware, 0.4.8, that is fixed in the upcoming version, which should be out soon.

1 Like

I’m happy to report a success. retained,variable works great for me and my electron system version 4.8-rc.6
I use retained to preserve a daylight savings time flag; otherwise, my electron goes back to sleep for an hour after waking up from deepsleep.
I learned a few tricks from your snippet. thanks.

1 Like

When my Electron wakes up from Sleep Mode the retained variables are working most of the time. In a 12 hour period with Sleep Mode called every hour after a 1 minute of wakeup and then back to sleep two or three times the retained variables did not hold. Is anyone experiencing this? I am using firmware 0.5.0.

1 Like

Update: I’m now satisfied retained variables work in 0.5.0. ignore the following: I ran a quick test on my electron and it certainly looks like the retained variables aren’t retained anymore with 0.5.0

1 Like

Thanks @Erhughes1944 for checking.

On testing a little more carefully, everything seems to be working. I print a counter after each warm start (recovery from deep sleep) and the counter is incrementing as it should. I use Time.now() to detect a warm start. My retained time should be close to the current time. On cold start the retained time is no where close to Time.now() . Of course, one has to wait for Time.year() > 2000 to be sure that unix time has been initiated. I’m using 0.5.0 on an Electron and the DEV system with DFU flashes.

@Erhughes1944 maybe you are experiencing the inconsistency I am. For a majority of times values are held. For example like I indicated earlier in a 12 hour period they hold for 10 out of the 12 sleep cycles. See if you can do more testing over a period of time. Why did you think earlier they were not working?

Based on your post, I started my test Electron at 0130 and just checked at 0630. It ran 210 deep sleep cycles. Unfortunately WIN 10 loses the serial connection so the log just has the start and end. However, the count is about what I expect and it did do 210 consecutive cycles of run 30 seconds and deep sleep 60 seconds. as before DEV, DFU flashes and 0.5.0
My previous claim of error resulted from not observing the requirement that the unit be restarted from no power to initialize the retained variables; although, that is not strictly true as they are initialized after DFU.

Thanks @Erhughes1944 . I will continue monitoring on my end. So far today the retained variables are holding their state.

Any news on this issue?
I am having the same problem…

What version of the Electron system firmware are you using? It should work properly in 0.5.0 and later; the current version 0.5.3. Also, could you show the variable declaration for your retained variables?

This is an excerpt from the code I use. The rest is just calculation functions or pin readings.
Does this make sense?
I am running 0.5.3

STARTUP(System.enableFeature(FEATURE_RETAINED_MEMORY));
SYSTEM_MODE(SEMI_AUTOMATIC);

// Send Values every 12th time the device boots up
const int sendInterval = 12;                                    
                                    

// Array to store the collected distance values

retained int aDist[sendInterval];
retained int counter = 0;

void setup() {
}


void loop() {
    
    aDist[counter] = getDist();    // Read Sensor
    delay(1000);
    
    if (counter == (sendInterval-1)){
        Particle.connect();
        waitFor(Particle.connected, 30000);
        Particle.publish("s",statusmessage);
        Particle.process();
        delay(3000);
        counter = 0;
        checkUpdates();
    }
    
    else{
        counter++;
        }

    System.sleep(SLEEP_MODE_DEEP, getWaitTime()); 
}

Okay, as a quick update on this issue - I just did a quick test with the following code and it seems to work. The strange thing is, I haven’t changed anything, but removing some application-specific things. PLUS the electron I tested it on is running 0.5.2…

STARTUP(System.enableFeature(FEATURE_RETAINED_MEMORY));
SYSTEM_MODE(SEMI_AUTOMATIC);

const int sendInterval = 12;
const int bootInverval = 20;
retained int counter = 0;

void setup() {
    }

void loop() {
    Particle.connect();
    waitFor(Particle.connected, 30000);
    Particle.publish("test",String(counter));
    Particle.process();
    
    delay(5000);
    
    counter++;
    
    if(counter == sendInterval){
        counter = 0;
    }
    
    System.sleep(SLEEP_MODE_DEEP, bootInverval); 
}

I modified the first example so it would compile standalone and the retained memory worked properly for me. I’m not sure why it wasn’t working for you in your actual app code. Mysterious.

Okay forget about the stuff I wrote before (the second one).

I have tried the code above on firmware 0.5.2 and 0.5.3 and the counter is not counting up - it just stays at 0.
Any ideas?
I have gone over the code several times and have no idea what causes these problems.

Tested your code with 0.5.2 and it just counts up.
How are you powering the device?
How does it behave if you cut down your sleep time to e.g. 5sec?

1 Like

Ok this is becoming weirder and weirder.

I am powering the device with a new LiPo battery (3.7V, 4400 mAh) with about 80% charge left (based on the measurements the electron did). For some tests I left the USB cable connected and for some I didn’t. It did not really make a difference. I am aware, that there seem to be some issues if you just power it via USB, that’s why I have tested both.

I will check a shorter sleep time tomorrow in the morning - just out of curiousity: If that worked - what would that mean?

As an info: There is not much connected to the device. There is just an Sensor hooked up to a digital pin as a power supply (the sensor only draws about 5mA) , ground and Pin D0 and D1 for I2C-communication.
Eventhough i couldn’t imagine why - is there any possibility that it’s a hardware issue?

BTW, I’d recommend using System.sleep(SLEEP_MODE_DEEP, bootInverval, SLEEP_NETWORK_STANDBY); to cut down on the reconnection time and data consuption.

Two other things I could think of:

  • could it be that you’ve got a beta Electron (if you backed the Electron on Kickstarter)?
  • there should be a 0Ohm resistor between 3v3 and VBAT - what resistance do you measure between these two pins (when not powered ;-))?

Okay the thing about reconnection time and nerwork standy sounds interesting. Does this also apply if there is 12 hours between connecting?
Basically how my code is supposed to work is:
every hour: boot, take a measurement, write data into an array
every 12th time: boot, take a measurement, write data into the array, connect to cellular and send the whole array into the cloud.

I don’t know exactly why, but I think I might get everything to work - I’ll keep you updated when I have found the reason/solution.

UPDATE:

I tried a lot of different stuff today and got it to work for a short period of time.
The code I posted above suddenly worked on both devices I checked and so I started building up my program based on that, gradually adding functionality to it.
At the point when I started implementing the sensor (I2C) in the code everything went down - retained variables stopped working again.

That’s why I thought, there might be a problem with the code talking to the sensor, which basically just consists of some I2C-communication using the Wire-Library.

So I removed it all again and then even went a step further backwards and uploaded the code I sent you again - and retained variables ceased working again.

At this point I have tested:
2 different boards (both not beta) - both not modified
Firmware 0.4.2, 0.5.2 and 0.5.3
with or without breadboard
with and without the USB plug plugged in
with and without hardware (D0,D1 for I2C, D7 for power to the sensor)
different deep sleep times between 5 seconds and 1-2 minutes

At this point I really don’t know what to do. I’ve reset my Electron, uploaded Tinker again, flashed all the firmware I had worked with before (the ones mentioned above) and tried almost all configurations possible.

Is there a way to completely reset an Electron? I think it might be a soft-/hardware-problem - maybe something i might have triggered with my code that keeps retained memory from working properly…

Any ideas?
(I have opened a Ticket/Support request parallel to posting here and if this leads to a solution I will document it here, so it will be helpful for somebody with a similar issue as well)

*push