Very strange firmware behavior

I apologize for posting so many requests lately for help with a variety of topics but there are so many experts here willing to assist a neophyte.

During frequent flashing of a variety of embellishments to my application, a block of code inside the loop (screenshot attached) executes immediately upon restart before a connection to the cloud is achieved. But during testing I discovered very strange behavior which I cannot explain.

The issue relates to lines 265 through 267. The Particle.publish statements are intended to populate a Google sheet via IFTTT, and they work just fine under normal operation. But if the 'delay' statement on line 266 is not commented out, the publish statement on line 267 executes upon application restart every time but the publish statement on line 265 never does! If I comment out line 266 as in the screenshot (and make no other edits of any kind anywhere), nothing gets published at all. Yet in both cases the rest of the code in the block executes because the relay closes every time due to the instruction on line 260.

The second attached screenshot shows a portion of my Google sheet "log" of events. Row 74 was added after a restart at 7:45:26 when the 'delay' statement was not commented out (the time stamp is missing because the associated variables don't get defined until a cloud connection is established). Another restart at 7:49:06, when the 'delay' statement was commented out did not result in an "auto on" event.

I added a Particle.connected() test at the start of the block. Thereafter, if 'delay' is not commented out, the 'auto on' event still gets published. When the 'delay' statement is commented out, things look normal, i.e. both publish commands execute (third screenshot).

Can anyone explain this? Can anyone think of a fix?

@blshaw45, couple of things. One is that I don't see lastTankUpdate being reset to Time.now() after if(Time.now() - lastTankUpdate > 43200) triggers. This would cause this test to trigger every time loop() gets around to it again. I assume somewhere lastTankUpdate is being initialized to Time.now().

The second thing is that you are likely hitting the max publish limit of 4 publishes within 4 seconds, explaining why it appears that your "Auto on" is not publishing when it is actually being blocked due to the publish limit. When the delay it added, the 5s delay effectively "resets" your publish count and things appear to work again. This is likely due to the issue above where lastTankUpdate is not being reset.

4 Likes

Brilliant!

lastTankUpdate is intially defined as unsigned long lastTankUpdate = Time.now();

In the problem routine, it is reset 30 minutes after the relay operates, but you've made a good point that it should be reset earlier, probably right after the relay operation on line 260. Otherwise, the test for 12 hours will evaluate as true repeatedly as fast as the processor can run.

Right now the cell signal where I am is so poor that my test Boron can't reconnect to the cloud. Once (if) it does, I'll give it a try.

@blshaw45, when you initialize lastTankUpdate, Time.now() may not be accurate and could be set as soon as a Cloud connection is completed. It would be best to set it in setup() and make sure you are Particle.connected() and Time.isValid(). Also, I would reset lastTankUpdate right after the if() statement so it won't trigger again for another 12 hours.

Gotcha! Will do. If I can ever get the thing back on line.

@blshaw45, when you say "back on line", do you mean by itself? Did you try putting it in Safe mode?

Had to wait until this morning to pursue a fix. Yes, I had to put the Boron into safe mode. Clearly the thing was hung up by that block of code that we've been discussing.

Even after makng the modifcation you suggested to put 'lastTankUpdate' right after the if statement, the Boron would still lock up. In order to get it to connect to the cloud I had to comment out all of that code. Now I have to play around with it some more to get it to work reliably.

Thanks for all your advice!

BTW, here's how that block of code looks at the moment; still locks up the Boron:

if ( Particle.connected()){
   if ( Time.isValid()){
      if ( Time.now() - lastTankUpdate > 43200){
         lastTankUpdate = Time.now();
         if ( pumpOn == false){ 
         controller.turnOnRelay(2); // Turn pump on
         relayState = controller.readRelayStatus (2);
         pumpOn = true;
         pumpOnTime = millis(); // Set start time
         Particle.publish("Well", "No message for 12 hours by " + myID + " @ " + TimeNow + " on     " + Date); 
         delay(5000); 
         Particle.publish("Well", "Auto on by " + myID + " @ " + TimeNow + " on " + Date); 
      }
      if ( millis() - pumpOnTime > 1800000){ 
         controller.turnOffRelay(2); 
         relayState = controller.readRelayStatus (2); 
         pumpOn = false;
         pumpOffTime = millis(); 
         lastTankUpdate = Time.now();
         Particle.publish("Well", "Auto off by " + myID + " @ " + TimeNow + " on " + Date); 
      }
    }
  }
}

2 Likes

Another follow-up:

since I suspected the issue was with my lastTankUpdate variable, I added a publish command to display the actual contents of both that variable and Time.now(). That code locked up the Boron too.
So I removed the Time.now() reference and the Boron was finally happy! So apparently I have been mis-using Time.now(). Any idea where I went wrong in that regard?

Look at your code just above the Particle.publish(). Instead of test = false;, you have test - false; which doesn't set test and so the if() statement will fire every time it runs. The "lock up" may be because you a) don't test for Particle.connected() before doing the publish and b) because you again broke the max 4 pubs in 4 seconds rule.

Instead of using Particle.publish(), if the device is connected via USB, you can use Serial logging output to provide independent (non Cloud) debugging output.

https://docs.particle.io/reference/device-os/api/logging/

You can use the preformatted text (</>) tool in discord to show your code. I used this Boiler Plate code from a new project in Workbench, for an example. It shows some setup for Serial logging output @peekay123 was referring to.

/* 
 * Project myProject
 * Author: Your Name
 * Date: 
 * For comprehensive documentation and examples, please visit:
 * https://docs.particle.io/firmware/best-practices/firmware-template/
 */

// Include Particle Device OS APIs
#include "Particle.h"

// Let Device OS manage the connection to the Particle Cloud
SYSTEM_MODE(AUTOMATIC);

// Run the application and system concurrently in separate threads
SYSTEM_THREAD(ENABLED);

// Show system, cloud connectivity, and application logs over USB
// View logs with CLI using 'particle serial monitor --follow'
SerialLogHandler logHandler(LOG_LEVEL_INFO);

// setup() runs once, when the device is first turned on
void setup() {
  // Put initialization like pinMode and begin functions here
}

// loop() runs over and over again, as quickly as it can execute.
void loop() {
  // The core of your code will likely live here.

  // Example: Publish event to cloud every 10 seconds. Uncomment the next 3 lines to try it!
  // Log.info("Sending Hello World to the cloud!");
  // Particle.publish("Hello world!");
  // delay( 10 * 1000 ); // milliseconds and blocking - see docs for more info!
}

It can't hurt to let us know what Device (Boron404, 404X, etc.) and Device OS you are using.

In a different thread, some coding methods were discussed. It might be time to show all the code. Or, some reasonable subset.

2 Likes

Duh!! And crap! Good catch. I hadn't had my second cup yet. Stay tuned.

OK, I changed the 'test - false' to 'test = false' and it still locked up the Boron. Next I converted 'Time.now()' and 'lastTankUpdate' to strings in the publish statement. That worked, in the sense that it did not lock up th Boron. But, of course, what I see in the output of the publish statement is two strings of digits.

if ( Particle.connected()){
if ( Time.isValid()){
if ( test == true){
test = false;
Particle.publish("Well", String(Time.now()) + " " + String(lastTankUpdate));
}
}
}

You can format it through the Time class too.

Particle.publish("Well", Time.format(Time.now()) + " " + Time.format(lastTankUpdate))

That will give you more readable strings instead of the raw time int.

1 Like

I just made that change (THANKS) and it gave me the following output:

Wed Oct 23 13:23:44 2024 Fri Dec 31 20:00:00 1999

Thankfully that demonstrates that the 'lastTankUpdate' variable has an incorrect date which is why the routine executes when it is not supposed to (always greater than 24 hours). Now I only need to figure out why since I have this statement in the setup() section:

lastTankUpdate = Time.now();

That's because Time.now() is not valid during setup() in many cases, because the RTC has not been set from the cloud yet.

2 Likes

Hmmm. Peekay123 suggested, yesterday, that lastTankUpdate be set in setup(). So where do you think it should go Rick? Somewhere in the loop, I'd think, to insure it always gets set as soon as possible upon startup. But it cannot go inside the subject block of code after 'if (Particle.connected()){' because the following 'if ( Time.now() - lastTankUpdate > 43200){' test would never return true. And it needs to be set only once at startup.

So maybe a new boolean variable 'Startup' set to true in setup() with the first statements inside the loop being:

if ( Particle.connected()){
if ( Time.isValid()){
if (Startup == true){
lastTankUpdate = Time.now();
Startup = false
}
}
}

Thoughts?

OK, just tried the above. It worked:

Unless you can think of a better way, I'm going to go forward with this.

Set lastTankUpdate to 0 in the initializer. If it's zero, don't do calculations based on it in loop.

In loop, once Time.isValid() and lastTankUpdate == 0 set the value to an appropriate value.

5 Likes

Many thanks, again, to all those who offered advice and counsel regarding my issue. I have made several modifications in alignment with the recommendations made here and have been testing my code extensively over the past few days and everything is working great, i.e. it's rock solid.

This forum is a fantastic resource for those of us who are relative newbies at programming Particle devices (and those, like me, who are older than dirt).

3 Likes