Photon hangs once a day

[aj8uppal]

I have a few photons and an electron that sends data to the cloud. Both use the same code, but the photons hang frequently (once a day). Usually the light is breathing green which means no cloud connection, but sometimes they are breathing cyan but there is no data being transmitted. I have added a couple of techniques from the community to prevent just thing thing but it does not work. Would appreciate any help!

First is the application watchdog:

unsigned long sampletime_ms = 30000;
void setup() {
  ApplicationWatchdog wd(60000, System.reset);
  Particle.variable("LPO_P1", &LPO_P1, DOUBLE);
  Particle.variable("LPO_P2", &LPO_P2, DOUBLE);
  pinMode(D2, INPUT);
  pinMode(D3, INPUT);
  attachInterrupt(D2, isrduration0,CHANGE);
  attachInterrupt(D3, isrduration1,CHANGE);
  Serial.begin(115200); 

Then is the check if the Photon is connected:

void loop() {
    
if(!Particle.connected())
  {
    Serial.println("Particle Disconnected");
    Particle.connect();
  }

  analogvalue = analogvalue + analogRead(0);
  analogcount++;
if(pulse0)
 {
   lowpulseoccupancy0 = lowpulseoccupancy0+duration0;
   pulse0 = false;
 }
if(pulse1)
 {
   lowpulseoccupancy1 = lowpulseoccupancy1+duration1;
   pulse1 = false;
 }
 currtime = millis();
  if ((currtime-starttime) > sampletime_ms)
  {
    analogvalue = analogvalue/analogcount;
    analogtotal = (analogvalue*3.3)/4095;
    LPO_P1 = lowpulseoccupancy1/10.0/sampletime_ms;
    LPO_P2 = lowpulseoccupancy0/10.0/sampletime_ms;
    int response1 = m2xClient.updateStreamValue(deviceId, stream1Name, LPO_P1);
    Particle.publish("Posted to M2X from PHOTON LPO_P1", String(LPO_P1));
    
    int response2 = m2xClient.updateStreamValue(deviceId, stream2Name, LPO_P2);
    Particle.publish("Posted to M2X from PHOTON LPO_P2", String(LPO_P2));

    lowpulseoccupancy0 = 0;
    lowpulseoccupancy1 = 0;
    analogvalue = 0;
    analogcount = 0;
    starttime = millis();
  }
 
}

With these two “fixes” , I still get hangs. I saw reference to a hardware watchdog but I could not find anything on it in the reference.

[BulldogLowell]

@aj8uppal

the photons hang frequently (once a day).

How long is the DHCP lease time on your router? Are the photons assigned a static IP on your LAN?

[aj8uppal]

Lease time is 1 day. Dynamic assignment but “sticky” i.e. same IP for the same MAC unless it runs out. I did say hangs once a day, but sometimes a bit more frequent than that i.e. 2-3 times.

[ScruffR]

if(!Particle.connected())
  {
    Serial.println("Particle Disconnected");
    Particle.connect();
  }

Since your loop() has no considerable delay (which is a good thing normally) the above code block will cause you trouble.
Due to the fact that the cloud connection process will take several seconds but your loop() will execute thousands of times meanwhile, you will constantly knock the connection back in the process.
Make this a one shot process or add a waitFor(Particle.connected, sampletime_ms); after the Particle.connect() statement.

I guess you are using SYSTEM_THREAD(ENABLED)

[aj8uppal]

Thanks for the response. I am using only one thread i.e. not using SYSTEM_THREAD(ENABLED).
So it is a one shot process, therefore would not Particle.connect() block ?

[aj8uppal]

It looks like Particle.connect() has a timeout of 5 seconds ? When I deliberately block the Photon’s IP address and therefore disconnect it from the cloud the Serial monitor shows:

172.27.96.193

374730,207696,0.91

ERROR: Cannot connect to M2X server!

M2x client response code: -1

ERROR: Cannot connect to M2X server!

M2x client response code: -1

Particle Disconnected

Particle Disconnected

Particle Disconnected

Particle Disconnected

Particle Disconnected

Particle Disconnected

172.27.96.193

47193,232935,0.97

ERROR: Cannot connect to M2X server!

M2x client response code: -1

ERROR: Cannot connect to M2X server!

M2x client response code: -1

Particle Disconnected

Particle Disconnected

Particle Disconnected

Particle Disconnected

Particle Disconnected

Particle Disconnected

so I get 6 calls to Particle.connect() in 30s.

Also I noticed that when I reenable the blocked IP address, then while the Photon is reconnecting to the cloud, Particle.connect() returns very quickly giving lots of loops (100’s ?) in a 30 seconds interval.

Still does not explain why the Photon itself disconnects periodically from the cloud and this loop and the watchdog does not allow it to reconnect.

[RWB]

It’s also possible that your internet connection is dropping out here and there daily. It used to happen to my cable internet router in the past.

[aj8uppal]

No not really. I have a rock solid DUAL internet. Check out www.velocloud.net, uses both cable and dsl simultaneously.

[aj8uppal]

Also unlike when I blocked the IP address and then removed the block, it recovered as expected with the Particle.connect() command. But when it looses cloud connectivity on its own, it NEVER reconnects, even though I have that check in the loop AND the Application Watchdog.

[RWB]

Do you have the ability to test with just one Internet connection input source instead of two? That may be the cause of problem for some unknown reason.

Trying a different access point would also let you know if it’s possibly your special router or not.

[aj8uppal]

Unfortunately I've tried just about everything including a different physical location and therefore different AP and internet connection, still the same symptom. Also saw elsewhere in the forum

Long delays causing loss of cloud connectivity

Thanks @peekay123 for the help!

"Alas, it is not enough to use non-blocking delays with millis(). I use it everywhere, but despite that I was always loosing cloud connectivity after a while with several such non-blocking functions. After loosing cloud connectivity the Photon did not reconnect by itself and I had to reset it."

Seems like others have this issue as well even of the code in loop does not have a delay ?

[RWB]

Has something to do with coding because I have a Photon here that has ALWAYS reconnected automatically after letting it run for about at least 6 months. The Photon has been rock solid for me in all respects.

[aj8uppal]

yes, that is what I am trying to find out. Even the application watchdog does not work to reset the photon.

[ScruffR]

@aj8uppal

Even the application watchdog does not work to reset the photon.

That's not surprising, since your code keeps checking out of loop() or calling Particle.process() (even implicitly) and hence feed the WD to not fire.

https://docs.particle.io/reference/firmware/photon/#application-watchdog

If the application has not exited loop, or called Particle.process() within the given timeout,

The absence of a cloud connection does not prevent this to happen.

[Jeff_Burch]

I have a similar problem with my photon locking up every few days. I have enabled the HW watchdog and have a simple logger using the static RAM that records what the photon was doing before it locked up. Once locked up, the watchdog will expire and the photon will reboot. When it comes back up, I can query the static RAM and learn what it was doing just prior to locking up.

I would like someone to inspect CloudClass::publish() in spark_wiring_cloud.cpp (line 52 in my version). The line in question is:

return f.wait().isSucceeded()

[Jeff_Burch]

Oops, I hit enter too soon! Note that the wait() function is called w/o any parameters. Digging through spark_wiring_async.h and you’ll find FutureImpleBase::wait( ). With no parameters, the timeout will be set to 0 which means “never timeout”.

My logging indicates that the photon had just called a function that sometimes issues Particle.publish( ) when an error occurs. So far the evidence points to Particle.publish() never returning… That wait() could be the problem.

[rickkas7]

Under certain circumstances Particle.publish will block for up to 5 minutes, but it does eventually return. That's presumably longer than your watchdog timer.

This post explains a variety of conditions in which Particle.publish will block and how to avoid them:

Particle publish and blocking

It can be a bit confusing knowing whether a call to Particle.publish will be blocking. This note will hopefully clear some thing up. Note: All of the behaviors described here are for system firmware 0.7.0. Earlier versions behave differently. All of the logs were taken on an Electron, though the behaviors are similar (but generally faster) on the Photon/P1. Updates to this document and the full example source can be found here. The basics Publish is a way to send some data to the cloud. Ther…

[jeffpeakmonitor]

I’m having the same problem (Photons disconnect once a day or so, never to reconnect). It only does this at one customer site but I’ve got other devices at that site which stay online.

After reading this thread, I tried adding the watchdog example straight out of the link above into void setup() . This caused my program to hang and never proceed to the main loop.

Running firmware 0.8.0 rc9.

Any suggestions?

[ScruffR]

@jeffpeakmonitor

It only does this at one customer site but I’ve got other devices at that site which stay online.

That might be a combination of IP lease expiry in connection with DHCP IP shortage - double check WiFi router settings.
What brand/type is the router?

[jeffpeakmonitor]

Router is Airport Extreme. The Photons have DHCP reservations in the Extreme, so they always get the same IP after reboot.

It’s on a ranch and they have flaky internet. 10-20 disconnection events per day are reported by Particle Console (and also by some IP cameras at the same site).

The cameras always come back, and 2 Raspberry Pi units I have there also come back although I had to implement some code that would ping 8.8.8.8 and if not found, restart wifi.