Cyan flash of death

I have two photons running in the same location, which regularly suffer the cyan flash of death after about 2 to three days running.

Both units lock up, flashing cyan.

I’m running net connections to particle (publishing one event), to using http , and blynk.
Lucky for me i have a laptop recording diagnostics - this setup is an hours plane ride away.

system.threading is enabled and the particle connection is in automatic mode.

I have wifi fail detection code that looks like this:

if (!Particle.connected())
      Serial.print("Reconnecting to particle ");
      if((!WiFi.ready()) or (retry>=5))
        // wifi not connected so lets restart the wifi
        Particle.disconnect(); // disconnect gracefully;  // power off wifi chip
        delay(1000); // wait 1 sec
        load_ssids(); // restart wifi
      Particle.connect();  //if the Wifi is not connected then try and connect now.  The Send_Any_Data will check if it is conected later and send all it can
      if (Particle.connected)

My debug stream on fail looks like this:

[100242687] Heartbeat timeout
[100247356] Connecting to
Membrane Pressure: 0.78
O2 pressure: -0.236508
Reconnecting to particle 0
Stop Pump
[100253141] Connecting to
Stop Pump
[100270731] Connecting to
Membrane Pressure: 0.78
O2 pressure: -0.354762
Turn on Pump Fill
[100276518] Connecting to

which suggests an attempt to reconnect to particle, which particle.connected() reports as successful, even though it clearly isn’t, as blynk is repeatedly trying to connect. Shortly after this the device locks up, and it takes a power cycle to recover - I have installed wifi controlled power points for this purpose.

I’ve been trying to track down the issue for weeks. Can anyone shed any light on why these things lock up like this?
I know this has been an issue in the past.

I’m desperate for assistance on this one.

That’s the tricky part with non-complete code:
Where do you call WiFi.on() once you had called
How frequently might this snippet be executed in case of connection loss?
What’s happening in load_ssids()?
Is your device accessible otherwise (e.g. Particle.function() or Particle.variable()) or can it Particle.publish() while in this state?
Do you see a “device came online” message in the console?

You may want to waitFor(Particle.connected, 30000) after your Particle.connect() call.
Also AUTOMATIC reconnect might not play well with your own code, you might rather try SEMI_AUTOMATIC.

1 Like

Sorry for the incomplete code, but its (1) too big, (2) under NDA so I have to be a bit careful.
load_ssids() is responsible for initialising the wifi module:

void load_ssids(void)
    if (!WiFi.clearCredentials()) {
         Serial.println("SSID clear failed");
    for(unsigned int i = 0; i < sizeof(ssid_creds)/sizeof(ssid_creds[0]); i++) {
        SSIDCredential *p_ssid = &ssid_creds[i];
        WiFi.setCredentials(p_ssid->ssid, p_ssid->password, p_ssid->auth, p_ssid->enc);
    waitFor(WiFi.ready,60000);  // changed to timeout after 60 seconds if no wifi PM 18-8-16

There’s no ‘device came online’ in the console when the recovery code executes. Can’t reflash either.
The check for particle connection occurs every 30 seconds. Based on the debug messages, its only executing once when the lockup happens. I know the wifi access point is still functioning as I can still access my remote pc on the same network.
Nothing else in loop() runs in the lockup condition - i’d be getting messages on my debug port otherwise.

Will try your code adjustments tomorrow. Thanks.

@twospoons, resetting and setting wifi credentials is a sure way to wear down the part so you may want to do that only once (or on command) since up to 5 sets of credentials can be stored in the Photon. How many are you trying to set?

As @ScruffR pointed out, you need to have the correct SYSTEM_MODE() configured and I noticed that you execute a Particle.publish() in load_ssids() but it is not preceded by a Particle.connect(). You can replace:

    waitFor(WiFi.ready,60000);  // changed to timeout after 60 seconds if no wifi PM 18-8-16

with the following which will connect the WiFi and then the Cloud:


Of course this assumes that WiFi is on. You may want to disable the credentials settings portion of the code and manually set them once to see if this is part of the problem. Do you have SYSTEM_THREAD() enabled?

1 Like

So I have made changes as suggested. I have a new wifi restart routine that looks like this:

    void restart_wifi(void)
    waitFor(WiFi.ready,60000); // wait for wifi connection (timeout 60seconds)
    Particle.connect(); // need this in semi-auto mode
    waitFor(Particle.connected,60000);  // changed to timeout after 60 seconds if no wifi PM 18-8-16

One of the two test units has locked up after 5 days running, the other is still going.
I’ve pinpointed the lockup to this line:

waitFor(WiFi.ready,60000); // wait for wifi connection (timeout 60seconds)

That’s supposed to have a 60 second timeout, right? But that’s where its got stuck.

I have


as suggested, and the wifi restart no longer rewrites the SSIDs.
Wifi coverage is poor where these units are, but I know this code works as I’ve seen it operate correctly without locking when the coverage was worse than usual.
Damn these intermittent failures are hard to debug! Especially when it takes days to check.