Electron reconnection to cell service

I’m trying to get an electron to gracefully handle poor cell service. My goal is that when there is no service, my program disconnects and checks for cell availability every so often and reconnects if it’s available, and keeps chugging along just doing the local things if it’s not.

My problem is that the re-connection doesn’t work. From a system.reset, the electron connects to the cloud just fine, but if I disconnect the antenna, and let the electron disconnect, then reconnect the antenna, the electron won’t reconnect (Cellular.ready() never comes true).

I’ve added the delays between Cellular.on() and Cellular.connect() based on a thread here I can’t seem to find again. I’ve also tried adding Cellular.disconnect() before Cellular.off() with no change.

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);

enum ConnectionStatus {
  NOT_CONNECTED,
  CONNECTING,
  CON_FAILED,
  CONNECTED
};

void cellConnect();
enum ConnectionStatus cstatus; 
unsigned long retryTimer;
unsigned long conTimer;

const unsigned long retryTime = 6000000; //check cell connection every 100 min
const unsigned long conTime = 600000; //give cell 10 min to connect

void setup() {
    cstatus = NOT_CONNECTED;
    Cellular.off();
    delay(5000);
    cellConnect();
}

void loop() {
   switch(cstatus){
      case NOT_CONNECTED:
        cstatus = NOT_CONNECTED;
        break;

      case CONNECTING:
        if(Cellular.ready()){ //if cell is connected then connect to cloud
          Particle.connect();
          cstatus = CONNECTED;
        } else { //if cell is not ready
          if((millis() - conTimer) >= conTime){ //if connection time has expired
            Cellular.off();
            cstatus = CON_FAILED;
          } else { //connection time not expired

            cstatus = CONNECTING;
          }
        }
        break;

      case CON_FAILED:
        if((millis() - retryTimer) >= retryTime){//time to retry connection
          cellConnect();
        } else { //wait to attempt reconnection
          cstatus = CON_FAILED;
        }
        break;

      case CONNECTED:
        if(Cellular.ready()){//still connected
          if(Particle.connected){//cloud connected
            cstatus = CONNECTED;
          } else { //cell good, cloud not connected
            Particle.connect();
            cstatus = CONNECTED;
          }
        } else { //cellular disconnected
          Cellular.off();
          cstatus = CON_FAILED;
        }
        break;
    } 
}

void cellConnect(){
    Cellular.on();
    delay(500);
    Cellular.connect();
    delay(500);
    retryTimer = millis();
    conTimer = millis();
    cstatus = CONNECTING;
}

Hmm… It seems to me that you can end up in a “deadlock” for some cases in your state machine implementation…

eg NOT_CONNECTED will always be NOT_CONNECTED.

Can you state more clearly how you are performing the test?

Do you:

  1. let the Electron connect successfully and breathing cyan
  2. unplug the antenna
  3. reset the Electron

or what’s the flow like?

Here’s one test…

  1. Connect the antenna and restart the Electron - connects successfully and is breathing cyan.

  2. Disconnect the antenna - connection fails, and electron goes into the CON_FAILED state. Connection retries happen on schedule and fail as expected. The electron goes to breathing white

  3. Reconnect antenna, connection retries happen on schedule and still fail.

I’ve set this up to print to the serial monitor at every piece of logic, and the program flow was as I expected. I feel confident that the cellConnect function is being called when it should.

I’ll set this up with a button and interrupt to begin the connection process, and eliminate the timing, just to be sure that the problem isn’t in the logic.

Tom

Does the LED ever flash green when you reinstall the antenna?

The Electron should automatically power cycle the modem if it fails to connect after 5 mins, it will do this again after 5 more mins of unsuccessful cell network connection.

Ok, so I checked it again, and made sure to note the status LED, and put the serial debugging messages back in.

I started with the antenna connected. The electron flashes dark blue once, flashes green, flashes cyan, then connects and breathes cyan.

I then disconnect the antenna. After a few seconds, the connection fails and the led goes to breathing white after Cellular.off() is called.

The antenna is reconnected, and when the reconnect timer expires, the program calls the cellConnect() function. The led breathes white for a second or two, then flashes green. It restarts this white, green cycle about every minute or so. The connection times out as expected, and the electron returns to it’s breathing white, connection failed state.

I hit the restart button on the electron (antenna connected) and the electron connects and breathes cyan, so I know things are still working in general.

Is the modem some sort of “discrete” thing that I could reset or power cycle manually?

Thanks,
Tom

Hitting the Reset button does not cut power to the Modem.

So it’s the code that once rest calls the right functions to reconnect to the cellular network correctly, it’s not that the modem is reset when you hit the Reset button.

Interesting. As another test, I added some code that calls Cellular.off and puts the device in the CON_FAILED state after a good connection has been established. When the retry timer expires, the cellConnect function gets called as expected, and the Electron is able to reconnect. The problem seems to be limited to when I disconnect the antenna, and the cell connection fails unexpectedly.

Tom

So if you do not disconnect the antenna and just call cellular.off then there are no issues?

That's correct. With my modified code (see below), if I leave the antenna connected, it connects on startup, disconnects on schedule, reconnects on schedule, disconnects on schedule, ect. If I disconnect the antenna while connected, the Electron will not reconnect.

Tom

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);

enum ConnectionStatus {
NOT_CONNECTED,
CONNECTING,
CON_FAILED,
CONNECTED
};

void cellConnect();
enum ConnectionStatus cstatus;
unsigned long retryTimer;
unsigned long conTimer;
unsigned long resetTimer;

const unsigned long retryTime = 600000; //check cell connection every 10 min
const unsigned long conTime = 300000; //give cell 5 min to connect
const unsigned long resetTime = 360000; //turn modem off after 6 min

void setup() {
Serial.begin(9600);
cstatus = NOT_CONNECTED;
Cellular.off();
delay(5000);
Serial.println("Startup");
cellConnect();
}

void loop() {
switch(cstatus){
case NOT_CONNECTED:
cstatus = NOT_CONNECTED;
Serial.println("Not Connected" + String(millis(), DEC));
break;

  case CONNECTING:
    if(Cellular.ready()){ //if cell is connected then connect to cloud
      Particle.connect();
      resetTimer = millis();
      cstatus = CONNECTED;
      Serial.println("Finish Connecting" + String(millis(), DEC));
    } else { //if cell is not ready
      if((millis() - conTimer) >= conTime){ //if connection time has expired
        Cellular.off();
        cstatus = CON_FAILED;
        Serial.println("Connection Timeout" + String(millis(), DEC));
      } else { //connection time not expired
        cstatus = CONNECTING;
        Serial.println("Connecting" + String(millis(), DEC));
      }
    }
    break;
  case CON_FAILED:
    if((millis() - retryTimer) >= retryTime){//time to retry connection
      cellConnect();
      Serial.println("Retry" + String(millis(), DEC));
    } else { //wait to attempt reconnection
      cstatus = CON_FAILED;
      Serial.println("Con Failed" + String(millis(), DEC));
    }
    break;
  case CONNECTED:
    if(Cellular.ready()){//still connected
      if(Particle.connected){//cloud connected
        if((millis() - resetTimer) >= resetTime){ //stut off cell for testing
            Cellular.off();
            cstatus = CON_FAILED;
            Serial.println("Shut Off" + String(millis(), DEC));
        } else { //continue with connection
            cstatus = CONNECTED;
            Serial.println("Connected" + String(millis(), DEC));
        }
      } else { //cell good, cloud not connected
        Particle.connect();
        cstatus = CONNECTED;
        Serial.println("Connect Particle" + String(millis(), DEC));
      }
    } else { //cellular disconnected
      Cellular.off();
      cstatus = CON_FAILED;
      Serial.println("Disconnected" + String(millis(), DEC));
    }
    break;
} 
delay(1000);

}

void cellConnect(){
Cellular.on();
Cellular.connect();
retryTimer = millis();
conTimer = millis();
resetTimer = millis();
cstatus = CONNECTING;
Serial.println("Start Connect" + String(millis(), DEC));
}

Weird but the antenna should never be disconnected in real world usage correct?

I’m unplugging the antenna as a proxy for any type of loss of cell signal. If for example, the electron was mounted to a vehicle and you drove out of cell service. I’d like my electron to handle that gracefully.

At the moment, I have a bricked electron that’s supposed to be providing a local display and cloud monitoring for an off-grid solar installation. Something happened to the cell service after I installed the device. It no longer has a strong enough cell signal to connect, and the modem is blocking all the local processes while it continually tries to reconnect. I’ve fixed the blocking problem by enabling the system thread, but it led me to work on a more robust cloud and cell connection. For this application, I could do a system reset if a re-connection is required. I’m just thinking down the road and trying to improve my library.

Tom

Here is the code I use on the Electron to reconnect and handle times were the cellular modem does not connect to the cellular network.

Cellular.connect();  // This command turns on the Cellular Modem and tells it to connect to the cellular network. 
  
   if (!waitFor(Cellular.ready, 600000)) { //If the cellular modem does not successfully connect to the cellular network in 10 mins then go back to sleep via the sleep command below. After 5 mins of not successfully connecting the modem will reset.  
    
    System.sleep(D0, RISING,sleepInterval * 2, SLEEP_NETWORK_STANDBY); //Put the Electron into Sleep Mode for 2 Mins + leave the Modem in Sleep Standby mode so when you wake up the modem is ready to send data vs a full reconnection process.  

You could also do the same with Particle.connect.

3 Likes

I believe there is a bug, at least in 0.6.2, and possibly others, if you interrupt a Cellular.connect() by calling Cellular.disconnect() and Cellular.off(). It seems to leave things in a bad state and I can’t get it to connect again. This is easily worked around, and with the code below I can successfully disconnect and reconnect the antenna at various times without problems.

#include "Particle.h"

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);

SerialLogHandler logHandler;

const unsigned long STARTUP_DELAY_MS = 5000; // To allow for serial connection to be made
const unsigned long CELLULAR_CONNECT_TIMEOUT_MS = 120000;
const unsigned long CLOUD_CONNECT_TIMEOUT_MS = 30000;
const unsigned long PUBLISH_PERIOD_MS = 30000;
const unsigned long RETRY_CONNECT_DELAY_MS = 60000;

enum State {
    STARTUP_STATE,
    CELLULAR_CONNECT_STATE,
    CELLULAR_CONNECT_WAIT_STATE,
    CLOUD_CONNECT_STATE,
    CLOUD_CONNECT_WAIT_STATE,
    CLOUD_CONNECTED_STATE,
    CLOUD_DISCONNECT_STATE,
    RETRY_WAIT_STATE
};
State state = STARTUP_STATE;
unsigned long stateTime = 0;

void setup() {
    Serial.begin(9600);
}

void loop() {
    switch(state) {
    case STARTUP_STATE:
        if (millis() - stateTime >= STARTUP_DELAY_MS) {
            state = CELLULAR_CONNECT_STATE;
        }
        break;

    case CELLULAR_CONNECT_STATE:
        Log.info("turning cellular on");
        Cellular.on();

        Log.info("connecting to cellular...");
        Cellular.connect();
        state = CELLULAR_CONNECT_WAIT_STATE;
        stateTime = millis();
        break;

    case CELLULAR_CONNECT_WAIT_STATE:
        if (Cellular.ready()) {
            Log.info("connected to cellular");

            state = CLOUD_CONNECT_STATE;
        }
        else
        if (millis() - stateTime >= CELLULAR_CONNECT_TIMEOUT_MS) {
            Log.info("failed to connect to cellular");

            Log.info("disconnecting from cellular");
            Cellular.disconnect();
            delay(2000);

            Log.info("turning cellular off");
            Cellular.off();

            // There appears to be a bug in 0.6.2 where if you interrupt a cellular connect,
            // you can't connect again. Work around this by going into deep sleep for a few
            // seconds which will reset the system firmware state. You can tell this is happening
            // if you go into breathing dark blue after doing the Cellular.connect().
            // https://community.particle.io/t/particle-connection-process-modem-cancel-bug-if-modem-registration-aborted/33138/
            Log.info("going into deep sleep to reset device and modem");
            delay(2000);

            System.sleep(SLEEP_MODE_DEEP, 10);

            // Not reached, system will reset and start over with setup and loop again after waking up
        }
        break;

    case CLOUD_CONNECT_STATE:
        Log.info("connecting to cloud");
        Particle.connect();
        state = CLOUD_CONNECT_WAIT_STATE;
        stateTime = millis();
        break;

    case CLOUD_CONNECT_WAIT_STATE:
        if (Particle.connected()) {
            state = CLOUD_CONNECTED_STATE;
            stateTime = millis();
        }
        else
        if (millis() - stateTime >= CLOUD_CONNECT_TIMEOUT_MS) {
            Log.info("failed to connect to the cloud");
            state = CLOUD_DISCONNECT_STATE;
        }
        break;

    case CLOUD_CONNECTED_STATE:
        if (Particle.connected()) {
            if (millis() - stateTime >= PUBLISH_PERIOD_MS) {
                stateTime = millis();

                // We send test events otherwise the device may not realize it has disconnected
                // from the cloud
                Log.info("sending test event");
                Particle.publish("testEvent", "", PRIVATE);
            }
        }
        else {
            Log.info("lost cloud connection");
            state = CLOUD_DISCONNECT_STATE;
        }
        break;

    case CLOUD_DISCONNECT_STATE:
        Log.info("disconnecting from cloud");
        Particle.disconnect();

        Log.info("disconnecting from cellular");
        Cellular.disconnect();
        delay(2000);

        Log.info("turning cellular off");
        Cellular.off();

        state = RETRY_WAIT_STATE;
        stateTime = millis();
        break;

    case RETRY_WAIT_STATE:
        if (millis() - stateTime >= RETRY_CONNECT_DELAY_MS) {
            state = CELLULAR_CONNECT_STATE;
            break;
        }
    }

}
4 Likes

This bug was fixed on firmware 0.7.0? Thanks

@rickkas7 do you know if this bug was ever fixed. I am currently running v0.8.0rc11 and notice this issue too when I was traveling I went through some bad signal areas and observed the same issue as above.

My code doesnt include the fix you mentioned as I just discovered it.

I’ve been doing some testing with the Eseries E402 module and I’ve been simulating dropped Cellular service by placing the device into an RF blocking bag. After a little bit the connection drops from Partilce and after I remove the device from the bag the device shows that Particle.connected = false and Cellular.connected = true.

The device is not reconnecting to Particle even while calling Particle.connect.

A system.reset call gets the device connected again.

Have you tried Particle.disconnect() before Particle.connect() (with some delay in between)?
If not, can you try?