Electron froze during cellular service outage

Troubleshooting
1

On Tuesday, there was a service outage and most of my Electrons were affected. I own 100+ Electrons and only one of them failed to connect when the service was restored.

When I checked the Electron, the RGB LED on the Electron was solid white. This is obviously caused by a resource deadlock. I was able to restore by resetting it.

I am operating my Electron in SEMI_AUTOMATIC mode. Since the RGB LED was white, my assumption is that the deadlock probably happened upon reset and before Particle.connect() is called in setup().

My setup function is simple enough and it involves mostly initializing timers, registering cloud functions etc.

  pinMode(MUX_RESET_PIN, OUTPUT);
  MUX(OFF);

  setCharging(false);

  Serial.begin(9600);
  Wire.begin();

  EEPROM.get(0, sampling_period);

  if(sampling_period == 0xFFFF){
    sampling_period = 60;
    EEPROM.put(0, sampling_period);
  }

  sample_timer = new Timer(sampling_period * 60000, sensor_trigger);
  ss_timer = new Timer(10000, strength_trigger);

  Particle.function("cmd", command_handler);

  sample_timer->start();
  ss_timer->start();

  reset_flag = true;
  trigger_flag = true;

  Particle.connect();

  strip.begin();
  strip.show(); // Initialize all pixels to 'off'

  delay(2000);

I am not able to reproduce the problem and this is the first time I have noticed this error. From what I understand, some of the shared resources between application code and system firmware include cellular radio and serial comms etc.

Since I am using timers, I only set flags and call Particle.publish() from the main loop().

The only resolution I could think is to introduce a delay so that the Electron doesn’t immediately attempt a connection upon reset but wait for two seconds or so. My other idea is to have an external hardware watchdog that triggers a reset of the Electron.

Am I missing something glaringly obvious here?

2

I would love to hear how you have put 100 Electrons to use :slight_smile:

It’s always interesting to hear about all the different uses.

1 Like
3

This doesn’t really pertain to your issue, but it’s recommended that you declare functions and variables “first” in setup(). You don’t have any explicit delays before your function declaration that could hose things up, but it’s just “best practice” to establish them first thing. (BTW I only learned about this today :face_with_raised_eyebrow:.)

cheers,
ParticleD

3 Likes
4

This assumption may be leading you down the wrong track.
I could imagine that the deadlock may have happened as direct consequence of the cellular outage during the auto-reconnect when the device realised the missing network and/or - if you take matters in your own hand - when your own code tried to deal with the lack of connectivity.
So I'd not just focus on setup() as the potential area of failure.

1 Like
5

Thanks for the insight. My device's main function is to read sensor data and publish once every hour. Since I am operating the device in semi-automatic mode, I don't read the sensor data when Particle.connected() returns false. I ignore the timer triggered flags.

The only activity that happens while the device is not connected to the cloud is a 5ms delay and a function call to obtain the RSSI information. I have since changed this to obtain RSSI when Particle.connected() returns true.

I will try check if there are offending function calls in my code. I have also unplugged the antenna on an Electron to see if I could reproduce the problem.

6

I am building a remote monitoring solution for industrial gases.

7

Thanks for the info!

@Gasgen is remote monitoring Nitrogen gas for machines he builds.

8

Howdy @sai_y,
As @RWB said I am building flow meter data loggers for my nitrogen generators. What kind of monitoring are you building? Always interested in fellow particle users especially if we work in the same industry. Hope all goes well for you and definitely keep us informed as to what solutions you discover to prevent the issue from duplicating itself. I have learned all that I know from this board and the many fine people that have already posted on this thread.

Cheers and happy coding.

Tom