Hard fault when connecting to Particle Cloud and serial problems with new and old firmware apps and tinker

So, today my Electron developed a strange problem. I have spent the past 6 hours trying to remedy it to no avail.

Beginning of the day: Electron 2G with firmware version 0.5.3 running.

My Electron started experiencing a “hard fault” (one red blink after an SOS) while trying to connect to particle server with my application code running (was working fine this morning). It resets after showing the SOS message, and proceeds to encounter another hard fault next time it tries to connect to the Particle Cloud. This loop repeats itself indefinitely.

I tried entering Listening Mode, but that didn’t work (no response from Electron).

I successfuly entered DFU mode, and issued “particle update” and “particle flash --usb tinker”.

This resulted in the same SOS loop again.

Manually downloaded tinker from the Web IDE (after checking that the firmware build it was targeting was correct), and manually flashed it using DFU utility. Same symptoms.

Added SYSTEM_MODE(SEMI_AUTOMATIC); to the top of the Tinker file so that the Electron wouldn’t try to connect to the Particle cloud.

This cleared the hard fault condition, and allowed me to enter Listening Mode.

While Electron is in Listening Mode, I am able to use Particle-CLI to issue particle serial list, which shows the Electron as a serial device.

I then issued particle flash --serial tinker.bin and was successfully able to re-flash the same (currently running) application to the Electron.

Put the device back in Listening Mode and issue particle serial identify which results in the blue blinking on the Electron to be suspended. After a few seconds the CLI command returns ! serial: Serial timed out. Same thing happens if I issue particle identify (without “serial”).

Exasperated, I tried simply flashing an empty program that blinks the on-board LED on and off with no Particle Cloud connection. Same result as with the tinker app (can’t use particle identify or particle serial identify in Listening Mode).

It would appear that no matter how complex the app is that I upload to the Electron, it will work without a hard fault unless it tries to connect to the Particle Cloud. I have verified this by running an SDFAT example program that outputs to the Serial monitor (all is good as long as I include SYSTEM_MODE(SEMI_AUTOMATIC); at the beginning.

I kept trying everything I could think of and one time randomly the Electron successfully connected to the particle cloud without encountering a hard fault while it was running Tinker. A minute later it encountered a hard fault and resumed its cranky behavior.

Even more exasperated, I then tried rolling back to 0.5.2 using the dfu utility to revert to the older firmware (part 1 and part 2 dfu upload method). Same exact symptoms as before (btw yes I ensured I was now building the apps for 0.5.2).

I then tried to uninstall and re-install the entire particle-cli, node, npm, dfu utility, windows driver, etc etc etc (more time wasted).

I even took a crack at using the Particle Keys Doctor via the Particle CLI as per the instructions here (by the way, the “guide” on this page doesn’t mention that you need openSSL… this would definitely be a good oversight to fix!!!), but unfortunately was unable to get anywhere with that because the command prompt complains that it doesn’t know what “openSSL” is (even though I installed it). I chased my tail trying to get that to work for a while, but couldn’t figure out how to get it to work

At this point, I have reverted the system firmware version to 0.5.3 via the dfu utility and flashed the Electron with Tinker both via CLI (particle flash --usb tinker) and by manually downloading the binary from the Web IDE.

At the present moment (after hours of trying different things and pouring over forum posts) it would seem that I have done all that can be done from a troubleshooting perspective. Is there some way for me to get more information about the nature of the issue I’m facing (i.e. get the Electron to output debug messages while it is connecting to the cloud?). If not, should I just double down and buy a new Electron and hope for the best?

This is very frustrating as my company has a deadline quickly approaching and I really needed to get some app development done today but couldn’t as a result of this ongoing Particle SNAFU.

FYI here is the CLI output when I re-upped to 0.5.3 and flashed tinker:

C:\Users\Tom\Dropbox\Jaza Energy\Technical\Electron\Old Firmware>particle update

> Your device is ready for a system update.
> This process should take about 30 seconds. Here goes!

! System firmware update successfully completed!

> Your device should now restart automatically.
> You may need to re-flash your application to the device.


C:\Users\Tom\Dropbox\Jaza Energy\Technical\Electron\Old Firmware>particle flash --usb tinker
Found DFU device 2b04:d00a
checking file C:\Users\Tom\AppData\Roaming\npm\node_modules\particle-cli\binaries\electron_tinker.bin
spawning dfu-util -d 2b04:d00a -a 0 -i 0 -s 0x08080000:leave -D C:\Users\Tom\AppData\Roaming\npm\node_modules\particle-cli\binaries\electron_tinker.bin
dfu-util 0.8

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2014 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to dfu-util@lists.gnumonks.org

Opening DFU capable USB device...
ID 2b04:d00a
Run-time device DFU version 011a
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 4096
DfuSe interface name: "Internal Flash   "
Downloading to address = 0x08080000, size = 5776

Download        [                         ]   0%            0 bytes
Download        [                         ]   0%            0 bytes
Download        [=================        ]  70%         4096 bytes
Download        [=========================] 100%         5776 bytes
Download done.
File downloaded successfully
Invalid DFU suffix signature
A valid DFU suffix will be required in a future dfu-util release!!!

Flash success!

@jaza_tom,

Recently, I had a similar problem (hard fault) with one of my Electron. After many unsuccessful test and reflash, I removed the SIM card and clean it and put it back. I the flashed firmware 0.6.0rc2. Everything returned to normal. Maybe I was just lucky. I hope it help.

1 Like

Thanks for the response! See below

As of right now it would appear to be an intermittent problem that is affecting me and my colleague who is working on an Electron 3G.

It would randomly connect to the cloud and drop out. I have tinker flashed to it so i should be able to monitor it remotely. It was connected when I left 1/2 hour ago and now is not… as I wrote that it is back again.

I think the documentation surrounding “Hard Fault” and the other SOS codes in general is seriously lacking… “hard fault” has been described in other forum threads as a memory leak, however in this case it is clearly also thrown by a network related error.

I’m guessing that the inability to use particle identify or particle serial identify has something to do with this as well, perhaps because of a blocking function in a core module (trying to deal with network error and thus unable to report the device ID, IMEI info etc to Particle CLI).

Come to think of it, a dirty SIM card could indeed be to blame (can’t read SIM card info so times out in Particle CLI)… I’ll try cleaning it tomorrow @Pleb

Using the usb tinker debug app, I have concluded that I simply don’t have any network to register to. The process gets hung up at the AR+CREG part.

Modem keeps returning +CGREG: 2,0 which means;

2 not registered, but MT is currently searching a new operator to register to
and
0 GSM

according to m2msupport.net