Photon - no lights after hours of operation

Troubleshooting
1

I have weather base station that used to run a Photon with 0.6.3 and an RFM69 radio. It ran for months without a single hitch running 0.6.3. Since the upgrade to 0.7.0, it has locked up twice in the last week. There is an LCD screen that alternates two lines of output every 5 seconds. Both times that the device locked up, the screen was not rotating the text. The weird thing was that when I lifted up the case there were no lights showing on the Photon.

I do have a watchdog timer set:

ApplicationWatchdog wd(60000, System.reset);

I don’t want to blame 0.7.0 yet, but it certainly seems funny that the device is locking up when it ran for so long before upgrading. I was thinking of downgrading to 0.6.3 but before doing that I wanted to tap into the collective knowledge of everyone here.

If there are specific questions I can answer about the setup please let me know. Here is a quick description of the hw/sw:

// Photon firmware: 0.7.0
// RFM69W radio
// A0 - DHT11 temp/humidity sensor
// D0 - I2C SDA 0x27 - LCD backpack 16x2
// D1 - I2C SCL - LCD backpack
// 4.7K p/u resistors to 3.3v
//
// Libraries:
// RFM69-Particle (0.0.2)
// Adafruit_DHT (0.0.2)
// LiquidCrystal_I2C_Spark (1.1.0)

2

One thing here is that System.reset() now needs a bigger stack for the AWD, otherwise you may end up in an SOS panic once the WD triggers.
https://github.com/particle-iot/firmware/issues/1513

And second the AWD does only provide "virtual" insurance against hanging code.
When you call delay() in non-threaded mode or Particle.process() the AWD will be checked-in and hence it won't fire.

Also it's possible that your free RAM got reduced with 0.7.0 just tipping your project over the edge in some cases which didn't happen with 0.6.3.
If you have a 0.6.3 device and a 0.7.0 you could check the stats side by side either by adding a System.freeMemory() readout or by adding SerialLogHandler traceHandler(LOG_LEVEL_ALL) and take the reading provided by the system.

3

Thanks for the reminder on the AWD stack size. I remember reading that when it came out but forgot to follow up. For what it’s worth, here is the output from the compile regarding memory:

29%20AM
29%20AM818Ă—244 18.7 KB

As you suggested, I also put the System.freeMemory() statements at the end of setup() and also in the routine that processes events on the hour.

Any thoughts on the absence of any lights on the Photon?

4

Just booted after making the AWD change and adding memory print. After setup() finishes I have 48,222 left.

I should also add that just yesterday I finished converting all Strings to char []'s.

5

Seems the memory issue may be not the cause - pitty tho’

These sporadic issues are a PITA to debug.
I’m not aware of any systematic problems that would explain your symptoms with 0.7.0 once it’s got installed correctly, so we may need more info to reproduce the issue.

6

FWIW, I’m logging to a file with date/time stamps so if it locks up again I’ll at least see what it was doing.

If there is any other info I can provide to help the debug process, I’ll be happy to provide whatever I can.

2 Likes
7

@ScruffR I see that the 0.8.0 firmware is out now, did the hardware watchdog feature get added back into this firmware as you were hoping?

8

0.8.0-rc.2 is out a while already, but it’s not in there and I doubt it’ll be in 0.8.0-rc.3 either.

9

Well, I started it at 5:41am EDT, Sunday morning and it crashed again at 5:12am EDT this morning. The last free memory reading at 5am was 49,168. There are no error messages of any kind in the logs. The last SRR was “Code Flash”, although I know I didn’t flash code at 5:12 this morning.

The interesting thing is that everything rebooted without hanging up. So, does that mean that at least the AWD fired after 60 seconds and didn’t lock things up since increasing the stack space? If that’s true, at least the cure worked but that doesn’t answer why the device reboots randomly after upgrading to 0.7.0.