Last night one of my five Xenon’s stopped reporting. It’s blinking cyan with an occasional set of 3 yellow flashes. Same behavior in safe mode. This matches this section of the docs https://docs.particle.io/tutorials/device-os/led/xenon/#red-blink-basic-errors 3 orange blinks: Connected to the internet, but could not reach the Particle Device Cloud. This sometimes is seen as yellow or red and indicates bad server keys.
So I followed the repair instructions listed below that, after upgrading particle-cli. A new key was created and sent to the cloud successfully. It did not fix my issue though. I haven’t tried a factory reset as I’d like to understand why this happened and how to fix without having to go through that process.
This and the other 4 still working Xenons are using system firmware 0.9.0, and I believe seeing the same issue in safe mode rules out the user code. I did power cycle the Argon gateway (also on 0.9.0) a few times but it didn’t help.
A bit of an unusual question, but when you attempt to get this Xenon online, are there other Xenons running on the mesh network when you try to bring this one online? If so, could you try having all other nodes offline and attempt to bring this one onto the network first and see if the issue persists?
Yes, the other 4 were connected but it turns out in the past few hours another Xenon had the same thing happen, blinking cyan + orange flashes. Unlike the original Xenon this one can connect again after resetting it. Though I had already unplugged another Xenon and the original one so only 3 were left at this point. I tried the original one again and it can connect fine now. The 4th Xenon turned out to be frozen, it appeared online but all calls to it fail and it’s status light was solid cyan. I have user code that blinks the D7 led and it was stuck as well so user code was not longer running either. The 5th Xenon was fine through this whole thing.
Not sure if it helps but my Argon gateway and Xenon 4 & 5 are running the marco polo code and have been for weeks. The other 3 are simple battery/temp loggers. I haven’t seen them freeze up like this before, the only recent change that affects them all is upgrading to 0.9.0.
This started happening again to 2 Xenons and I can’t seem to fix it as easy as I used to. Both Xenons and the Argon gateway have been power cycled, to no effect. I’ve tried re-positioning all of them as well, so they aren’t too close or far away from each other.
After exhausting all other options I powered up only 1 Xenon with the Argon and it connected. Then I powered the second one and it worked as well.
…
Just happened again. The main difference this time vs Feb is that I started using sleep, so one of these devices has to re-establish the connection every minute. I’m going to disable that and see if it helps, hopefully the next firmware update can resolve this.
Any idea what causes this and how to prevent it? I just had it happen again, took me a day to remember this thread. Powering off all Xenons then re-powering them one at a time worked. All are using firmware 1.1.0.
Luckily I haven’t seen this happen in a few months, perhaps the more recent firmware solves the issue. But when in that state I’d have to take my other Xenon’s offline until the stuck one could reconnect, potentially after a reset. It seems to be one other Xenon blocking it, so on many occasions it would reconnect before I had turned off all my Xenons.
I’ve been seeing this happen again lately and I found a new but disappointing way to fix it. The issue is almost exclusively happening with my Xenon furthest from the gateway. If I move my Argon so it’s closer to that far Xenon than anything else on the mesh then it’ll reconnect. Perhaps that’s why powering down all other Xenons worked in the past as it guaranteed the gateway was the closest mesh device. If I then move the Argon back to it’s original point then that Xenon will work fine for awhile and likely until the next connection loss or reset. Moving the Argon isn’t always required, in many cases it will eventually reconnect, I’d say 30min is typical. The Argon has a backup battery so moving it doesn’t require/cause a power cycle, so the other 2 Xenons remain connected.
The distance between that far Xenon and the Argon is about 50ft with 3 walls in between; but there are 2 other Xenons in between, roughly 20 and 35ft away with 1 and 2 walls respectively. I’ve since added an external mesh antenna to the Argon and far Xenon but it doesn’t seem that have helped. I’m using the latest firmware, 1.4.4.
Is there a known issue where on startup the device must connect with the gateway directly before being able to relay through other nodes?