Photon stuck in rapidly blinking cyan

Connectivity WiFi
1

I have two Photons that are stuck in rapidly blinking cyan. I’ve read many of the threads, but still struggling to pin point the cause.

What I’ve tried so far:

  1. reset device keys
  2. re-connect WiFi several times
  3. flashing and reflashing multiple times, including latest firmware 1.5.2

I also used this serial monitor debug + tcp dump on my router to see that the device is receiving a DHCP IP and starting the connection with device.spark.io, but for some reason the Particle.connect() never completes.

Any idea how to further debug the root cause? It happens 90% of the time. If I leave it blinking for hours, it will eventually connect. It used to connect pretty quickly a few weeks ago, but it has been intermittent ever since.

Serial connection closed.  Attempting to reconnect...
Serial monitor opened successfully:
0000005496 [hal.wlan] INFO: Using internal antenna
configured credentials:
ssid=secretSSID security=wpa2 cipher=1
available access points:
<list other SSDIs>
SSID=secretSSID security=wpa2 channel=11 rssi=-51
connecting to WiFi
0000006195 [hal.wlan] INFO: Joining secretSSID
0000006195 [hal.wlan] TRACE: Free RAM connect: 45704
0000007045 [hal.wlan] INFO: Bringing WiFi interface up with DHCP
connected to WiFi!
localIP=192.168.x.11
subnetMask=255.255.255.0
gatewayIP=192.168.x.1
dnsServerIP=192.168.x.1 (often 0.0.0.0)
dhcpServerIP=192.168.x.1 (often 0.0.0.0)
ping gateway=1
ping dnsServerIP=1
ping addr 8.8.8.8=1
device.spark.io=18.215.131.110
0000007204 [wiring] TRACE: TCPClient socket=20009208
connected to device server CoAP (testing connection only)
connecting to cloud
^C

Here’s the raw output of tcpdump.

2

Hi @guiambros -

There seems to have been several similar issues the last couple of days, not sure why. I used to be able to quickly resolve any matters running particle doctor via the CLI but the lately that too presented errors and did not resolve the problem.

After hours of trying, here is the only solution that has worked for me:

Sure, are you using OSx or Windows? I am using OSx do here is is what I did.

  1. Download the four files below form Github. Make sure to download both Part1 and Part2 for photon firmware.
  2. Install and update Particle CLI on you computer
  3. On OSx… open Terminal (or CLI in workbench)
  4. Connect device to you USB port
  5. Make sure devices is in DFU mode (flashing yellow)
  6. In terminal type: particle flash --usb and then simply drag the downloaded files into the terminal (one by one) window. It will automatically enter te file name and path. The device should remain in DFU mode after each flash, unlike Particle Doctor.
  7. Hit Enter. When done, repeat with the next file, in this order:
  • photon-system-part1@1.5.bin
  • photon-system-part2@1.5.bin
  • photon-bootloader@1.5.0+lto.bin
  • photon-tinker@1.5.0.bin

When dragging each file into terminal, you should see something like below.

Screenshot 2020-05-16 at 09.48.33
Screenshot 2020-05-16 at 09.48.331252×772 183 KB

After this the photon should be back to factory settings. You will have to reconnect it to your WiFi using the app or you can do this via CLI as well.

Good luck, hope this helps.

Regards,
Friedl

1 Like
3

Thanks Friedl!

Quick update: I just stepped away from the computer for a couple of hours, and when I came back and Photo was connected. I hit reset around 10 times, and it turned breathing cyan every single time in <5s. Log below from one of these sessions.

Also tried with the other Photon, and same - it connected in < 5s.

This really feels like a registration issue with Particle cloud system. I’ve been keeping an eye on status.particle.io and no incidents, but I’m wondering if there’s something going on that is not widespread outage and doesn’t get detected automatically.

There was a scheduled maintenance today between 11am-12pm PDT, but I was having problems last night and also this morning before 11am PT, so not sure they are related. Interestingly, the RTT graphs on the status page showed abnormal increase last night, so it might be related.

I’ll make a note and follow your instructions next time it gets stuck in blinking cyan again, and see if it helps. Thank you!

best,

Gui

Serial connection closed.  Attempting to reconnect...
Serial monitor opened successfully:
0000005485 [hal.wlan] INFO: Using internal antenna
configured credentials:
ssid=myWifiLegacy security=wpa2 cipher=1
available access points:
SSID=TG1672GD2 security=wpa2 channel=1 rssi=-76
SSID=Esper Internetter security=wpa2 channel=6 rssi=-72
SSID=myWifiLegacy security=wpa2 channel=10 rssi=-76
SSID=myWifi security=wpa2 channel=10 rssi=-76
SSID=myWifiLegacy security=wpa2 channel=11 rssi=-47
SSID= security=wpa2 channel=2 rssi=-42
SSID=BRIS security=wpa2 channel=2 rssi=-73
SSID=BRIS security=wpa2 channel=2 rssi=-43
connecting to WiFi
0000006179 [hal.wlan] INFO: Joining myWifiLegacy
0000006180 [hal.wlan] TRACE: Free RAM connect: 45704
0000007003 [hal.wlan] INFO: Bringing WiFi interface up with DHCP
connected to WiFi!
localIP=192.168.x.11
subnetMask=255.255.255.0
gatewayIP=192.168.x.1
dnsServerIP=192.168.x.1 (often 0.0.0.0)
dhcpServerIP=192.168.x.1 (often 0.0.0.0)
ping gateway=1
ping dnsServerIP=1
ping addr 8.8.8.8=1
device.spark.io=54.86.95.155
0000007594 [wiring] TRACE: TCPClient socket=20009208
connected to device server CoAP (testing connection only)
connecting to cloud
connected to the cloud!
^C
1 Like
4

It was a particle outage. My thread on the topic: Outage! Particle cloud down

2 Likes
5

Hi @guiambros -

Good to hear there problem is resolved!

My apologies, I assumed it was a problem that has persisted for some time :slight_smile: Indeed, rapid flashing Cyan is attempts to connect to Particle Cloud.

For future problems, you can also try the following first and see if this helps.

  • particle update -v or
  • particle flash --usb tinker

IMO... If neither one resolves the problem, save some time on trying to fault find further and simply manually refresh the four files :slight_smile:

Regards,
Friedl.

1 Like
6

All good points, but even easier is turning on a known ‘good’ device and having two devices exhibiting the same behaviour if you have the luxury of multiple.

I simply pulled out one of our products that I’d used the night before and turned it on to see if it matched my desk development devices. That, and about 10 customers who are FB friends messaging me asking why our product isn’t working. The red flash at the end of rapid cyan is either broken cloud TLS keys or a cloud outage and it’s fairly rare for keys to corrupt (though over the years we’ve had about 30 cases, i think the self healing mechanisms are working more effectively now).

1 Like
7

HI @mterrill -

Agreed! I was going with the assumption it is isolated problem on a particular device :wink:

8

Thanks both! I started using Photon not too long ago, and this was the second time I experienced the same behavior in 2 months, so I thought I was doing something wrong, or a defective device. I couldn’t be that unlucky, right?

A few takeaways from this:

  1. (Partial?) Outages don’t trigger an automatic update to status.particle.io
  2. Outages are not that uncommon, per the thread that @mterrill pointed above
  3. Keep a known good device at hand at all times; it helps with differentiating between wifi/cloud outage from device/keys/bad update
2 Likes