Photon WiFi corner condition

Have just had reported a rather obscure scenario of operation that is leaving the Photon with no credentials but LED flashing cyan - trying to cloud connect.

The scenario is this;
Customer is using a SmartPhone hotspot to connect devices from time to time. When the hotspot is not operating they remove the credentials from the device. The devices queue events whilst not connected and when reconnected will dequeue the events.

Customer complained that when reconnecting (reentering credentials) they needed to perform a restart for the connection to work.

Investigation shows that the hotspot was being turned off before the credentials cleared. When the credentials were then cleared (WiFi.clearCredentials()) the result was true however the call took longer than normal 5-6 seconds and afterwards the LED is left flashing cyan and not breathing dark blue. This then prevents scanning for available WAPs or entering credentials and reconnecting.

Calling Particle.disconnect() does not work and Particle.connected() returns false!

Any ideas other than a hard reset of how to get out of this situation programmatically?

Is there a reason to clear the credentials each time? Is it a different phone each time? It's usually easier to just use SEMI_AUTOMATIC mode and have some way of enabling or disabling cloud connectivity so the device will either try to connect or breathe white when not connecting.

In any case, I would probably try to disconnect from Wi-Fi before clearing credentials.

Also you always need to reset a Photon or P1 after setting credentials for it to take effect, so I'm unclear why that would be an issue.

@rickkas7 The reason for clear credentials each time is simply that without a reliable reply on the failed connection due to bad credentials, we have adopted a method of only storing 1 credential. Through use of a number of flags updated through different scenarios of connection we can manage the connection retry in a way that doesn't impact UI response.

Setup is SEMI-AUTOMATIC and THREAD ENABLED.

I am trying to disconnect before clearing credentials and normally (no WAP or WAP in range) this works fine. After the clearing of credentials the WiFi module is powered off and on.

The reset of a Photon has not been required before with the approach used and it is only in the case that the WAP is out of range that we have seen this problem.

As stated - Particle.disconnect() and WiFi.disconnect() do not work under this condition. Even WiFi.off() and WiFi.on() still leaves the LED flashing cyan!

If the device is unable to disconnect, or unable to connect for a period of time, I'd System.reset() the device. I do that normally on the Photon/P1 as it sometimes could fix a failure to connect.

Trying hard to avoid a reset - this is what the customer is complaining about having to do. This is a 'corner condition' then that can only be exited by a system reset. Would you know whether this anything similar with the P2?

The devices are completely different, and I wouldn't expect the behavior to be at all similar.

The Photon 1 uses a ST Microelectronics STM32F205 with what was formerly known as the Broadcom/Cypress BCM43362 Wi-Fi hardware, software, and TCP/IP stack. The Particle cloud connection uses TCP on the Photon 1 and P1.

The Photon 2 uses a Realtek RTL8721DM MCU containing Wi-Fi and BLE hardware and software. The TCP/IP stack (TCP, UDP, DNS) is LwIP and is the same across all Gen 3 and Gen 4 devices because the underlying module (Wi-Fi or cellular) is run in PPP mode so it can send and receive raw frames from our TCP/IP stack. The Particle cloud connection uses UDP, as does every device other than the Photon 1/P1.