WiFi Connection Issues - troubleshooting on site

I am visiting a customer site at the end of the week. They have 8 Photon based products in a room, 2 devices in one part of the room near to a window with a radiator underneath, never manage to connect to the WiFi. There is a WAP in the room with a dedicated 2.4GHz SSID.

To date we have tried swapping devices - however this appears to prove that the devices in this particular area don't connect and when moved somewhere else do connect.

The strength and quality of signal is good across the room and mobile devices (phone) will connect in this apparent black-spot.

I am going to watch the system log of a device trying to connect and I am also going to use Wireshark on my laptop to capture packets whilst connection is being attempted.

Question to the community - is there anything anyone can suggest or recommend or point me towards as a connection diagnostic method to follow?

I presume it's stuck at blinking green? I would flash it with firmware that does not attempt to connect to Wi-Fi or the cloud and instead does a WiFi.scan() and outputs the visible SSIDs and RSSI values to USB serial. Walk around the area with the Photon and laptop and see if anything interesting is printed out.

1 Like

Thanks - flashing green correct. It then will timeout since WiFi connection is blocking and the firmware notes that the stored credentials are bad so that it won't keep trying. We have a screen on the device so that you can see the result of the WiFi Scan - SSID name, signal strength and channel - what sort of funnies do you think this is going to show up?

Originally they had a 5GHz SSID with the same name as the 2.4 GHz one. That issue apparently has been fixed. Could this be a channel overlap issue? I assume simply that the IP address is not being assigned - what specific should I look for in the log?

DHCP occurs during the fast blinking green. It's usually so quick you can't see fast blinking green. The full sequence for Wi-Fi is blinking green (connecting to Wi-Fi AP), to fast blinking green (DHCP), to blinking cyan (connecting to cloud), to fast blinking cyan (cloud authentication) to breathing cyan (connected).

This may be tricky to debug on the Photon 1 because that part of connecting is probably within the WICED code so there won't be much debug logging, except for the eventual failure.

Thanks @rickkas7

@rickkas7 In testing the logging before going to site I notice something in the behaviour I wanted to confirm. Recap - OS 2.3.1 Photon, the firmware was developed going back a long way (started Jan 2017) and on 0.8.X! I think what I am seeing is because of the way SEMI_AUTOMATIC works under 2.3.1 compared to how it did before.
These are the macros:

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);

In setup()

an application function is called which turns on the WiFi module WiFi.on(); delays 100 mSec and checks if any credentials are stored. There is Log output after this and then an if statement (has credentials and no known bad credentials) that if true then calls WiFi.connect();. However, from what I can see from the output the logic never gets to the if statement and returns control to setup() so that my application logic to waitFor(WiFi.ready, timeout); is never called. Indeed, the log output next shows cloud connection started without ever calling Particle.connect();!

Does this make sense? Is WiFi.on() also initiating WiFi connection? And therefore I can't control the connection to be able to determine whether there are bad credentials?

The Photon 1 should not make a connection in SEMI_AUTOMATIC mode if you call WiFi.on(). It should only occur after Particle.connect. I tested a Photon with 2.3.1 and this firmware and the device breathes dark blue, as expected.

#include "Particle.h"

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);

SerialLogHandler logHandler(LOG_LEVEL_INFO);

void setup() {
    waitFor(Serial.isConnected, 10000);

    WiFi.on();
    delay(100);

    WiFiAccessPoint ap[5];
    int found = WiFi.getCredentials(ap, 5);
    for (int i = 0; i < found; i++) {
        Log.info("ssid: %s", ap[i].ssid);
        // security is one of WLAN_SEC_UNSEC, WLAN_SEC_WEP, WLAN_SEC_WPA, WLAN_SEC_WPA2, WLAN_SEC_WPA_ENTERPRISE, WLAN_SEC_WPA2_ENTERPRISE
        Log.info("security: %d", (int) ap[i].security);
        // cipher is one of WLAN_CIPHER_AES, WLAN_CIPHER_TKIP or WLAN_CIPHER_AES_TKIP
        Log.info("cipher: %d", (int) ap[i].cipher);
    }

}

void loop() {
}

A common situation is that the system is missing a required dependency like a bootloader. In this case, the device will go into safe mode to get the dependency OTA, which will cause the device to connect to Wi-Fi. This happens without your firmware running, because your firmware can't run, because it's missing a required dependency. This can be checked using particle serial inspect.

I have now been to the customer site where the problem is.

The photons used to test were all up to date vis bootloader. All the test photons had connected and behaved correctly in testing in the office. Once on site the scan shows the same wherever in the room - a WPA2 enterprise/eduroam network and another WPA2 for the 8 photon. The WAP is on 2 channels 1 and 6. 6 of the 8 devices are connecting to the WAP and easily reconnect if restarted. 2 of the 8 devices will not connect. These photons are not easy to get at and thus the LED status is not visible. A test device with log output same PCB same firmware shows the following sequence at startup - 20 seconds flashing green (there are some hesitations during this) then a short solid white and then breathing dark blue - the connection fails and after this the credentials have been deleted. Does this make anymore sense? I captured the network traffic using Wireshark for both a successful connection (1 of 6) and also the test photon failing. Any advice what to look for in this log?

A test device with log output same PCB same firmware shows the following sequence at startup - 20 seconds flashing green (there are some hesitations during this) then a short solid white and then breathing dark blue - the connection fails and after this the credentials have been deleted

That sounds like a power-related issue. The connecting to Wi-Fi and connecting to cloud state uses the most power. The white blink is a brownout reset. The credentials are getting erased because of the same issue that causes the Dim D7 issue on STM32 devices, except it's erasing the DCT (configuration flash) instead of the bootloader.

OK - thanks - I am surprised that this is a brownout issue - we have done 2000 of this PCB and I have not seen before - the PSU is pretty chunky. I will explore with a DSO and see if I can see it happening.

[Update] - thought about this some more. I will need to explain the problem cause to my non-technical customer. What would explain the reason that some Photons and some carrier boards and some WAPs just happen to cause a brownout? The same board and Photon happily connected in the office to the same make of WAP (Unify). Are we just talking a stack of poor tolerances - higher current requirement on the Photon + poor tolerance of the board power support + a difficult to connect to WAP?

[Update+] I discovered this post that explains the client device will try to match the power output of the AP - do you agree this is what the Photon will do when faced with connection to a high power setting AP? 8 reasons to turn down the transmit power of your Wi-Fi - Metis.fi