Managing WiFi and Cloud With optional Wifi Support on Device

I’m working on a device which should work whether the Particle Photon has WiFi access or not. It is a consumer product that can optionally be controlled by a mobile device.

As of now I have the Photon in SEMI_AUTOMATIC mode so that I can only connect to WiFi if there are credentials. In my setup I have:

WiFi.connect(WIFI_CONNECT_SKIP_LISTEN);

I have that so that it will connect to WiFi if there are credentials and if it fails it will not go into Listening mode. This allows my customer to control the device without a WiFi connection.

In my loop I have:

if (WiFi.ready() && !Particle.connected()) {
   Particle.connect();
}

This is so that if there is a WiFi connection (success from setup and the device is not connected to the particle cloud it will connect)

The device would obviously ship with no WiFi credentials. With it being in SEMI_AUTOMATIC mode and the WiFi.connect(WIFI_CONNECT_SKIP_LISTEN) in setup the user should be able to turn on the device and just use it with the buttons provided on the device. Am I missing something there or that is the correct approach?

I have a button on my device that when pressed for 10 seconds, it clears all WiFi credentials, and puts the device in Listening mode. This allows me to use the SoftAP to then provide the credentials to the device. Once it gets credentials the Photon automatically tries to connect to the internet. The loop would then detect the WiFi is ready and not connected to the Particle Cloud and then connect.

Hoping someone can validate this approach.

Yup, several things :wink:

There is a more straight forward way that's actually intended to be used - rather than that workaround: Check WiFi.hasCredentials() and only try WiFi.connect() when that returns true otherwise proceed with your offline jobs.

You may also want to use SYSTEM_THREAD(ENABLED) to decouple your code from the system tasks.

You can also use WiFi.scan() in conjunction with WiFi.getCredentials() to first check whether there is a known network in reach before trying to connect.

You should also actively disconnect once your code learns that it's not connected anymore to prevent the system from wasting time trying to reconnect when the network has gone out of reach.

That would be the default behaviour of the SETUP/MODE button.

1 Like

You may also want to use SYSTEM_THREAD(ENABLED) to decouple your code from the system tasks.

Yup, this is already done.

There is a more straight forward way that’s actually intended to be used - rather than that workaround: Check WiFi.hasCredentials() and only try WiFi.connect() when that returns true otherwise proceed with your offline jobs.

I actually had this before but saw the connect skip listen and changed to that. I'll put it back to the other way. I like the idea of the WiFi.scan as well... great tip.

You should also actively disconnect once your code learns that it’s not connected anymore to prevent the system from wasting time trying to reconnect when the network has gone out of reach.

I'll build some type of max retries on reconnecting and then call WiFi.disconnect if it reaches the limit. I am guessing I would then also need some sort of timer that would check !WiFi.ready() && WiFi.hasCredential() along with network being available to try to reconnect again?

That would be the default behaviour of the SETUP/MODE button.

Got it. So this should be fine then. I can't use the button mirror option b/c the button I'm using is used for the features of the device so I'm doing the long click approach to enter listening mode.

I hate to sound like an idiot but:

Yup, several things :wink:

Was this in reference to the things you addressed or some other larger issues not addressed. Its early so I probably am not picking up on the jest. Thanks so much for your reply as well. Great tips!

2 Likes

Nope, nothing major. It was just meant as a "juicy" opening statement and direct answer to a direct question - sorry, if that didn't fly well :blush:

1 Like

LOL no it was totally fine… I wanted to make sure I wasn’t a complete moron and you just didn’t want to dive into all the other details. Thanks again for all the tips.

1 Like

@ScruffR - hoping you can help me on this one:

So I’m having an issue with

WiFi.scan()

I have the following:

WiFiAccessPoint aps[20];
Log.info("Scan for networks");
int found = WiFi.scan(aps, 20);
Log.info("Found %d networks", found);

I get 0 for found networks. However, right after this I do a WiFi.connect() and it connects to my SSID. The SSIDs are not hidden.

Have you switched the WiFi module on before scanning? :wink:
https://docs.particle.io/reference/device-os/firmware/photon/#on-

When starting off in SYSTEM_MODE(SEMI_AUTOMATIC) or SYSTEM_MODE(MANUAL) the WiFi module is off (as it isn’t needed), so it won’t be able to do anything unless powered up.

Yup… I’ll add a delay to see if that helps.

BTW, what device OS version are you targeting?

Added a delay of 5 seconds after calling WiFi.on and it works now. My Photon currently has 1.2.1 on it.

@ScruffR

Hoping I can tap your brain again! As of right now what I have will only connect to WiFi in setup if there are credentials and an available network with the SSID from the stored credentials. If these aren’t true, it will not attempt to connect to WiFi. It starts the wifiWatchDog. My goal was this watchdog is checking for reconnecting if it can’t connect on setup but also if the connection is dropped after setup.

I tried these tests:

  1. Can’t get WiFi connection on startup. It correctly does not try to connect b/c the WiFi network is not available even though there are creds. The timer kicks off and when WiFi becomes available it should connect via my timer. This does not work b/c when the timer kicks off 10 seconds after setup (short time for testing) the Photon blinks red (SOS with 1 blink for hard fault) at line WiFiAccessPoint credAP[1];. This function works fine when called in setup but pukes when called via the timer at that point. Can this code not be done in the context of a timer?

  2. WiFi is lost while running. My idea was the wifi timer would reconnect in this case for me as well b/c it would lose connectivity… timer would recognize its not ready, has creds, and network is available and do the reconnection. My problem is that when WiFi is lost while running, the Photon immediately tries to reconnect on its own indefinitely. I can’t find a place to disable this behavior. I was trying to build to your suggestion of attempting a reconnection but eventually stop trying but can’t get that far b/c I can’t get the Photon to stop trying to reconnect on its own and b/c of #1 above.

Here are snippets of my firmware:

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);
Timer wifiWatchDogTimer(10000, retryWifiConnection);
bool networkFromCredsAvailable(){
    bool foundNetwork = false;
    
    WiFiAccessPoint credAP[1];
    int found = WiFi.getCredentials(credAP, 1);
    char* ssidFromCreds = credAP[0].ssid;
    
    Log.info("Need to find SSID from credentials: %s", ssidFromCreds);
    
    WiFiAccessPoint availableAPS[20];
    Log.info("Scanning for networks");
    int foundAvailableAps = WiFi.scan(availableAPS, 20);
    Log.info("Found %d available networks", foundAvailableAps);
    
    for (int i=0; i<foundAvailableAps; i++) {
        if(!foundNetwork && strcmp(availableAPS[i].ssid, ssidFromCreds) == 0){
            Log.info("Found network SSID from credentials %s: ", ssidFromCreds);
            foundNetwork = true;
        }
    }
    
    return foundNetwork;
}

void retryWifiConnection(){
    if(!WiFi.ready() && !WiFi.connecting()){
        Log.info("Attempting wifi reconnection test");
        
        if(WiFi.hasCredentials() && networkFromCredsAvailable()){
            Log.info("Network found and can try to connect");
            WiFi.connect();
        }
    }
}
void setup(){
    Serial.begin(9600);
    waitFor(Serial.isConnected, 10000);
    delay(2000);
    
    Log.info("System version: %s", System.version().c_str());
    Log.info("Device id: %s", System.deviceID().c_str());
    
    if(WiFi.hasCredentials()){
        WiFi.on();
        delay(1000);

        if(networkFromCredsAvailable()){
            WiFi.connect();    
        } else {
            Log.info("Did not find a matching network so not going to try to connect to WiFi");
        }
    }
    
    wifiWatchDogTimer.start();
}
void loop(){
    if (!Particle.connected() && WiFi.ready()) {
        Log.info("Particle not connected... Starting connection");
        Particle.function("func1", remoteFunc1);
        Particle.function("func2", remoteFunc2);
        Particle.connect();
    }
    delay(5)
}

I put the Particle.function calls in my loop here b/c I wasn’t sure what would happen on setup if there is no WiFi and they are in there. Do they queue up and will send whenever the particle connects? Are they safe to stay in the Setup even if there is no WiFi on boot?

You need to do a bit of reading of the documentation!

A bit of searching of the forum topics would also highlight the (long) discussions that have gone on before.

Some quick observations and advice;

SYSTEM_THREAD(ENABLED);
SYSTEM_MODE(SEMI_AUTOMATIC);

Check for Particle.connected() before any Particle.publish() or other cloud call like syncTime. If not connected to the cloud then these are blocking the application thread even with system thread enabled.

The best way to handle the wifi credentials is to only store one (and not five). The reason for this is that when connecting it will try each credential in turn until it can connect to one and this can take 7-8 seconds per credential - so that’s your application blocked for 30-40 seconds. Also, the ‘simple’ logic described below becomes too complicated.

Only ever call Particle.connect once.

I hold a couple of global bool variables; wasConnected - did the device have a wifi connection before, isWAPoutOfRange, isBadCredentials. Using these flags and testing for number of credentials stored and WiFi.ready it is possible cover all the scenarios and then decide whether to not connect or retry connection.

BTW, Particle.function() needs to be in setup() and fairly soon after the startup (i.e. at the start) and not in the loop().

1 Like

I am reading the documentation. Please don't assume I'm not and be crass. I am also searching the forums as well as the internet in general when I have a question before posting.

I don't do any Particle.publish calls in my code and I am checking if Particle is !connected before trying to connect. The docs say that once Particle.connect is called it will not call the loop again until it is complete. I do need to add a globalVar to manage the state that I have called Particle.connect b/c the docs say it will NOT call the loop again until its complete but a log statement shows its called numerous times.

I'm not storing 5 sets of creds. I store 1. The user enters Listening Mode via holding a button on my device (not mode button) for 10 seconds. It clears the credentials and when they provide creds it stores them. Each time the device enters listening mode it clears the credentials assuring I only ever have 1 set of credentials.

I also read the docs about Particle.function being in the setup but also read something about it only sending 30 seconds after connecting etc. I must have just misunderstood this section. I'll put those calls back into the setup.

When using SYSTEM_THREAD(ENABLED) you must be careful of when you register your functions. At the beginning of setup(), before you do any lengthy operations, delays, or things like waiting for a key press, is best. The reason is that variable and function registrations are only sent up once, about 30 seconds after connecting to the cloud. Calling Particle.function after the registration information has been sent does not re-send the request and the function will not work.

I hold a couple of global bool variables; wasConnected - did the device have a wifi connection before, isWAPoutOfRange, isBadCredentials. Using these flags and testing for number of credentials stored and WiFi.ready it is possible cover all the scenarios and then decide whether to not connect or retry connection

Great, how are you keeping the Photon from automatically retrying to connect if your connection fails after a successful connection was made. I can't find this anywhere. If my Photon is connected and I turn off my AP it will on its own begin trying to reconnect. How are you preventing that?

1 Like

What kind of product is this if you don’t mind?

It is a product that should function with or without wifi. It has a companion mobile application that can be used to control it but is optional.

My primary goal is to allow the user to operate their product via its physical buttons or via their mobile app. Because wifi is optional I am running in semi automatic and with system thread enabled which allows them to interact with the physical buttons even at first startup.

I want the user to have the option to connect their device. To do that they press and hold a physical button on the device and it enters listening mode. They will then use the softAP approach to provide credentials to the device and claim it. It will then show in their app for them to control it.

As of now in setup it will attempt to connect to wifi if there are credentials stored and the ssid stored with those credentials match an available network scan. The device is moveable so its possible the user is running it somewhere else so there would not be an available network with credentials stored so no need to try to connect to wifi.

My prev approach was just this in setup:

WiFi.connect(WIFI_CONNECT_SKIP_LISTEN);

And in my loop:

    if (!Particle.connected() && WiFi.ready()) {
        Particle.connect();
    }

That would try to connect to wifi if creds were available and if it failed it would skip going into Listening mode. Doing this allows the user to just use the physical buttons and still operate the device. However with this approach it won’t try to reconnect later if the AP comes available. I went down the path to break this out per the suggestion of @ScruffR so that I now check in setup that the user has creds and there is an available network with the ssid of their creds. This works just fine but again if it doesn’t do the WiFi.connect in setup (no creds or no available ap) it won’t try again which is why I added the timer to try the reconnection.

One of my general issues is that if it gets a WiFi connection from setup… and then loses it while running… the Photon will automatically begin retrying to connect indefinitely. I was trying to prevent that or at least limit its attempts per the suggestion of @ScruffR but I can’t get the Photon to not auto reconnect in this case .

I’ve tried to boil down one of my issues to a simple test case. The setup here calls the function and you can see it print the result out to the serial logger. However, 10 seconds pass when the timer tries to run the same function and it throws a SOS. Can’t understand why this fails.

SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);


SerialLogHandler logHandler;

Timer credsTest(10000, tryToGetCreds);

void tryToGetCreds(){
    WiFiAccessPoint credAP[1];
    int found = WiFi.getCredentials(credAP, 1);
    Log.info("GOT HERE");
}

void setup(){
    Serial.begin(9600);
    waitFor(Serial.isConnected, 10000);
    delay(2000);
    
    tryToGetCreds();
    credsTest.start();
}

void loop(){
}

I think I am going to leave what I have for now. Going to outline below in case others are reading and have similar needs.

My current use cases:

  1. There are no stored credentials. Device doesn’t try to connect and does not go into listening mode. This would occur if the user has never setup their device with their smart phone or they initiated setup which clears the credentials and the device prematurely exited the process before the user finished. They could still use the device and would initiate our setup mode to configure their credentials to have the device attempt to connect.

  2. There are stored credentials and an AP is available with a matching SSID. It will attempt to connect. If the connection fails for some reason (like bad credentials etc) the Photon will not go in listening mode so the user can continue to use the device and via its physical buttons. I just tested this and had to put the WIFI_CONNECT_SKIP_LISTEN back in the connect call b/c a failed connection attempt makes the Photon go into listen mode. I never want it to go into listening mode without the user asking it to. User would see the device is not connected in their app, access troubleshooting and eventually re-initiate setup to provide new credentials or select a new AP etc.

  3. There are credentials and no available matching AP. It will not attempt to connect so the user can use the device via its buttons. This scenario would happen if they moved the device or their wifi is down. User would see the device is not connected in their app, access troubleshooting and possibly re-initiate setup to provide new credentials/select different AP or fix their wifi etc.

  4. There are credentials, an available AP, and connection is successful. The connection is lost. The Photon will already try to reconnect indefinitely. I was going to try to optimize this to a max attempt or some other limiter but it seems it can’t be done and may just not be worth the worry. I’ll just have to be good with this.

  5. If for any reason the device does not connect at setup (no credentials, no matching AP, failing connection attempt) the device will not attempt to reconnect. I’m still going to try to work through this one a bit as I’m not a fan of this but worst case this can be part of our troubleshooting that they need to power cycle the device to have it try again or reinitiate setup. My timer approach here is just puking and I haven’t been able to figure out why using my sample test code from my prev reply.

Current Code Snippets (just supplying items as it relates to the WiFi management). I am using ClickButton (not in snippet below) to detect a long click on a button to clear credentials and enter Listening Mode. Tried adding some comments to the snippets to explain why they are there.


SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_THREAD(ENABLED);

//Used to ensure Particle.connect is not called multiple times.  Was finding in loop it was called multiple
//times even though I was using a !connected check and the docs say it doesn't call the loop until its
//done. This just ensures its only called once b/c we set to false right before calling connect so its
//a guard on calling again
bool canCallParticleConnect = true;

//Logger 
SerialLogHandler logHandler;

//This checks the credentials stored ssid has a matching available AP to connect to
bool networkFromCredsAvailable(){
    bool foundNetwork = false;

    WiFiAccessPoint credAP[1];
    int found = WiFi.getCredentials(credAP, 1);
    char* ssidFromCreds = credAP[0].ssid;
    Log.info("Need to find SSID from credentials: %s", ssidFromCreds);

    WiFiAccessPoint availableAPS[20];
    Log.info("Scanning for networks");
    int foundAvailableAps = WiFi.scan(availableAPS, 20);
    Log.info("Found %d available networks", foundAvailableAps);

    for (int i=0; i<foundAvailableAps; i++) {
        if(!foundNetwork && strcmp(availableAPS[i].ssid, ssidFromCreds) == 0){
            Log.info("Found network SSID from credentials %s: ", ssidFromCreds);
            foundNetwork = true;
        }
    }

    return foundNetwork;
}

void setup(){
    Serial.begin(9600);
    waitFor(Serial.isConnected, 10000);
    delay(2000); //Small delay b/c version/device would sometimes get written to serial via logger and not show up.

    Log.info("System version: %s", System.version().c_str());
    Log.info("Device id: %s", System.deviceID().c_str());
    
    if(WiFi.hasCredentials()){
        WiFi.on(); //module needs to be on before you can scan for networks etc
        delay(2000); //added delay b/c no delay always returned 0 networks so it needs time to turn on I suppose

        if(networkFromCredsAvailable()){
            WiFi.connect(WIFI_CONNECT_SKIP_LISTEN); //If connection fails for any reason do not go into listening mode
        } else {
            Log.info("Did not find a matching network so not going to try to connect to WiFi");
        }
    } else {
        Log.info("Did not find any credentials so not going to try to connect to WiFi");
    }
}

void loop(){
    //canCallParticleConnect guards us here from multiple calls to connect. Without this we see multiple 
   //log outputs of trying to connect even though loop isn't supposed to be called after connect until
   //connect is done. If you remove the guard here you'll see multiple log entries
    if (canCallParticleConnect && !Particle.connected() && WiFi.ready()) {
        Log.info("Particle not connected... Starting connection");
        canCallParticleConnect = false;
        Particle.connect();
    }

    delay(5);
}

Thanks for sharing the code.

I’m sure others building a product based on a Photon could find this useful.

Before clarifying some of the "sideshow" topics I want to address your immediate issues

That's most likely due to the fact that Software Timers run on a separate thread with very a limitted stack quota. Henc you should not call any "elaborate" functions and need to consider the call depth of your function and its functon calls.
The typical way to go about this would be to keep the hard work in the application thread (aka loop()) and essentially let the timer callback only set a flag to tell loop() that it's time to do something (some conditional checks and brief precessing is fine tho').

I'd have to double check, but for this scenario I'd rather opt for SYSTEM_MODE(MANUAL) in conjunction with SYSTEM_THREAD(ENABLED) to keep the system from getting "opinionated" about the need to reconnect.

Now on to the "sideshow" topics raised:

While it is true, that Particle.function()s (same as Particle.variable() and Particle.subscribe()) need to be registered the latest a few seconds after a connection is established but there is absolultely no need to hold back from registering them before a connection is made.
The above function calls only tell the device which entities should be registered with the cloud once the cloud connection becomes available. So doing it at the earliest possible point in time (e.g. setup()) is the safest bet.

The reason that such nuances may be missing in the docs is probably owed to the fact that the majority of the reference docs stem from a time before non-AUTOMATIC system modes were introduced or widely used and with AUTOMATIC mode there only is a time after the cloud connection was established because user code won't be executed without it :wink:

As a safe guard, you could setup a timer that checks for WiFi.listening() and if your application currently wouldn't agree with that (e.g. by means of a dedicated flag) WiFi.listen(false) can be called to end Listening Mode.

In order to get earliest info about loss of connection you could register a System.on() event handler to be informed ASAP.

I'd rather wirte this

like that

    WiFiAccessPoint credAP;
    if (WiFi.getCredentials(&credAP, 1)) 
      Log.info("Need to find SSID from credentials: %s", (const char*)credAP.ssid);
    else
      return false;

For one, this makes it clear that you are only interested in a single set of credentials and also ensures that you are dealing with the correct string pointer.
Without checking, WiFiAccessPoint::ssid may well be a String object and requesting a pointer to it may not actually give you the pointer to the string buffer but to the object itself.
So - since I tend to be too lazy to look it up - I'd opt for the safe bet which will work either way (that's also why I prefer using (const char*)someString over someString.c_str() as the former works for both - C strings and String - while the latter only works with String objects :wink: )
And finally, you should really check whether you actually got back a set of creds before trying to access the WiFiAccessPoint fields.

Also when traversing your availableAPS you may want to break the loop once you found your desired AP.

BTW, scanning WiFi networks may take some time, so it shouldn't be done too frequently.

Finally, about this

in connection with

You can either follow the Particle.connect() call with a waitFor() or waitUntil() or - if you want your code to keep running during the ongoing connection attempt, you'd set a timeout-guard for the retry (I'd personally not trust Particle.connecting() for that as your might just slip between the change of state and get a false report).

This would be my take on a timeout-guard

const uint32_t msRetryConnect   = 30000; // allow at least 30 sec between retries
      uint32_t msOngoingConnect = 0;
  ...
  // when not connected and last attempt was long gone or it's actually the first
  if (!Particle.connected() && ((millis() - msOngoingConnect > msRetryConnect) || !msOngoingConnect) 
  {
    msOngoingConnect = millis();         // set the timeout base
    Particle.connect();                  // initiate (re)connect
  }
  // carry on with business
  ...
1 Like

Thanks for that info. I wasn't aware of the limited stack on the timers. I can def do the work in a loop and evaluate there if I need to attempt a reconnect using some vars.

I'll also investigate the SYSTEM_MODE(MANUAL). I know I have to do some extra work with that mode like call Particle.process() on my own but not sure what else that entails so I'll dive in on that suggestion to stop the auto wifi reconnect.

While it is true, that Particle.function()s (same as Particle.variable() and Particle.subscribe()) need to be registered the latest a few seconds after a connection is established but there is absolultely no need to hold back from registering them before a connection is made.

Thanks for this clarification. I put the function registration back at the top of setup. I think my use case of the device starting with no wifi available and reconnecting sometime later when it became available made me overthink this piece which is why I put it in the loop later.. its back in setup.

  WiFiAccessPoint credAP;
    if (WiFi.getCredentials(&credAP, 1)) 
      Log.info("Need to find SSID from credentials: %s", (const char*)credAP.ssid);
    else
      return false;

I didn't know that was a possibility. The docs only show passing an array of WiFiAccessPoints to getCredentails and scan. Again, thanks for the insight. Maybe we can get them to update the docs that you can pass a single pointer in vs an array.

So - since I tend to be too lazy to look it up - I’d opt for the safe bet which will work either way (that’s also why I prefer using (const char*)someString over someString.c_str() as the former works for both - C strings and String - while the latter only works with String objects :wink: )

Gotcha! I honestly just followed the docs on that approach assuming the docs were complete. I like the optimization though!

And finally, you should really check whether you actually got back a set of creds before trying to access the WiFiAccessPoint fields

Yes totally agree. My initial plan was to just check if(found) but I like your approach better with passing in the pointer to a single WiFiAccessPoint. Maybe they should add that capability to the docs for others to see as well.

Also when traversing your availableAPS you may want to break the loop once you found your desired AP.

Agreed, was just laziness on my part.

You can either follow the Particle.connect() call with a waitFor() or waitUntil() or - if you want your code to keep running during the ongoing connection attempt, you’d set a timeout-guard for the retry (I’d personally not trust Particle.connecting() for that as your might just slip between the change of state and get a false report).

Docs said it was only about 1 second of wait and I'm ok with that. My issue was that it was actually printing my log 20+ times from my loop after Particle.connect was called. Below is what I had and the "Particle not connected..." line would print many times which means it was not immediately stopping the calls to loop. So I just put a simple guard to prevent the multiple calls to just stop it but I will need to have it retry of course. Thanks for your sample there.

void loop(){
    if (!Particle.connected() && WiFi.ready()) {
        Log.info("Particle not connected... Starting connection");
        canCallParticleConnect = false;
        Particle.connect();
    }
}

Again, thanks so much for the guidance and experience in areas not covered in the docs. I'll make some changes and explore the manual mode once I make some of these other suggestions. Once this is done I may create a new clean post with a sample for others to leverage.