Listening mode on the Photon cannot work reliably in current implementation

[Elco]

I think the biggest issue here is not that SoftAP uses a lot or RAM, but that listening mode not only starts SoftAP, but also a http server with 10 pages.
You can probably make your app work by using the simpler TCP protocol over SoftAP. The memory hungry http pages are by no means a requirement for SoftAP to be functional.

The reply I got from particle on GitHub was that SoftAP is part of WICED, so it is not something they can do something about directly, so they are flagging it as a feature request.
This is wrong for 2 reasons:

• A huge chunk of the memory is allocated by Particle code, not by WICED, and not all of it is essential. It could easily be made optional.
• If a dependency is in a closed source third party dependency, that doesn’t make it a feature request. It is just a bug in a dependency. It is still your responsibility to do something about this bug. We cannot even access the code. I would be glad to do so otherwise.

[no1089]

SoftAP won’t be arriving on Gen3, at all. (I’ve confirmed this internally)

[Mjones]

Sad to hear. That takes our product out of Particle the Particle lineup.

[dan.s]

100% agree with this. We currently have 481 photons online (in a 1652 fleet) all on v0.6.2 because when we attempted upgrading to v0.7.0 when it came out, we were getting strange problems when changing WiFi details (via softAP). Our app requires the HTTP server, and works fine out of the box, but as soon as our firmware is installed and the customer tries to change the WiFi credentials in exactly the same way as they did originally, it would fail (and we would get a very frustrated customer).
The major bugbear was that we had no idea why it would do this - everything in our code worked fine in v0.6.2, and almost entirely worked fine in v0.7.0. It was only after a few days of debugging did we figure this issue out - the time was spent checking our own code and assuming Particle side was fine and tested (this version was not a pre-release)!

We have made do with v0.6.2 for the time being, but would really like to gain some advantages of the updated versions. To get round this we have decided to add expandable memory to our board for all future hardware.

I have just come across this thread and would like to give my support for this issue.

My question would be; what happened between v.0.6.2 and v0.7.0 to precipitate this need for memory?

[Elco]

I know they introduced support for customizing the HTTP pages, but don’t know when this happend.
If you use the default pages, it should be possible to use a pointer to program memory, which uses no ram at all. But the current implementation first copies all the pages to an array in RAM memory, which I think can be avoided.

Even if the users wants custom pages, I think most of the page content can still be defined as static const or constexpr and never be copied to RAM.

[dan.s]

Yes a pointer would be ideal, maybe have the ability to stream the data from any source too e.g. external memory.

I think the problems started around v0.7.0 when Particle started supporting WPA/WPA2 Enterprise credentials, which prompted the creation of the WiFiCredentials class.

[dan.s]

To further a workaround, I have a simpler, alternative solution to @UMD (which was making a setup2 to allocate heap memory only if not in listening mode).

Let me know if you see any “new” downsides to this one (I’m hoping to roll this out to new firmware in our fleet);
When listening mode detected, restart the device and go into listening mode before any initialisations.
Sample code looks like this:

STARTUP(System.enableFeature(FEATURE_RETAINED_MEMORY));
retained uint8_t listenMode = 0;
void setup() {
  if(listenMode){
    listenMode=0;
    WiFi.listen();
  }
  // initialisations and heap memory allocation
}
void loop () {
  if(WiFi.listening()){
    listenMode=1;
    System.reset();
  }
  // do useful stuff here
}

[UMD]

@dan.s, brilliant solution, clever!

Had to review: https://docs.particle.io/reference/device-os/firmware/photon/#enabling-backup-ram-sram- to fully understand that the following line:

retained uint8_t listenMode = 0;

only initialises listenMode on a power cycle, ie NOT on a soft reset.

@Elco, is this a solution for you?

I will test myself and report back.

[dan.s]

Thank you @UMD ! Yes that’s right, it persists on soft reset but we’ve also found it persists on power cycle too with a large enough input capacitor.
We use retained memory a lot because our devices need to work offline too and they are useful for resuming the previous state without having to connect online (we also use manual mode and system threads) there is a very good description written by @rickkas7 here: Retained Memory Tips

A caveat though if using system threading, you must remember to add waitUntil(WiFi.ready) after entering listening mode to prevent it ruining our plans to avoid the subsequent heap memory allocation!

Let me know if your tests work and if you see any possible drawbacks.

[UMD]

Oh oh… Am using system threading… To me adding waitUntil(WiFi.ready) does not make sense because it is a Catch 22 - the fact that you are in listening mode is typically because you don’t have a WiFi connection or am I interpreting this requirement incorrectly?

[UMD]

@dan.s, unfortunately I was unsuccessful with my testing, endless reset loops…

I note that I did not add waitUntil(WiFi.ready).

Am pretty sure that my issue with your work around is SYSTEM_THREAD(ENABLED).

Keep up the good work.

[dan.s]

If using system threading, you will need to waitUntil(WiFi.ready) before moving on, otherwise the other thread will be free to run the loop and then reset the device before you’ve had chance to do anything!

In our user flowchart, we generally have:

1. Out of the box -> factory firmware with enough memory to program WiFi credentials normally.
2. Once connected online, our firmware is downloaded and memory is at a premium. If WiFi stays the same, there is no need to softAP so no problem.
3. However, if the user wants to change WiFi, this is where the trick comes in. The device reboots, enters listening mode before memory taken up, user inputs WiFi credentials and the WiFi.ready becomes true, therefore moving past the waitUntil statement and into the main code - taking up lots of memory once again.

[UMD]

Aha, now understand the need for WiFi.ready()!

This explains the issue I was having.

Will need to change the position of the if (listenMode) block however as access is needed to the SDCARD for my implementation of SoftAP (I store the Javascript there to save code space).

Will report back on Take II…

[dan.s]

Yes it should still work - we also initialise the SD library (used for logging) beforehand.