Listening mode on the Photon cannot work reliably in current implementation

The “cash on demand” statement was technically correct!!

Re the button handler - just test it out using the Web IDE (I find that it is the quickest way of performing little experiments like this). Let us know how it goes. Hope it works for you.

By disabling SoftAP, I mean just not including the handler in the STARTUP() macro:

STARTUP(
    Keyboard.begin();    // Allows the HID device attach for the first time after boot 
                         // with *both* Serial and Keyboard

#ifdef SOFTAP_ENABLED
	// Refer https://community.particle.io/t/application-softap-http-pages-issue/22499/4
	// Be sure to initialize the softAP pages in a STARTUP() macro so they
	// are setup *before* the device connects to the internet.
	//
	// If it is initialized in the setup() method, then SoftAP pages
	// won’t be available until the device has connected to the cloud.
	softap_set_application_page_handler(myPage, nullptr);
#endif  // SOFTAP_ENABLED
);

I can’t tell you if performing the above saves on heap memory as I have not bothered checking.

Re the WiFi factory default, we just find it really handy for a number of reasons.

That just overrides the default pages. If you don’t add that handler, softap is still enabled but with the defaults.

Not new at all… I’ve slammed into the AP mode RAM consumption issue since the initial WPAE compatibility integration starting with 0.7.x. I have a large fleet of P-series devices that continue using 0.6.3 as we have an app-based AP mode provisioning process.

1 Like

This is just in. Particle considers this a feature request, not a bug.

:angry:

Well, technically, having your wifi setup process work without having 70% of your memory free could be considered a feature request.
But given how unlikely it is that 70% of memory is free in any serious application, I think assuming that’s the case and just starting the HTTP server with SoftAP, without checking memory, is a bug.

And given how easy it would be to NOT make SOFTAP_HTTP default 1 on the photon, as a first remedy, I think this should be part of the 1.5.1 milestone.

@avtolstoy, did you get a look at this? I find it hard to believe you think this implementation is okay too.

3 Likes

Shall we collect some names of application developers affected by this issue?

The folks at Particle don’t appear to think this is a serious issue. I think @jimini being stuck at 0.6.3 is a telling example of how this bug affects developers of serious applications on the device-os framework.

I feel a bit like a whistle-blower trying to get attention for a major flaw in the framework, but so far I have not gotten one serious reply from Particle that showed me they understood what I am trying to tell them.

1 Like

One of our applications is quite RAM intensive doing a lot of extended 1KHz analog sampling/recording for waveform analytics, and we’ve gotten around this by staying put @ 0.6.3. These devices are WiFi provisioned and softAP mode operation is paramount.

While we don’t feel that hamstrung by not being able to use a DevOS > 0.6.3, we can indeed debate the merit of feature vs. bug (where have I heard this trade-off before?).

If we’re on the SoftAP subject in general, I think it’s another bigger oversight/omission that I can’t surface SoftAP on Gen3 platform (that might be on the release radar).

I think the biggest issue here is not that SoftAP uses a lot or RAM, but that listening mode not only starts SoftAP, but also a http server with 10 pages.
You can probably make your app work by using the simpler TCP protocol over SoftAP. The memory hungry http pages are by no means a requirement for SoftAP to be functional.

The reply I got from particle on GitHub was that SoftAP is part of WICED, so it is not something they can do something about directly, so they are flagging it as a feature request.
This is wrong for 2 reasons:

  • A huge chunk of the memory is allocated by Particle code, not by WICED, and not all of it is essential. It could easily be made optional.
  • If a dependency is in a closed source third party dependency, that doesn’t make it a feature request. It is just a bug in a dependency. It is still your responsibility to do something about this bug. We cannot even access the code. I would be glad to do so otherwise.
4 Likes

SoftAP won’t be arriving on Gen3, at all. (I’ve confirmed this internally)

Sad to hear. That takes our product out of Particle the Particle lineup.

100% agree with this. We currently have 481 photons online (in a 1652 fleet) all on v0.6.2 because when we attempted upgrading to v0.7.0 when it came out, we were getting strange problems when changing WiFi details (via softAP). Our app requires the HTTP server, and works fine out of the box, but as soon as our firmware is installed and the customer tries to change the WiFi credentials in exactly the same way as they did originally, it would fail (and we would get a very frustrated customer).
The major bugbear was that we had no idea why it would do this - everything in our code worked fine in v0.6.2, and almost entirely worked fine in v0.7.0. It was only after a few days of debugging did we figure this issue out - the time was spent checking our own code and assuming Particle side was fine and tested (this version was not a pre-release)!

We have made do with v0.6.2 for the time being, but would really like to gain some advantages of the updated versions. To get round this we have decided to add expandable memory to our board for all future hardware.

I have just come across this thread and would like to give my support for this issue.

My question would be; what happened between v.0.6.2 and v0.7.0 to precipitate this need for memory?

1 Like

I know they introduced support for customizing the HTTP pages, but don’t know when this happend.
If you use the default pages, it should be possible to use a pointer to program memory, which uses no ram at all. But the current implementation first copies all the pages to an array in RAM memory, which I think can be avoided.

Even if the users wants custom pages, I think most of the page content can still be defined as static const or constexpr and never be copied to RAM.

1 Like

Yes a pointer would be ideal, maybe have the ability to stream the data from any source too e.g. external memory.

I think the problems started around v0.7.0 when Particle started supporting WPA/WPA2 Enterprise credentials, which prompted the creation of the WiFiCredentials class.

1 Like

To further a workaround, I have a simpler, alternative solution to @UMD (which was making a setup2 to allocate heap memory only if not in listening mode).

Let me know if you see any “new” downsides to this one (I’m hoping to roll this out to new firmware in our fleet);
When listening mode detected, restart the device and go into listening mode before any initialisations.
Sample code looks like this:

STARTUP(System.enableFeature(FEATURE_RETAINED_MEMORY));
retained uint8_t listenMode = 0;
void setup() {
  if(listenMode){
    listenMode=0;
    WiFi.listen();
  }
  // initialisations and heap memory allocation
}
void loop () {
  if(WiFi.listening()){
    listenMode=1;
    System.reset();
  }
  // do useful stuff here
}
2 Likes

@dan.s, brilliant solution, clever!

Had to review: https://docs.particle.io/reference/device-os/firmware/photon/#enabling-backup-ram-sram- to fully understand that the following line:

retained uint8_t listenMode = 0;

only initialises listenMode on a power cycle, ie NOT on a soft reset.

@Elco, is this a solution for you?

I will test myself and report back.

Thank you @UMD ! Yes that’s right, it persists on soft reset but we’ve also found it persists on power cycle too with a large enough input capacitor.
We use retained memory a lot because our devices need to work offline too and they are useful for resuming the previous state without having to connect online (we also use manual mode and system threads) there is a very good description written by @rickkas7 here: Retained Memory Tips

A caveat though if using system threading, you must remember to add waitUntil(WiFi.ready) after entering listening mode to prevent it ruining our plans to avoid the subsequent heap memory allocation!

Let me know if your tests work and if you see any possible drawbacks.

Oh oh… Am using system threading… To me adding waitUntil(WiFi.ready) does not make sense because it is a Catch 22 - the fact that you are in listening mode is typically because you don’t have a WiFi connection or am I interpreting this requirement incorrectly?

@dan.s, unfortunately I was unsuccessful with my testing, endless reset loops…

I note that I did not add waitUntil(WiFi.ready).

Am pretty sure that my issue with your work around is SYSTEM_THREAD(ENABLED).

Keep up the good work.

If using system threading, you will need to waitUntil(WiFi.ready) before moving on, otherwise the other thread will be free to run the loop and then reset the device before you’ve had chance to do anything!

In our user flowchart, we generally have:

  1. Out of the box -> factory firmware with enough memory to program WiFi credentials normally.
  2. Once connected online, our firmware is downloaded and memory is at a premium. If WiFi stays the same, there is no need to softAP so no problem.
  3. However, if the user wants to change WiFi, this is where the trick comes in. The device reboots, enters listening mode before memory taken up, user inputs WiFi credentials and the WiFi.ready becomes true, therefore moving past the waitUntil statement and into the main code - taking up lots of memory once again.
1 Like

Aha, now understand the need for WiFi.ready()!

This explains the issue I was having.

Will need to change the position of the if (listenMode) block however as access is needed to the SDCARD for my implementation of SoftAP (I store the Javascript there to save code space).

Will report back on Take II…

Yes it should still work - we also initialise the SD library (used for logging) beforehand.