SYSTEM_THREAD 0.4.7 Photon

Troubleshooting
1

Hi,

I suspect there is a problem with the new(ish) SYSTEM_THREAD mode.

I have an app which quite happily builds and loads into my target and reboots the target as soon as the new code is loaded. The DEV compiler tells me ‘Success…’

I thought I should try out the multi-task SYSTEM_THREAD mode…

S0…- if I add ‘SYSTEM_THREAD(ENABLED);’ in my primary ino file (right near the top after the #includes) - my app now takes a further 20 seconds plus before it reboots. The old code is still running.

Dev tells me all is OK - but the app doesn’t restart - I noticed because I have a USB serial port and get the drop sound from the driver followed by driver start sound when my Photon reboots.

Sit there and wait for 20 secs or so - and reboot :-O.

NB I have repeated this a dozen or more times and each time I comment or uncomment the SYSTEM_THREAD line the behaviour changes.

I DO use the new software timers, but these work just the same on both compiled versions…

Seems that something is holding back a reboot after upgrade when in multi-tasker mode :-O.

In fact its even more bizarre - as I also define some cloud variables during init, and in SYSTEM_THREAD(ENABLED) mode - these don’t even get defined properly (ie not visible in ‘particle list’

This is the end of my setup code

//********************************************************************
//********************************************************************
  waitUntil(Particle.connected);
  Serial.print("\r\nOK looks like we're connected then...");
//********************************************************************
//********************************************************************
  dbg.Init(56100, controller.getBanner());
  myudp.Init(56200, controller.getBanner());

  Particle.variable("version", ver);
  Particle.variable("currTime", now);
  Particle.variable("freeMem", freeMem);

  //OK so finally start up the system timers
  setupTimers();
  Serial.print("\r\nTimers setup");
}

Hope this is enough detail to reproduce, please ask if not…

I added the waituntil after starting multi-tasking mode - just to be sure as the docs say that setup is run sooner in this mode - before wifi is up…

BR
Graham

2

Sorry, I can’t quite follow what the actual problem here seems to be (got it now, see Edit below)

It’d be always better to see the whole code or at least a stripped down version that at least builds as is.

So I did this and don’t quite see a problem with or without SYSTEM_THREAD(ENABLED).
e.g. the variables get exposed and updated both ways as expected.
What should I be looking out for?

This is my test code

SYSTEM_THREAD(ENABLED)

String ver;
String now;
int freeMem;

void setup() 
{
  Serial.begin(115200);

  waitUntil(Particle.connected);
  Serial.print("\r\nOK looks like we're connected then...");


  Particle.variable("version", ver);
  Particle.variable("currTime", now);
  Particle.variable("freeMem", freeMem);
}



void loop() 
{
    ver = System.version();
    now = Time.timeStr();
    freeMem = System.freeMemory();
    Serial.println(now);
    delay(1000);
}

The main difference in behaviour between the multi threaded version and the single threaded is that with MT enabled now still will be printed while WiFi is off - unlike the ST version.

Edit:
OK, after a few more tries, I think I noticed what you meant, but I would not actually see it as a problem but merely unexpected behaviour.
It seems as if the reflashing of new code will be handled in the background of the currently running code, causing it to take longer to actually finish although the magenta LED already was turned off and the device reverted to breathing cyan and then all out of a sudden the device resets and activates the new firmware.

As said, I’d not see it an immediate problem, but it might cause users to suspect their code not to have taken and so they might retry it without actual need.
I guess this is something @mdma might like to know and maybe he can give some insight about the background of this behaviour and if it will stay like this or will get changed.

1 Like
3

Hi,

OK so - glad that you ‘see’ the behaviour.

I have a simple app now which takes a few seconds after ‘Update starting…’ before it actually reboots, and its nothing to do with SYSTEM_THREAD :open_mouth:

Is it maybe your server is on a ‘go slow’ ???.

Not sure how your flash then restart actually works, but if there could be a delay beween successfully flashing and rebooting - whilst we await a further command from the server, then this would exhibit just like this.

Do you do a server update early on a Saturday maybe ??? - as I ran into ‘other’ problems a week ago :open_mouth: - on a Saturday as I recall…

Anyway - the following code takes a few seconds before it reboots - from here anyway :open_mouth:
OR it DID before I started typing this reply…now it seems OK again

//
#include "application.h"

#define COMPILED_TIME __TIME__
#define COMPILED_DATE __DATE__

//SYSTEM_THREAD(ENABLED);

String ver;

int loopCount = 0;
String now;
int freeMem;

IPAddress myip(255,255,255,255);
unsigned long starttime;
unsigned long endtime;
void setup()
{
  uint8_t* pb = 0;
  ver = String("Sys:" + System.version() + " App:MyApp built:" + COMPILED_DATE + " " + COMPILED_TIME);
  Serial1.begin(57600);
//********************************************************************
//********************************************************************
  waitUntil(Particle.connected);
  Serial1.print("\r\nOK looks like we're connected then...");
//********************************************************************
//********************************************************************
  Particle.variable("version", ver);
  Particle.variable("currTime", now);
  Particle.variable("freeMem", freeMem);
}
void loop()
{
  delay(1000);
  Serial1.print(".");
  loopCount++;
  }

.BR
Graham

4

Just to avoid misunderstandings, Elites - like myself - are no employees of :particle: Particle; we are only enthusiastic Particle users who got granted the privilege to be Admins on this forum and also have some short(ish)-cut communication avenues with Particle staff.

So to get back to your questions above - I don't know :blush:
I doubt it, but this would need to be answered by Particle employees.

5

Hi,

OOPS - Ok sorry this wasn’t clear (well not to me).

You ARE doing a great job though as an excellent ‘enthusiast’. Your feedback is always welcome and hits the nail on the head…

So lets see if we get an employees reply then :-O. It is all pretty stable again at the moment…

Maybe I should start taking Saturdays OFF :-O.

Could be a time-zone effect as its still pretty early (I guess) where ‘Particle live’ - I am in the UK so quite a few hours ahead… its early afternoon here already but still only 9AM plus in the US.

Thanks again for all your most valuable support. Please keep up the great effort…

BR
Graham

1 Like
6

Yes, we Europeans have to keep the world spinning :wink: (I’m from Austria and @mdma is in stationed in Norway and originally from the UK too - I think (sorry if I’m wrong Mat :blush:) )

7

Well I must commend your mastery of English ;-)) beats the cr*p out of my German !!!

Graham

1 Like
8

Another one of those darned "elite" folks chiming in again. I'm not personally aware of any issues last weekend (either Friday, Saturday, or Sunday), but additional resources were added to the build farm yesterday (Friday), so that should help going forward!

Their main offices are in GMT -6 and GMT -8. The East Coast of the USA is GMT -5. We just transitioned from the cursed "daylight savings time" to "standard time" last weekend.

He is also very fluent in C!

1 Like
9

Thanks, you make me :blush: blush - but having an English wife helps, so I’m quite Anglophile :sunglasses:

And I’ve quite a few C-files on my boxes too :stuck_out_tongue_closed_eyes:

10

My wife speaks English, too! :wink:
I have friends who are native English speakers that still can't speak the only language they've ever known.

1 Like
11

You haven’t heard my German tho’ - and writing English is more forgiving than speaking it (or the listeners ;-))

12

Hey we have LOTS of them ;-)) - and ask them to spell things correctly - forget it…

Anyway - as for the start of this thread - the bizarre behaviour continues…

My app is getting quite complex now and has a couple of UDP ports set up, so maybe something is waiting for them - they are ONLY used to send though…

Compile same code - see ‘Update started’ - Photon flashes magenta. My app is still running (monitoring Serial1 interface now = no USB!!). Then 20 seconds or so later, it reboots and starts the the newly built code up again. My app never even misses a heartbeat (printing a dot every second on serial1), until it finally reboots.

Comment out the SYSTEM_THREAD line, rebuild and it has the same delay FIRST time !! (as we are currently running in that mode). Repeat and the next time it reboots immediately :-O.

So - obviously the SYSTEM_THREAD mode is waiting for something on the user thread before it can reboot - or a timeout…

Its no big deal but damned annoying :wink: as you’ve no idea if it did flash OK or not.

Hmmmm - theres a thought…

So I rebuild into a mode where its delayed reboot, then press the reset button. It reboots immediately with the NEW code, so it has successfully flashed the code, but is still running the old code :open_mouth:

All it seems to be waiting for is to actually reboot.

Oh enough time wasted on this for now :-O. Need to get back to real code and wait for ‘Particle’ to comment I guess

Graham

1 Like
13

I read the thread but I’m not following what the problem is. Could someone please describe succinctly what the problem or unexpected behavior is, and the steps to reproduce it? Thanks!

14

Hi Mat,
if you flash my code above once with SYSTEM_THREAD(ENABLED) and have it run, the next OTA flash will turn on magenta, quickly flash the new code, go back from magenta to cyan breathing without reboot, still Serial.printing for several seconds and only after a while (10 to 15sec) reboot into the newly flashed firmware.

I’d see this unexpected, since a “normal” OTA without the previous code running under SYSTEM_THREAD(ENABLED) immediately reboots into the new firmware afte magenta goes off.

I tested this about half a dozend times with and without - each time about 20sec from magenta to reboot with and less than 5sec without SYSTEM_THREAD(ENABLED) on previous firmware.

1 Like
15

Is it possible this is to do with the fast OTA updates? it receives the bulk of the update but its waiting to receive a missed part?

16

Possible, but I’d be surprised to see this kind of consistency to “always” loose packs the one way but “never” loose any the other way.
And if there are lost packets to wait for, the LED should still show magenta till all packets are successfully received and applied.

17

Guys,
Thanks for taking this one up :wink: I’m glad its not just me seeing this behaviour.

It has put me OFF using threading but I now find that my app stops when the network ‘goes away’ - so I really need to get back to using it I guess :-O. More-so as my current development seems to stop if/when I have network issues…lost my broadband overnight and the app is now ‘locked up’ - no I haven’t put in any code (yet) to detect wifi state etc, ;-)) Next on the list…

Be good to have a reply from Particle on this one, so maybe I should also raise a support request instead of just using this forum ???.

BR
Graham

1 Like
18

Well…actually…

It seems to be fine now. Turned on SYSTEM_THREAD and not yet noticed any slow-down at all :-)).

Now lets ‘break’ my network and see how it performs…

So - whoever fixed it - a BIG thankyou ;-).

Graham

19

I have been trying out a few things with a simple test program on a Photon using SYSTEM_THREAD(ENABLED) to learn the pros and cons of using this new mode (which I’ve been needing badly for a particular project, unfortunately it’s using a Core at the moment).

In any case, I have noticed this exact behavior myself…do a OTA update with some new code and the magenta flashing starts promptly, and the update completes within a second or two while my “old” code continues to run. Then I get back to the breathing cyan, but the “old” code is still going strong for about 10 more seconds or so. Then all of the sudden the device resets and my “new” code begins running (right away though, while WiFi is still connecting, which is nice).

The only time this really is a “problem” is during quick iterations of the code during a troubleshooting session. Kind of negates those nice super-fast OTA updates we now get by slowing down the process considerably.

20

As a followup to my “me too” post, here is something I noticed today that may be useful/related to the underlying issue…

I had an event stream going (https://api.particle.io/v1/devices/events?access_token=…) while I performed a flash of new code and got these events:

event: spark/flash/status
data: {"data":"started ","ttl":"60","published_at":"2015-12-13T21:27:28.301Z","coreid":"redacted"}

event: spark/flash/status
data: {"data":"failed ","ttl":"60","published_at":"2015-12-13T21:27:48.584Z","coreid":"redacted"}

event: spark/status
data: {"data":"online","ttl":"60","published_at":"2015-12-13T21:27:48.587Z","coreid":"redacted"}

Note there was a "failed " event, despite the fact that my code was successfully flashed & the device did restart and begin running the new code.

Also note the 20s gap between "started " and "failed " events.

The the "started " event came just after the magenta flashing ended. Then there was the 10-15s delay before automatic device reset, and another 5-10s for boot & WiFi connection. The "failed " and “online” events came through together once the device had booted up and connected (breathing cyan).

I’m wondering if whatever is “failing” here is the culprit?