Update to 0.7.0 goes wrong

Thanks for testing that. I’m not sure why it’s failing that way. I need to think about that some more. It’s strange, for sure.

I think it’s something related to WiFi credentials. I even tried setting them in the code WiFi.setCredentials (and before that WiFi.clearCredentials).

Same issue - one by one.
Update everything

Summary

Platform: 6 - Photon
Modules
Bootloader module #0 - version 101, main location, 16384 bytes max size
Integrity: PASS
Address Range: PASS
Platform: PASS
Dependencies: PASS
System module #1 - version 207, main location, 262144 bytes max size
Integrity: PASS
Address Range: PASS
Platform: PASS
Dependencies: PASS
System module #2 - version 207, main location, 262144 bytes max size
Integrity: PASS
Address Range: PASS
Platform: PASS
Dependencies: PASS
System module #1 - version 207
Bootloader module #0 - version 101
User module #1 - version 5, main location, 131072 bytes max size
UUID: DB88160717098B7A06661AD00F6011F1F08745A352FA0D2B9F034ECFFA4533E0
Integrity: PASS
Address Range: PASS
Platform: PASS
Dependencies: PASS
System module #2 - version 207
empty - factory location, 131072 bytes max size

and fast blinking green LED on “doctor” sketch .
If I upload my code - it blinking green 5-6 sec, and 1 time white (in cycle), and don’t want go to listening mode.
My code use some system call like:
SYSTEM_MODE(MANUAL);
STARTUP(WiFi.selectAntenna(ANT_INTERNAL));
@kasper - what system mode you used on 0.6.3? Or antenna?
May be bug in this settings - they persistent like wifi credentals setting.

P.S. And great thanks for cure:
reverting to 0.6.3 (particle flash --usb system-part1-0.6.3-photon.bin/particle flash --usb system-part2-0.6.3-photon.bin)
_ press reset_
_ nice light blue fading status led… (with firmware 0.6.3)._

it helps me too!! So now I have frankenstein - bootloader 101 from 0.7.0 and main firmware from 0.6.3, and it works!!!

Well I still have to do one try. I also used WiFi.listen() and that seemed to work with 0.7.0. However after that I thought that I actually never say 0.7.0 connecting to the cloud. I did a quick try and it didn’t seem to work. But after spending the entire day on debugging several devices I decided to quit. I’ll see tomorrow if I can do another test.

Not sure if this the same or similar issue but I updated two 0.6.3 devices in the field to 0.7.0 both appear to update fine and both reconnect to the MQTT server as they are supposed to do. The MQTT client app I use to get data has access to the devices and from that side all seems normal.

However, the devices do not list on the particle console (listed as ‘off-line’), the info for both devices states ‘0.7.0-rc.6’ and a particle flash no longer works i.e., I cannot update them.

These devices are both in a remote location so I cannot do a USB trick of some sort.

Something is botched on particle’s side it seems.

@rickkas7 Any updates on this issue. Today trying out some things again. Particle doctor, re-claiming. I get fast green blinks and then kind of soon the fast blue blink. Something seems wrong regarding WiFi although I’ve resetted everything (including the keys) with Particle Doctor.

I am having the exact same problems with 4 photons. After they upgraded to 7.0 I started showing them showing device coming online in the console throughout the day. I thought I had a wifi problem and started troubleshooting it. I had the wifi up and down several times changing some configs. Once back online I noticed most of the Photons were showing a fast cyan going to red, then fast green.
I couldn’t get them back online after setting them in listening mode even though they appeared to connect to the wifi. The only way I got them back was to manually updating them in DFU mode. I had to also load tinker or they wouldn’t connect. Particle doctor got stuck at the wifi setup also.
I am at the same point of them showing going online in the console now. My internet is by cellular phone so I moved some of them to the cellular wifi directly to see if that would stop the device online problem but it hasn’t. I have had these devices online for a year without this problem. I am going to try and set a couple of them back to 6.3 to see if the problem clears.

For me connecting them with my mobile phone hotspot worked. So there is clearly something going on in certain WiFi network configurations. I’m investigating it with the help of @ScruffR.

@kasper can you share? Anything you and @ScruffR may have investigated and concluded as dead-ends is something I dont have to spend my time on anymore. Alternatively, it may inspire me/us to try some test we had not thought of yet.

I’ve got several devices permanently (so it seems) marooned at 0.7 so am back on 0.6.3 until the sky clears but wouldnt mind bringing back some of these from the dead if there is time and a new idea.

I also have moved 5 Photons back to 6.3 and they are working fine. I also believe they may have worked directly connected to the phone wifi as I found a couple had moved to the router wifi as i hadn’t cleared them completely before adding them to the phone wifi. I would be interested in the results and willing to help troubleshoot the problem also.

One route to investigate has to do with the fact that the 0.7.0 updates will “reboot” the device more often than previous updates due to the also shipped bootloader.
As it seems some WiFi networks don’t play well with devices that appear to be unstable.
Making sure that the device has only one set of WiFi credentials will reduce the risk of conneting to another network potentially disrupting the process.
Also make sure that your device gets the same IP address assigned on reconnect - this should prevent your DHCP server to run out of free IPs.

When doing an 0.7.0 OTA update you should see five event log entries (e.g. in console) event: spark/safe-mode-updater/updating with the data payloads 1, 2, 0, 1 and 2 for a successful update.

1 Like

Thanks for the update @ScruffR. Couple of questions if I may:

  1. “some WiFi networks don’t play well with devices” any particular characteristics for those networks? (i.e. particular modems, routers etc.?)
  2. “making sure that the device gets the same IP address…” This is not a forward going recommendation I hope i.e., for actual product releases one cannot ensure IP addresses stay the same. If 0.7 (sort of) requires this, I’d say it is dead in the water for real products.

I have a few units in the field marooned on 0.7 (they stopped working after the ‘upgrade’) so I took a sacrificial unit from the stash in the lab to see whats up. It appeared to start the upgrade but eventually gets stuck in a cycle whereby it cannot connect to the particle cloud (light blue blinking rapidly followed by a few red flashes and reset). I see my firmware kicking in at some time during that cycle (evidenced by peripherials being initialized). The FW has software watchdog that will reset after 15 seconds. It is possible that the reset is initiated by my FW - I cant tell. A “particle identify” shows it is on 0.7 (but again, no idea if all the pieces are there: part 1, part 2 and bootloader).

After a lot of fudzing (tech term meaning: “i’ve got no idea what i am doing here!”) with particle doctor, I got the unit back onto 0.6.3 and playing normally. One of the field units I got back has no USB so I cannot play doctor with it and is (i guess) permanently gone bananas…

No, that's just one way to avoid some other issues that shouldn't be a problem in professional setups (e.g. running out of DHCP IPs on consumer routers).

particle serial inspect will provide a more detailed info about the state of the device.

One thing that has come up with 0.7.0 is that the Application Watchdog call outlined in the docs will now cause an SOS crash when the AWD fires and calls System.reset()
Currently for 0.7.0 you need provide the stacksize parameter (~1536 - three times the default of 512) due to higher stack demand of `System.reset()´

I had to do an update to the code on one of my photons today and didn’t realize when using Particle Desktop that it defaults back to 7.0 so when I updated the code it also updated the device back to 7.0 from 6.3. I didn’t see the sequence that @ScruffR stated should happen. I never saw it do part 1.

That’s interesting?
Could you provide the output of particle serial inspect and/or the raw output of spark/status/safe-mode?

Here is the first safe mode event.

  {"data":"{\"f\":[],\"v\":{},\"p\":6,\"m\":[{\"s\":16384,\"l\":\"m\",\"vc\":30,\"vv\":30,\"f\":\"b\",\"n\":\"0\",\"v\":11,\"d\":[]},{\"s\":262144,\"l\":\"m\",\"vc\":30,\"vv\":30,\"f\":\"s\",\"n\":\"1\",\"v\":207,\"d\":[]},{\"s\":262144,\"l\":\"m\",\"vc\":30,\"vv\":30,\"f\":\"s\",\"n\":\"2\",\"v\":109,\"d\":[{\"f\":\"s\",\"n\":\"1\",\"v\":109,\"_\":\"\"}]},{\"s\":131072,\"l\":\"m\",\"vc\":30,\"vv\":26,\"u\":\"53821C91ECA602B546DEFBDCD322B58D63E46D62E2B5ED4CB93DE70385D9BCAB\",\"f\":\"u\",\"n\":\"1\",\"v\":5,\"d\":[{\"f\":\"s\",\"n\":\"2\",\"v\":207,\"_\":\"\"}]},{\"s\":131072,\"l\":\"f\",\"vc\":30,\"vv\":0,\"d\":[]}]}","ttl":60,"published_at":"2018-04-23T11:38:47.703Z","coreid":"XXXXXXXX","name":"spark/status/safe-mode"}
particle serial inspect
Platform: 6 - Photon
Modules
  Bootloader module #0 - version 101, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 207, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #2 - version 207, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 207
      Bootloader module #0 - version 101
  User module #1 - version 4, main location, 131072 bytes max size
    UUID: 03CCAF9BD4FE7A15AFB8485BA6CA6B9C7B95917662A6457E1EDA68CF4B733930
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 109
  empty - factory location, 131072 bytes max size

I had problems downgrading my Photons to 6.3. I first installed Tinker. I put in DFU mode and did part 2 next. Then waited as it OTA the bootloader. Then I put back in DFU mode and did part 1. After part 1 it stayed in DFU mode so I had to do part 2 again to get it to come out of DFU mode… It came out of DFU mode after installing part 2 and I installed Tinker.

Part 2 should not have been required there.

In order to downgrade, you should first install any application firmware targeted at your system to be - to prevent an accidental OTA update in the process.

Next system part 3 (for Electron only), part 2 & part 1.
If you then still find your device in DFU Mode, you need another application firmware flash (e.g. particle flash --usb tinker, or your previous application).
After that you probably also want to flash the bootloader that belongs to that version via particle flash --serial bootloader-<....>.bin.

For some reason I seem to miss the location of the bootloader for photon 6.3. Can someone direct me to it? Thanks. https://github.com/particle-iot/firmware/releases

Earlier today, I mentioned in another post that I had hosed up one of my devices by rushing through the update procedure and I found myself in a pickle that I seemingly could not get out of. So…I ended up running particle_firmware_manager-v0.6.0-osx several times, each time getting me closer to being “fixed” (as shown by particle serial inspect), until I was finally out of the woods. I then did a standard update to 0.6.4 and then particle update to finally get me solidly on 0.7.0.

I don’t understand what all goes on under the covers with the firmware managers, but multiple runs seemed to be required to get things right. In any case, this may not make any sense, but I know it worked for me for whatever reason. Just wanted to share my experience in case it might be of some benefit to you or others.

Hi @jstobaugh

I believe you will find the 0.6.2 bootloader in the 0.7.0rc.3 section. https://github.com/particle-iot/firmware/releases/tag/v0.7.0-rc.3

The bootloader was not updated very often on the past but some new features in 7.0 required changes to it.

As @ScruffR warns above, you want to be careful not to let the automatic OTA update system undo your changes.

1 Like