Automatic over-the-air Device OS updates now enabled for cellular devices

Hello all,

As of today, the Device Cloud will automatically perform over-the-air Device OS updates when required for all Particle devices. This will make it much easier to upgrade to the latest and greatest version of Device OS.

Device OS is Particle’s lightweight operating system for embedded IoT devices (formally known as system firmware). Every Particle device runs Device OS on its microcontroller to simplify firmware development.

There are two main components of Device OS: the application layer (the logic you write), and the system layer (the code written and maintained by Particle). These layers have dependencies on one another that must be carefully managed.

UPGRADING DEVICE OS VERSION
Application firmware that is written in the Web or Desktop IDEs are compiled against a specific version of Device OS system firmware before being sent to a device to run. That is, the Device OS acts as a translator — taking the human-readable code you write and translating into a binary that the device is able to run.

When you flash an application firmware that is compiled against a newer version of Device OS, the device will enter safe mode (RGB LED breathing magenta). Safe mode prevents the device from running the incompatible application firmware that will cause it to hard fault. In this mode, the device connects to the Particle Device Cloud, but does not run the application firmware.

The Device Cloud will then automatically upgrade the Device OS version over-the-air to resolve dependencies. After blinking magenta for a while (receiving the new Device OS versions), the device will reconnect to the Particle Device Cloud and start running the application firmware.

WHAT HAS CHANGED
Previously, for cellular devices (Electron and E Series), the automatic resolution of Device OS dependencies was turned off. We did this with the intent to help you keep cellular data usage (and in turn, overage costs incurred) low.

It turns out that without this automatic resolution process, it becomes quite tricky to manage firmware dependencies on a Particle device. What is normally a painless process that “just works” quickly turns into a puzzle of file downloads, CLI wizardry, and strict ordering.

Now, the Device Cloud will heal cellular devices that require a Device OS version upgrade automatically, the same way it always has for Wi-Fi devices.

WHY MAKE THIS CHANGE
It became clear that disabling automatic version resolution and constantly warning about cellular data usage was causing more harm then benefit. We’ve recognized this friction in the developer community and want to make things better.

In parallel, we’ve also increased the included amount of cellular data to 3MB per device/mo. as part of our new pricing. This gives you more breathing room to take advantage of helpful features like this without incurring data overage costs.

WHAT CHANGES WILL I NOTICE?

  • Cellular devices will automatically upgrade to a newer version of Device OS if it receives application firmware compiled against a newer Device OS
  • Developer tools like the Web IDE and CLI will no longer prevent your from sending application firmware that will cause your cellular device to enter safe mode, or warn you about cellular data usage each time you want to flash. These changes will be rolled out over the next few weeks
  • You may notice a slight uptick in the amount of cellular data consumed per month by cellular devices as a result of these changes

WHAT WILL STAY THE SAME

  • When flashing a specific Electron, developer tools like the Web IDE and CLI still default to compiling for the Device OS version currently on the device to avoid unexpected data usage from Device OS upgrades
  • You will still be able to update Device OS versions locally using the Particle CLI

We’re confident that with this change, the developer experience building with Particle continues to improve. We’re always grateful to you, the community, for sharing your feedback with us and helping evolve the platform for the better.

Happy building,

Jeff

13 Likes

That’s great news.

Will this work with 3rd Party SIMs as well?
(Will the Device update the Device OS first, before actually upgrading the Application Firmware and entering ‘safe mode’? or will it only work with the ‘Particle APN’ ?)

As far as I’m aware, it’s not related to the SIM you’re using. That merely provides the connection, so it should work just the same for 3rd party SIMs.

Excellent, that it is finally up to end users to decide!

Safe mode requires a network connection (Safe mode explained) and a 3rd party sim normally has a different APN and shorter keep-alive period, both set in user code that is not running in safe mode (Electron 3rd-party SIM tips).

The APN is stored in the modem RAM, so as long as you don’t cold boot the device or go sleep mode with cellular off, safe mode will continue to work. In other words, in the most common case of getting an OTA software updates that requires a system firmware update, a normal System.reset() is done so when the Electron boots into safe mode it will still work with a 3rd-party SIM card.

However, there are cases where it won’t work, such as if you unplug power, so definitely be careful to not do that when you need to safe mode heal.

1 Like

Thank you, Particle!!! This is a huge benefit for those of us who are deploying devices built on the cellular devices. I’ve been wondering how to handle updating field device OS and won’t burn any more cycles thinking about this issue. Again, thank you!

1 Like

Good move!

1 Like

Forgive this possibly naive question.

If we have a product that has firmware binaries uploaded to a fleet of devices how do we update the OS for the entire fleet?

Do we verify the product firmware against a device that has the target OS that we want and upload that to the products firmware manager page in the console and then release it?

Does latest non-prerelease OS automatically update to entire fleet when new product firmware is released?

Do we have to use Build or the CLI to update fleet devices individually (what I do to updates development devices within the product)?

If things go to plan, then the cloud knows about the currently installed Device OS version and will update it if your application firmware requires an update.

1 Like

Yes, what ScruffR said.

The cloud never pushes a new Device OS to devices by itself.

You flash your user firmware to the device, whether by USB, OTA, product lock and flash or product fleet release, and that firmware specifies the minimum Device OS that is required to run it.

When the device reboots, it will check the dependencies and if a newer system firmware/Device OS version is required, then it will be updated OTA. This may require as few as two or as many as five more reboots, but then the device will come back online running your firmware and an upgraded Device OS.

If the Device OS is the same or newer than the version required by the user firmware, then booting proceeds normally.

There is never a check for a newer Device OS version by either side. If you want to upgrade your devices to a new Device OS (pre-release or release), you flash a new user firmware version that requires it. It’s always entirely under your control.

2 Likes

Thanks for the clarification. I was mainly asking due to the release of 0.8.X and getting ‘Device Vitals’ up and running on a product fleet.

Will it downgrade to a previous Device-OS as well?

I’m just testing it now, and it didn’t appear to… so I’m downgraded via CLI to test the "up"grade

It’s not a major if it doesn’t downgrade, just to be explicit about what it can and can’t do.

Also… We have an external WDT for surety purposes, with about 15sec timeout, I suspect our OTA Device OS upgrade is not going to work so well when it enters safe mode.

Subsequent testing.

A WDT at 15secs is too fast, it just sits in a loop trying to upgrade and then resetting via a WDT reset. (not sure how you would recover from this if it was deployed in the field)

A WDT at 125secs is fine, and everything upgrades - I didn’t check to see whether the WDT reset the device in between the Device OS initiated resets, during the update process. Everything seemed to work seamlessly.

Regards

1 Like

That's an important piece of info now with the Electron.

Thanks for sharing.

Downgrading is never done automatically since application firmware that's targeted at a previous Device OS will still run on higher version Device OS (backwards compatiblity).

Currently there is no single-action downgrade option.
If you want to downgrade, you need to download the individual system binary modules and flash them to the device in revers order (app targeted at old version, part3 [if required], part2, part1).

1 Like

Arghh, what a headache - I loaded an application compiled for 0.8.0-rc.4 into a product in the field which has the external WDT, thinking it would run on the old Device OS. but that is the reverse of this comment!

All the units are merrily attempting to update the Device OS and rebooting in an endless loop as per my above comments.

Any way to recover from this? I see a safe mode event occur in the console every 15sec (my WDT period)

The update process relies on the device staying connected during the process, so you'd need to have some means to deactivate the WDT in code once your device senses a pending update.

And in the meantime I would compile a temporary version of your code that is very lightweight (small) that simply disables the watchdog and connects to the internet. Compile it for the current version of Particle OS that your device is running. Then flash it OTA. It will flash in a small amount of time (compared to the full app that you originally tried to OTA). If your WDT still resets even trying to flash this small program, then you may need to get someone near each of your devices to put it in safe mode. This assumes that when your device is in safe mode that the WDT is disabled by design. In general, it is a good design practice to make sure that the hardware is laid out in such a way that the external watchdog is disabled when the MCU is allowing the watchdog control pin to tristate (float).

If you hardware is not laid out in such a way that the watchdog is disabled when the MCU is in safe mode or resetting, then you are probably out of luck and will have to separate the device and the watchdog physically in order to flash new firmware, or else flash it via USB so quickly that it gets updated within the watchdog reset window.

1 Like

Thanks, We can’t disable the WDT, it’s like this by design. I’ve had to go and reflash them all… It was really painful - lesson learned though.

When an OTA system update occurs, is there any notification via console events, or is it done behind the scenes?

Is it possible to lock devices so they cannot receive OTA system updates?