How to force a handshake for OTA updates

A reset does not seem to kill the session keys, pulling the RST pin does not, and pulling the EN pin does not on a Boron, unless it is done long enough, and I have not be able to determine that limit yet.

Have you tried this:

I am currently trying this for when a module hangs. My quest is to find a way to reset the device and session keys that work every single time from 1.0.1 and up, (It is a MAJOR pain). Otherwise we can not use this in a product.

Have you had any luck figuring out how to reset the session keys?

Not consistently longterm.

The above quoted way of doing this does seem to work on earlier releases, but it seem to be more of a hack than an actual platform supported way of doing it longterm. A HW watchdog in itself does not seem to be able to do it reliably either.

It can be or may already have been blocked in future OS updates, as session key handling in connection with deploying SW updates was recently “featurized”. If an when that happen(ed) we would likely not be informed about it, and our customers would be back “in harms way” (a few days to a weeks outage, as seen recently would kill our business).

The lack of consistent control over resetting the session keys in a supported way, also in connection with a HW watchdog, and in combination with a cloud that is not yet reliable enough for our purpose, is too much of a risk for a product in large numbers at this point.

@BDub can you comment on whether there is a supported workaround that can be used to cause a fleet of Electrons to “choose” to download a firmware update simultaneously? Has fleet-coordinated firmware updating been “featurized” and therefore locked down with no workarounds as @thrmttnw posits?

The feature I am referring to is called intelligent firmware releases:
https://docs.particle.io/tutorials/device-cloud/ota-updates/#intelligent-firmware-releases

Hi,

Before the Intelligent Firmware Releases we were enableing updates System.enableUpdates and triggering a handshake event like this to force an update. We are currently on firmware version 1.4.4

Particle.publish(“spark/device/session/end”, “”, PRIVATE);

That no longer works. Per the comment below, I have tried sending firmware binaries to a specific device to trigger an update, but I get a cryptic error: "{\"error\":\"Nothing to do?\"}"

Here is how I’m triggering the device firmware update, where filename is compiled .bin file:

curl -X PUT https://api.particle.io/v1/products/:productId/devices/deviceId?access_token=ACCESS_TOKEN  -F file=#{filename} -F file_type=binary

I’ve tried using a device bearer token AND the product-level access token. Either way I get the same error. How can I flash a device to customers?


I have not been able to get this to work.on product devices. I receive a ‘disconnect initiated’ response, but there is no new handshake that is forced.

I too find this no longer works. I was unable to update our devices. I wonder if it’s related to this Particle bug which last checked is still not fixed: Delay in 'online' status and failed particle.function Basically Particle’s cloud side tracking onfdevice connection status seems broken.

I hope it isn’t. That issue looks like it has been around for a bit and we have customers waiting!

I am knocking on wood because who knows what will change, but we solved this issue for our firmware by creating a handler that watches the system events for the firmware update status and then waits if an update is coming through. Currently working on v1.2.1 and doesn’t seem to require a handshake any longer.

The firmware_update system event will give one of the following on connection. (firmware_update_begin , firmware_update_progress , firmware_update_complete , firmware_update_failed).

We set up a handler similar to this.

void firmware_update_handler(system_event_t event, int status) {
  switch(status) {
    case firmware_update_begin:
      gFirmwareUpdateStatus = FW_UPDATING;
      appLog.info("Firmware update beginning event was triggered.");
      break;
    case firmware_update_progress:
      gFirmwareUpdateStatus = FW_UPDATING;
      appLog.info("Firmware update progress event was triggered.");
      break;
    case firmware_update_complete:
      gFirmwareUpdateStatus = FW_DONE;
      appLog.info("Firmware update complete event was triggered.");
      break;
    case firmware_update_failed:
      gFirmwareUpdateStatus = FW_DONE;
      appLog.error("Firmware update failed event was triggered.");
      break;
    default:
      appLog.warn("Unknown firmware_update_handler status %i", status);
      break;
  }
}

Then, once our state changes to FW_UPDATING, we wait and Particle.process() like this.

while (gFirmwareUpdateStatus == FW_UPDATING) {
    Particle.process();
  }

Here is the documentation. Hope this helps someone!
https://docs.particle.io/reference/device-os/firmware/electron/#system-events-reference

3 Likes

dcliff9, thanks for the help!

One follow up question: How are you triggering firmware update events (firmware_update_begin, firmware_update_progress)?

The Device OS API will send a status of “firmware_update_begin, firmware_update_progress, etc” depending on if a firmware update is coming through. This automatically happens when your device is part of a product. You should always get a status from this if a firmware .bin has been released and is marked to deploy to that device.

You can add an event handler to handle these system events.

System.on(firmware_update, firmware_update_handler);

In your handler, you can then use that info to switch your own state for your firmware so you can do whatever you want during each state. For instance, you can change a switch state based on these parameters that come through the system event. See the “status” variable below.

void firmware_update_handler(system_event_t event, int status) {
  switch(status) {

This is the parameter from the Device OS API. When this changes, we log to the console and switch our gFirmwareUpdateStatus variable so we can use that outside of this handler.

  switch(status) {
    case firmware_update_begin:
      gFirmwareUpdateStatus = FW_UPDATING;
      appLog.info("Firmware update beginning event was triggered.");
      break;
    case firmware_update_progress:
      gFirmwareUpdateStatus = FW_UPDATING;
      appLog.info("Firmware update progress event was triggered.");
      break;
    case firmware_update_complete:
      gFirmwareUpdateStatus = FW_DONE;
      appLog.info("Firmware update complete event was triggered.");
      break;
    case firmware_update_failed:
      gFirmwareUpdateStatus = FW_DONE;
      appLog.error("Firmware update failed event was triggered.");
      break;
    default:
      appLog.warn("Unknown firmware_update_handler status %i", status);
      break;
  }

For instance, outside of this handler, we want to have the device hang there and do a particle.process while the update is happening, so we add this while loop.

while (gFirmwareUpdateStatus == FW_UPDATING) {
    Particle.process();
  }

The other states don’t really matter to us as the device reboots automatically after an update. But, for instance, we could do something like flash an LED if our state switches to “firmware_update_failed” to let us know something bad happened.

3 Likes

Awesome! Thanks :slight_smile:

Do you just use this on start up, or do you allow users to “Update Firmware” and then enable these handlers?

@dcliff9 I just added this to our project and it still doesn’t solve the issue of triggering the update. If the update is triggered, it looks like it will work, but part of my issue with this is that I can’t trigger the device to end a session and try to reconnect (when all these firmware update events are initiated). Have you been able to solve this?

p.s. Thanks so much, we’ve been scrambling to try to get this done for our customers, so this is super helpful!

That handler should run as soon as the status changes. That is triggered by a firmware release to the product automatically.

Are you following this process to upload and release a firmware binary?
https://docs.particle.io/tutorials/device-cloud/ota-updates/#fleet-wide-ota

@dcliff9

We are using the Standard releases. We aren’t enterprise, so we don’t have access to the Intelligent method. But we are creating releases and assigning them to devices, if that is what you mean.

Something you can try. Put this in your code instead.

void setup()
{
    // listen for network events and firmware update events
    System.on(all_events, handle_all_the_events);
}

void handle_all_the_events(system_event_t event, int param)
{
    Serial.printlnf("got event %d with value %d", event, param);
}

This will allow you to see all of the system events in a console. You can set up a device and watch this while you try to release your firmware. If it is getting released correctly, you should see the “firmware_update_begin” value come through on the “firmware_update” event.

1 Like

We have opted not to automatically update the device on start, but instead allowing the user to select “Update Firmware”. This is probably the crux of our problem and maybe some of the confusion in this conversation. We run System.enableUpdates() when they want to update firmware, then end our session and put the device in safe mode to accept updates. This previously worked, but no longer does.

Are you doing this?

  • Your devices are in a product.
  • The product and firmware version number is in your code.
  • One of the devices in that product is already running your firmware version. (Flashed via USB).
  • In the particle console, you go to that product, click the “firmware” button on the left, upload your new firmware binary, then hit the “release firmware” button.

That shouldn’t be an issue. You can default to System.disableUpdates() and then enable system updates via System.enableUpdates() when the client chooses. Once that is enabled, the firmware update status should change as long as there is a newer released firmware waiting. I have seen it take as much as 10 seconds for that firmware_update system event to update, but it eventually does. Note that this only appears to work in v1.2.0 or later.