Flashing speed?

Hi all,

Are there any plans to reduce the time it takes to flash + reboot the Spark Core?

Reducing it by 10-15 seconds, or maybe even remove the need for a hard reboot, would greatly increase the development experience.

1 Like

Hey @jfernandez!

So when we originally released, the build time was really significant, usually 20-30 seconds before we even started flashing. Now, thanks to @mattande’s pull request in response to our feature bounty, building the code typically takes just a second or two!

From there, over-the-air flashing time is basically at its limit. The Core has to download the firmware binary, which is about 65KB over the 802.11b/g connection, in checksummed chunks small enough to fit in the microcontroller memory before being offloaded to the external flash chip. Then we have to reboot into the bootloader because the you can’t modify the firmware in the microcontroller while the firmware itself is running.

If you want to have a faster code-to-flash development cycle, you should build the firmware locally. Instructions are in the README for the core-firmware repo.

1 Like

Of course, as I read my response again, my imagination’s spinning with hypothetical ways we could make it faster. They’re all significant architecture changes though. So, let’s say this: (1) I appreciate the feedback! (2) After we’ve significantly built up the stability and feature set of the Spark Core, we’ll consider some bigger changes down the road to make the coding turnaround time faster.

Cheers!

1 Like

Ok I saw my name so I’ll chime in.

I don’t have hardware so I’m not certain how long the build+flash process from the online IDE takes, but one of the typically slower aspects is the time required to write to flash, especially an external flash.

I took a quick look and notice that in the OTA programming process and notice the current image is backed up from the STM flash to the external flash before the OTA image is written. This might be handy if, say, the new load has a problem and the user wants to quickly restore the previous (good) load without having to connect back with the cloud. Though (currently) it appears there is no way to restore the backed up flash image.

In a rapid development environment maybe give the developer an option to skip backup of the old firmware if they know they don’t need it? Might save a second or so.

Another idea would be to have the online IDE automatically build a patch between the last OTA image and the new image (say, only the changed flash pages), reducing the size of the download and flashing time which are selectively applied by the bootloader.

1 Like

Thanks, @zachary. Thinking back to my experience flashing the core with the online IDE, I remember now that most of the time was spent on the reboot. For a web developer like me with no hardware experience, there’s -a lot- of trial and error, and making the coding turnaround faster would really help.

Also, when did @mattande’s changes take effect? I’m out of the country ATM, but the last time I tried flashing was on Dec 10th.

Matt’s makefile changes got merged into the compile service over the weekend, so basically, it’s been available since this morning.

1 Like

Compile and flash over USB is definitely faster…about 10-15 seconds in comparison to about a minute over the air.

Re: external flash: the reason that we write to the external flash is so that we never “brick” your Core by writing the OTA image and getting interrupted (lost power, etc.). But yes, we could make this an option down the road for developers who want to trade reliability for speed.

Your second thought - having the online IDE build a patch - is a good suggestion, and definitely something we could look into longer term to make the whole process significantly faster.

sending a ‘patch’ might not be as easy as it sounds. when you add a single line somewhere, all code behind it shifts. its possible the compiler optimises things differently.

an option is to compile the ‘OS’ and the ‘user application’ separately, but embedded systems like these are not designed to do that. they usually don’t have loadable executables like PCs have.

Remco

@zach, re: external flash: I was referring to the backup of the current STM firmware to EXTERNAL_FLASH_BKP_ADDRESS made before the OTA firmware is flashed. Not the factory firmware at EXTERNAL_FLASH_FAC_ADDRESS. It doesn’t look like the firmware backup at the BKP address is ever used.

But I’m not sure because I cant find the bootloader source. Is it posed on github?

The bootloader does revert to the previous backed up firmware if the firmware that gets flashed is bad. We’ve tested this, for example, by flashing something like a jpg image as “firmware” to the Core. :smile:

  1. The transfer from the server successfully finishes
  2. We drop into the bootloader
  3. The lolcat gets copied into internal memory
  4. The Core does a sanity check to see whether the new firmware is runnable
  5. Decides “nope”
  6. Copies the backup firmware into internal memory
  7. Run!

The bootloader README needs some serious love. It’s very out of date. We’re ready to open source it other than that. It would cause a lot of confusion if we did though, so as soon as we can give the README some love, we’ll make it public.

Cheers!

Ah, now I’m tracking. Makes perfect sense for the bootloader to automatically restore the previous firmware if the OTA load doesn’t run because of a hard processor fault.

some more-or-less random thoughts:
*) ring-buffer for 2 firmwares, that way you don’t have to copy it but its still there, you can use a few bytes in external flash that function as a ringbuffer too, search for the highest value and check the LSB to see which firmware is the most recent one.
*) do a sanity check server-side, then flag it as ‘safe’, then the application receiving the firmware an skip some tests
*) can the sanity check be done on the first block that the core receives? that way it doesn’t even need to write it to flash or jump to the bootloader
*) a Cortex is very fast and the data is received by the ‘big’ application code anyway, may be a light compression algorithm can speed up the transfer a bit?

*) Could the compiler return the size of the hex file to the web ide? may be with some nice percentage of how full your core is :smile:

Remco

2 Likes

Nice suggestions @remcohn!

We have to do the final (pretty minimal honestly, nothing fancy) sanity check on the Core as opposed to the server because the most likely point of corruption is the wireless connection. Even if the server had a perfectly good file, some packets could have gotten dropped or duplicated, or the wireless transfer may have otherwise gotten goofed up.

Great suggestion—added to backlog! “Flash & Verify responses should include size of binary”

Compression is tough on tiny devices, and it would have to impact the download time enough to compensate for the additional decompression time, but I’ve added a task to research it to our backlog. Interesting discussion here:

I was skeptical how generally compressible the binaries might be, but a quick gzip reduced one from 68148 bytes 49483 bytes (72%), so that’s not bad—less entropy than I would have thought.

Thanks!