Hi all,
Are there any plans to reduce the time it takes to flash + reboot the Spark Core?
Reducing it by 10-15 seconds, or maybe even remove the need for a hard reboot, would greatly increase the development experience.
1 Like
Hey @jfernandez!
So when we originally released, the build time was really significant, usually 20-30 seconds before we even started flashing. Now, thanks to @mattandeās pull request in response to our feature bounty, building the code typically takes just a second or two!
From there, over-the-air flashing time is basically at its limit. The Core has to download the firmware binary, which is about 65KB over the 802.11b/g connection, in checksummed chunks small enough to fit in the microcontroller memory before being offloaded to the external flash chip. Then we have to reboot into the bootloader because the you canāt modify the firmware in the microcontroller while the firmware itself is running.
If you want to have a faster code-to-flash development cycle, you should build the firmware locally. Instructions are in the README for the core-firmware repo.
1 Like
Of course, as I read my response again, my imaginationās spinning with hypothetical ways we could make it faster. Theyāre all significant architecture changes though. So, letās say this: (1) I appreciate the feedback! (2) After weāve significantly built up the stability and feature set of the Spark Core, weāll consider some bigger changes down the road to make the coding turnaround time faster.
Cheers!
1 Like
Ok I saw my name so Iāll chime in.
I donāt have hardware so Iām not certain how long the build+flash process from the online IDE takes, but one of the typically slower aspects is the time required to write to flash, especially an external flash.
I took a quick look and notice that in the OTA programming process and notice the current image is backed up from the STM flash to the external flash before the OTA image is written. This might be handy if, say, the new load has a problem and the user wants to quickly restore the previous (good) load without having to connect back with the cloud. Though (currently) it appears there is no way to restore the backed up flash image.
In a rapid development environment maybe give the developer an option to skip backup of the old firmware if they know they donāt need it? Might save a second or so.
Another idea would be to have the online IDE automatically build a patch between the last OTA image and the new image (say, only the changed flash pages), reducing the size of the download and flashing time which are selectively applied by the bootloader.
1 Like
Thanks, @zachary. Thinking back to my experience flashing the core with the online IDE, I remember now that most of the time was spent on the reboot. For a web developer like me with no hardware experience, thereās -a lot- of trial and error, and making the coding turnaround faster would really help.
Also, when did @mattandeās changes take effect? Iām out of the country ATM, but the last time I tried flashing was on Dec 10th.
Mattās makefile changes got merged into the compile service over the weekend, so basically, itās been available since this morning.
1 Like
Compile and flash over USB is definitely fasterā¦about 10-15 seconds in comparison to about a minute over the air.
Re: external flash: the reason that we write to the external flash is so that we never ābrickā your Core by writing the OTA image and getting interrupted (lost power, etc.). But yes, we could make this an option down the road for developers who want to trade reliability for speed.
Your second thought - having the online IDE build a patch - is a good suggestion, and definitely something we could look into longer term to make the whole process significantly faster.
sending a āpatchā might not be as easy as it sounds. when you add a single line somewhere, all code behind it shifts. its possible the compiler optimises things differently.
an option is to compile the āOSā and the āuser applicationā separately, but embedded systems like these are not designed to do that. they usually donāt have loadable executables like PCs have.
Remco
@zach, re: external flash: I was referring to the backup of the current STM firmware to EXTERNAL_FLASH_BKP_ADDRESS made before the OTA firmware is flashed. Not the factory firmware at EXTERNAL_FLASH_FAC_ADDRESS. It doesnāt look like the firmware backup at the BKP address is ever used.
But Iām not sure because I cant find the bootloader source. Is it posed on github?
The bootloader does revert to the previous backed up firmware if the firmware that gets flashed is bad. Weāve tested this, for example, by flashing something like a jpg image as āfirmwareā to the Core. 
- The transfer from the server successfully finishes
- We drop into the bootloader
- The lolcat gets copied into internal memory
- The Core does a sanity check to see whether the new firmware is runnable
- Decides ānopeā
- Copies the backup firmware into internal memory
- Run!
The bootloader README needs some serious love. Itās very out of date. Weāre ready to open source it other than that. It would cause a lot of confusion if we did though, so as soon as we can give the README some love, weāll make it public.
Cheers!
Ah, now Iām tracking. Makes perfect sense for the bootloader to automatically restore the previous firmware if the OTA load doesnāt run because of a hard processor fault.
some more-or-less random thoughts:
*) ring-buffer for 2 firmwares, that way you donāt have to copy it but its still there, you can use a few bytes in external flash that function as a ringbuffer too, search for the highest value and check the LSB to see which firmware is the most recent one.
*) do a sanity check server-side, then flag it as āsafeā, then the application receiving the firmware an skip some tests
*) can the sanity check be done on the first block that the core receives? that way it doesnāt even need to write it to flash or jump to the bootloader
*) a Cortex is very fast and the data is received by the ābigā application code anyway, may be a light compression algorithm can speed up the transfer a bit?
*) Could the compiler return the size of the hex file to the web ide? may be with some nice percentage of how full your core is
Remco
2 Likes
Nice suggestions @remcohn!
We have to do the final (pretty minimal honestly, nothing fancy) sanity check on the Core as opposed to the server because the most likely point of corruption is the wireless connection. Even if the server had a perfectly good file, some packets could have gotten dropped or duplicated, or the wireless transfer may have otherwise gotten goofed up.
Great suggestionāadded to backlog! āFlash & Verify responses should include size of binaryā
Compression is tough on tiny devices, and it would have to impact the download time enough to compensate for the additional decompression time, but Iāve added a task to research it to our backlog. Interesting discussion here:
I was skeptical how generally compressible the binaries might be, but a quick gzip reduced one from 68148 bytes 49483 bytes (72%), so thatās not badāless entropy than I would have thought.
Thanks!