Is it possible to use the factory backup region of flash memory to expand the User application and OTA backup regions of flash memory?
Motivation
I reached my maximum application size months ago, and have been struggling to make room for more features in my application ever since. I have had to disable the vast majority of my debug messaging in order to save program space, which makes debugging extremely difficult.
Idea
Since there is 128 kB of flash memory dedicated to the factory backup region, it would seem to me that it might be possibly to re-purpose this memory region to store a bigger user application. If I understand things correctly, that would allow my application to be 64 kB bigger (since this extra space would have to be split between OTA backup region and user application region in order to be able to OTA update the device).
Consequences Downsides: I have never (to my knowledge) used the factory backup region of flash memory for anything, so it doesn’t seem like I would be giving anything up here. Upsides I could continue to add features to my application for the foreseeable future.
There are a few problems with doing this. Most notably, you cannot OTA flash code that extends beyond the normal firmware image location. You can only flash by USB. You cannot OTA flash the factory backup image slot, either.
On the Photon/P1, there is another issue. There really isn’t support for backup firmware on the Photon/P1 because that sector is used when doing a system firmware update OTA. The OTA sector is used when doing a user firmware update, but when doing a system firmware update, both the OTA sector and the backup sector are used, because 256K is needed do to a system firmware update. So your code there would get overwritten on an OTA system firmware update.
On the Electron, the system parts are 128K so you’d have better luck using the backup sector.
Also, none of the build tools are set up to use the extended size. It’s also a pain because the space is non-contiguous.
Since you can’t OTA flash an extended image anyway, the easiest workaround if you need a lot of flash memory and don’t need OTA is to build a monolithic binary. That combines the system and user parts, and I think you can even extend into the OTA sector.
Forgot to mention that I'm using an Electron. Looking again at the memory map for modular firmware section of the Electron datasheet I noticed that the factory backup sector is labelled as "Factory Reset/Extended Application"... which makes me think that I should be able to do what I am suggesting....
Is this a limitation of the the Particle servers, spark firmware, bootloader, or the bare metal STM32 (or all 4?)
I understand that the OTA backup sector needs to be at least as big as the user application sector, but I don't understand why I wouldn't be able to increase both of those sector sizes by the same amount (i.e. 192 kB user application sector and 192 kB OTA backup sector).
Any solution to increase the size of my program would need to preserve the ability to send OTA updates, so the monolithic firmware build is not an option in my case.
It might be possible to make it work on the Electron, with some limitations.
The problem is that the sector size of 128K is set in hardware on the STM32F205. In order to write to flash you need to erase it first, and you can only erase to sector boundaries. So you can’t split a sector in half and use it for two different things unless you plan to only ever write both at the same time.
So you basically need to have a 256K extended application and also a 256K OTA, they can’t be 192K each. Now there appears to be enough room if you take over the OTA backup and decompress region, but I’ve never tried it.
At minimum it would require changes to system firmware, and of course the build tools.
Correct. That’s what I’m thinking. I think it has a pretty good chance of working, though there’s obviously the possibility that something I haven’t considered will cause problems. But I think it’s worth a try.
This is something that I think my company would be willing to put a bounty out for, as limitations on app size have been a fairly sore pain point for us.
Anyone out there who is reading this and thinking “I think I could develop the ability to make this work”, PM me and we can talk details.
Hi guys,
I’m slowly hitting the limit of application firmware size as well. Has been there any development done on the topic of extending into factory reset and decompress region sectors?
@peekay123 could any form of application code run from Flash if it was installed on Flash via a HTTP download instead of a OTA update?
@jaza_tom I was asking for the same feature the other day to expand storage area for a OTA update. I just wanted more room for large bitmap images without needing code to do it after a OTA.
No, just saying I was asking the other day if there was a way to free up more room or write to the flash during the OTA update process similar to what your wanting to do.
I had Paul create some code years ago to download BMP image from a website and the store them on a SD card but that application alone took up tons of space so it wouldn’t help you with extra program code space.
