Hoping someone else has found a good workaround to our problem 
Problem: Lacking EEPROM storage space to fit two certificates (2450 characters).
Situation: Using AWS MQTT requires the use of certificates, they add up to being 2450 characters. The certs are different for each device (hardcoding would mean different code for every device).
Looking for: Advice from others who may have come across this and found a good solution.
Why are we doing this? AWS IoT MQTT broker is being used a ābackupā to the Particle cloud (our devices poll every 15 seconds to make sure they are still there, having a 2nd MQTT broker helps reduce the chance of false alarms).
Current findings:
- Certs can fit (just) into the flashed code, but this means we need to have different code for each device. This means we canāt readily use the Particle Products deployment feature and makes code management/maintenance very difficult.
- EEPROM has a limit of 2047 bytes and we use half of it already for storing phone numbers, customer ID.
- Particle.publish limits the number of characters we can input at any given time, but we can upload the cert line by line (tedious) but we again hit the EEPROM size limits.
Would greatly welcome any suggestions or ideas!
Ping @peekay123, who usually has good advice regarding external memory devices.
If you use some sort of external memory attached to your device, then you can expand your non-volatile memory resources (such as another EEPROM of suitable size). If you go that route, then you can probably program in all your stored data including the certificates into the external memory as part of your manufacturing/production process. Your code could then read from the external memory as required.
Side effects of that approach would be freeing up the code space so you can evolve your product, as well as free up EEPROM space which could be used to store data during operation of the device.
I am sure there are other ideas out there; this came to mind first.
1 Like
We did consider adding external storage, but it added significantly to the PCB design/re-certification costs sadly.
Hoping for a software based solution (fingers crossed!)
It may be less than ideal, but how about storing 1 certificate in code and the other certificate in EEPROM?
That means that you want one certificate (the one in code) to be same for all devices.
If your EEPROM if already half-full, this could still be a challenge, assuming a single certificate is 1225 characters.
As you say, code management/maintenance as well as production could be difficult.
However, it might work as a software-only solution.
1 Like
How prone to change are your phone numbers?
Will the user need to change them?
EEPROM uses a lot more flash than the available 2047 for wear leveling. But if you can do without the convenience of the EEPROM layer, you could use the 128K (IIRC) memory page (otherwise used to move the 2047B window around in that area) more efficiently.
flash_storage_impl.h might provide you with some starting points
ScruffRās idea should work. Also, which device? The P1 has additional flash, and the Electron and E series have a spare flash sector. Also, itās possible to store stuff in the u-blox modem flash on cellular devices.
1 Like
The phone numbers may change once every 6months. I wouldnāt expect many writes. We do have a lot of reads (whenever the device restarts). I assume writes only impact āwearā considerations?
Thatās a good idea! 131072 bytes would be more than enough.
I had a look at the library, looks like:
InternalFlashStore.write(1,certInChar, certInCharLength)
is the correct usage? I canāt quite figure out what the address ranges should be though? Any advice on what is āsafeā to use in terms of ranges / no go zones?
We are using the Electron. That's interesting, do you know how we can access the extra flash sector? Is that via the EEPROM commands as well?
thanks!
One of the difficulties with flash is that you can only set a bit to 0. The only way to set a bit back to 1 is to erase the entire sector, which ranges from 16K to 128K depending on the sector. Thereās also the issue of wear, you can only erase the sector a finite number of times, but itās well over 10000 times so thatās not normally an issue.
If youāre writing the whole thing, think updating your user firmware, itās not an issue because the whole sector is written at once. For incremental updates like the way the EEPROM works, it requires two sectors, and the overhead of keeping track of which data is at what location in flash. Thatās why the EEPROM area is so much smaller than the flash sector size. It allows many writes to occur before the flash sector needs to be erased.
I have some code to use the extra flash sector on the Electron, I just need to find it and make sure it still works. Iāll try to do that later today.
2 Likes
Thank you! That would be greatly appreciated.
The certs generated by AWS IoT appear to expire in 10+ years. So I suspect it would be the case of a single write once. The reads would be more frequent though - does that impact anything?
Read actions donāt wear the flash memory.
1 Like
I didnāt think so - but worth checking with the experts
1 Like
I ran into a little problem in that it looks like the system firmware prevents writing to the backup firmware slot from user firmware using the updates API. This is not an unreasonable thing to do, but it limits the usefulness.
However, if you have a situation where you can pre-load the flash by USB DFU, I published that part of the code.
Basically, thereās a tool that allows you to create a virtual file system of files into a single .bin file which you can upload by DFU. Then, thereās a really small bit of code you can use to read these āfilesā from your code.
3 Likes
Damn, so if we needed to reissue the certs weād need to recall the device.
Is there a work around that you know of?
@ScruffR answer IIRC use might be the way to go. But not knowing how that flash memory is used and the safe ranges are making me hesitate until i can find more info on it