Sorry if this question has already been asked, I couldn’t find anything related in my quick search, feel free to point me to any existing answer.
So, we’re planning on switching one of our major project using an E-series (gen2) to a gen3 version (either E404X or Boron). We’ve been using SPIFFS for a few years now (thanks @rickkas7 for the Particle adaptation of the driver + support), but we encountered a lot of corruption issues with it and are now planning to move to LittleFS as per Particle’s recommendation. Note we use a 128MB external flash with it, but I lately updated our firmware to only use 16MB only on it, due to possible issues with the large size.
My question is: if we move now to LittleFS on the gen2 processor, it means using an external LittleFS lib (LittleFS-RK, thanks again @rickkas7 !) with it, but when we upgrade to gen3, will there be a conflict with internal LittleFS implementation in the deviceOS? (the 2MB FS offered would be too small for our project). Will I need to / Is it possible to disable the internal LittleFS completely?
Also, I have another project in the works, where we’ll likely be using a P2 (we started on an Argon) and may need to also add an external memory (nothing decided yet), so the question applies here too.
There is currently no way to make LittleFS-RK work on Gen 3 as-is because of the conflicting definitions. You definitely can’t disable the internal definitions, however I can probably make it work using a namespace. I’ll check.
That’s bad news… Finger crossed there is a workaround… From what I read SPIFFS should be deprecated at some point, in favor of LittleFS. Then I’ll still have the option to directly manage mem myself, but I’d have to implement some kind of wear & tear handling system, and that seems a bit heavy for my simple use (I mostly need less than 10 (large) files on the FS for the first project, and maybe 3 or 4 on the second project…) Actually direct mem access would be the simplest if there was no wear & tear…
I tried to derive a version of LittleFS-RK for external men and testing on a Boron. I access properly the external chip (16MB, Winbond, SparkX board) and can confirm the JEDEC, but littleFS failed to mount and format with a -52 corruption error. I tried to do a chipErase before mounting, but still get that error. Any clue what the issue could be? Any way to confirm my littleFS derived class is actually accessing the external chip and not the internal mem?
I took a quick look at it over the weekend and one problem is that the inner functions in LittleFS in Device OS are visible from user firmware, which I did not expect. The easiest solution is to make every file .cpp and then encapsulate the LittleFS functions in a namepsace. However, you’ll also find that you get a pile of ambiguous namespace errors. This is actually good, because it indicates every place where the library could pick up the wrong definition, but it does mean having to manually modify all of those locations.
You can also wrap the POSIX wrapper in a namespace, and that seems to work.
The only other thing I immediately noticed is that there are some internal Device OS files that I replicated in the library. Those need to be conditionally compiled only for gen 2, or if you’re making a completely separate library, you can probably just remove them.
Yeah I thought about changing the .c to .cpp and add namespaces.
So far the test compiles fine with the changes I made, but yes, I suspect some calls affect the internal FS instead of the external one… I might well have re-formatted the internal FS at this point
I’ll continue investigating. I don’t want to move our existing gen2 devices to littleFS if I can’t make this work on gen3 because we have a bunch of upgrades coming this summer and I’d like to keep some compatibility between the devices… (limited since we were always stuck with the 128KB on gen2 and moving to gen3 will open a lot of space for new features - even though I somehow feel our device already has a monster firmware…)
I think I have it working eventually I finally converted lfs.c (called lefs.c for this fix) to lefs.cpp. Had to fix a bunch of stuff the CPP compiler didn’t like.
So, my initial test failed, using the large files test from the lib, and it stopped under 2MB, which was the sign I was actually working on the internal FS. Then I realized I was using the posix functions from the code (open, read, …) yet I had to use the new functions lefs_open, etc…
After fixing the calls, I’ve run the test again and it seems to work, but external memory (16MB) is soooo much slower… I’ve tried passing different block sizes when initializing the littleFS instance with no obvious changes in timings.
E.g. one 256KB file written on the internal FS takes 25s (512 blocks of 512 bytes as defined in the code)
whereas the same file on the external FS takes 170s !!
Just realized the timings represent n blocks x ( write + seek + read) for a total of 256KB of data
Just did some timing for a 16384 bytes write in one call (FS block = 4096B) and it takes 1.3s
So, not so bad eventually. I think I can handle that. One benefit of LittleFS is the supposed redundancy, so re-reading afterwards may not be necessary, right?
I pushed the code, it’s not properly a library currently and there’s only a benefit of using it on a gen3 device, since the posix calls are not mapped and you need to call specifically lefs_xxx instead of the xxx posix functions (open, read, write, close…). Calling the posix functions should access the internal LittleFS instead, so you can still use both if necessary.
OK, so I posted a new library, called LittleExtFS, available as a community library from the CLI or others. I also used a slightly modified version of SpiFlashRK (called SpiFlashPG since I had to change the name to publish it) that runs on bigger chips (it’s technically a version 0.0.10 of SpiFlashRK, but I have no clue if the changes are backwards compatible, so I preferred to make it a new one, somehow). Modified fork here: SpiFlashPG
There’s an example in it, called little-test.cpp that mostly writes a bunch of 256KB files and then deletes them. It’s derived from Rick’s LargeFileTest, but only doing the writes, which will mostly confirm the size of your formatted chip. It also shows how much time is required to write each 16KB block.
I finally converted