[Submission] Flashee-eeprom library: eeprom storage in external flash

Firmware Libraries
1

I’ve created a library that provides eeprom-like access to the external flash. Please see the repo for documentation: https://github.com/m-mcgowan/spark-flashee-eeprom

Some highlights:

  • provides direct flash access and eeprom-like access
  • can use all the external flash - emulates address erasable eeprom over specified regions (up to 1MB per region.)
  • performs wear leveling so erases to the same page are distributed across all pages in the flash device, improving endurance.
  • circular buffer for dumping in data for later retrieval (e.g. buffering data to be sent out over wifi.)
  • streamed read/write interface for convenient saving and loading of data
  • small memory overhead - uses 9 bits of data for each page of flash managed. No overhead for direct access to flash.

I’m rather proud of how this library was developed. :slight_smile:

The library was developed first as a regular C++ GCC library, with unit and integration tests using Google Test. This was possible because of the key FlashDevice abstraction.

I then coded a SparkFlashDevice and compiled the code for the spark. I also ported over ArduinoUnit - let’s call it SparkUnit - and ported the integration tests from Google Test over to SparkUnit so that the same core set of integration tests that I had running on the desktop were now running on the spark. This gives a high confidence that the library is correct.

16 Likes
2

Flashee now supports the file storage via the FAT filesystem!!! You can allocate upto 1MB for a FAT12 filesystem. Since it’s based on the wear leveling layer in flashee, the files are completely rewritable, and there’s no danger of the first few sectors containing the filesystem structures being prematurely worn since the wear will be distributed over the free space.

The FAT filesystem is provided by the excellent fatfs library, I simply had to provide a block device based on the FlashDevice provided by flashee, and some logic to detect if the filesystem needs formatting.

The library contains an example application to experiment with creating, deleting and viewing a file, as well as listing the root directory and showing used/free space in the filesystem.

Here’s an example session - each time the user presses a key the system prints "cmd: ".

FAT filesystem successfully mounted.
Commands:
c - create file abc.txt
d - delete file abc.txt
l - list files and show used/free space
p - print file contents

cmd: create file abc.txt
Created file abc.txt

cmd: delete file abc.txt
Deleted file abc.txt

cmd: delete file abc.txt
Unable to delete file :File not found

cmd: create file abc.txt
Created file abc.txt

cmd: dir
06/22/2014 23:14:50     26 ABC.TXT      
                                                       1007 KiB total drive space.
                                                       1006 KiB available.
    
cmd: print file abc.txt
The world's smallest NAS?

cmd: create file abc.txt
Unable to create file :File exists

cmd: delete file abc.txt
Deleted file abc.txt

Hehe, I think this is pretty cool! :smiley:

So dfu-util doubles as a filesystem backup and restore program! :wink: And since we are using standard FAT partitions, you can use one of the many virtual FAT drivers to create a image file and upload that to the spark with dfu-util to pre-populate your app’s filesystem.

3 Likes
3

@mdma, that is fan-freaking-tastic! I HAVE to adopt you now :stuck_out_tongue:

Seriously though, I can’t wait to try it out on some of my projects. Great job :smile:

3 Likes
4

That’s awesome! :slight_smile: Nice!

5

Thanks @dave. As a follow up, I ported a HTTP server library and built an app to make the spark a real networked file server!

1 Like
6

Do I need to put everything from firmware/ directory if i’m compiling locally and don’t want FAT support - just write like 2 values only? Won’t it increase memory usage?

7

Hi @ryotsuke! Great that you want to use my library! :beers:

The FAT support doesn’t increase memory usage unless you actually use it, but to be sure you can just leave out the ff.cpp file.

Pull the headers into a inc/flashee-eeprom, and pull flashee-eeprom.cpp into your src folder. Edit build.mk, and add flashee-eeprom.cpp to the list of source files. And since you don’t want FAT support then you don’t need to pull ff.cpp.

2 Likes
8

It certainly will simplify page write and erase problems.128K storage option is fine for my data logging application but do you have any example code for writing strings of data to this NV storage area. I tried using the header info but on compilation received an error message “fatal error: flashee-eeprom/flashee-eeprom.h: No such file or directory”.

Many thanks

9

Are you compiling locally or using the CLI/online IDE? In the online IDE, you’ll need to at least add the library to your app before you can use it. If you’re compiling locally, the instructions above describe how you get the sources into your local build.

I hope that helps!

10

Thanks for the advice. I am a relative novice with the ‘Spark’ and am on a steep learning curve. I must profess to being a bit lazy and looking for the easy route. I have been using the online compiler in ‘Build’ but when I am more conversant will try and compile locally. I assume the library can be added with an “include” statement.

11

  1. Hit the “libraries” button

  2. Select “flashee-eeprom”

2 Likes
12

@stevespark, what @kennethlimcp said, but select “flashee-eeprom” as the library to add.

(Of course, you’re welcome to add the unit test library too and write some tests, that would be awesome!)

1 Like
13

Thanks. I think I need “Spark Core for Dummies”! I have a history in hardware, Z80 assembler and C but am getting back up to speed.

1 Like
14

You will do fine. As far as i know, :spark: is the only Wifi OTA Arduino Web IDE owner :smiley:

15

Have you tried using Flashee? Have now found that “Flashdevice* flash = Devices::createAddressErase();” needs to be called within each function before writing or reading data rather than being called just once in setup();

16

Yes, I have tried it. :slight_smile:

You don’t want to do it within each function. You will quickly run out of RAM since you are allocating a new object each time.

Put

FlashDevice* flash;

in global scope - outside of any function - near the top of the file

Then initialize in setup()

void setup() {
    flash = Devices::createAddressErase();
}
1 Like
17

Thanks. I have now done that but the “Getting Started” documentation is a little ambiguous on this and I understood it to mean just enter “FlashDevice* flash = Devices::createAddressErase();” in setup();

P.S. I have compiled my app with this code and it is now an 88kB binary file instead of 84kB. When I flash it seems to go through the motions with the usual high speed flashing of the LED for writing and read verification but then I get the red flashing kiss of death. Could it be that the compiled code is too large?

18

You’re right, the current docs aren’t clear in this regard. I had rewritten the getting started section but didn’t push to the github repo. (D’oh!) But I just pushed the changes now, so hopefully it’s clearer, but let me know if not!

19

Thanks again. Still having trouble with the red flashing after flash of new code. I have stripped everything out to the very basics:-

// This #include statement was automatically added by the Spark IDE.
#include "flashee-eeprom/flashee-eeprom.h"
using namespace Flashee;

static const int MINUTE_MILLIS = 60 * 1000;       // 1 minute milliseconds count


FlashDevice* flash;

void setup() 
{

    flash = Devices::createAddressErase();

    delay(1000);
}



void loop() 
{
static int lastdata = millis();
static int x = 0;

  if (millis() - lastdata > MINUTE_MILLIS)  // every  minute
  {
    lastdata = millis();
    if(x < 59)
        x++;
    else
        x = 0;
    flash->writeString("Hello there!",x*13);
  } 
    delay(1000);    // Delay in loop of 1 second
}
1 Like
20

I’m afraid as far as I know, this is an issue with the online IDE - please see

1 Like