Best way to store preference in flash?

Firmware
1

Hi Guys,

I am currently being a basic device that controls some LED lights, with this I am looking to store some basic variables so they are persistent

For example

INT max_brightness = 80;
INT ambient_limit = 15;
INT mode = 3;

What is the simpliest method that also makes it easy to modify and retrieve the variables? Ideally it would be useful to be able to define default entries should the variable not exist in flash etc?

Thanks :smiley:

2

EEprom? https://docs.particle.io/reference/firmware/photon/#eeprom

3

I was hoping for a bit more of an example of how others are using itā€¦

As a wild guess I suppose this is how it is used?

To read, not sure how this handles defaults? Eg. I may not of even set them yet

int max_brightness = EEPROM.read(1);
int ambient_limit = EEPROM.read(2);
int mode = EEPROM.read(3);

To set would be

EEPROM.write(1, max_brightness);
EEPROM.write(2, ambient_limit);
EEPROM.write(3, mode);
4

In your case it doesnā€™t seem too necessary, but if you want to find out whether your stored data is valid or not, you might want to add some magic number and/or a checksum of all your stored data.
First check for magic num, then read the data and compare checksum and only when ut matches use the data.

5

Hows this looking? Basically the CRC is just the value + a known number, when retrieving the value we remove the known number from the CRC and if both values match then we have a valid value :smile:

#define MAGIC_NUMBER 34
#define CRC_INCREMENT 20
#define MAGIC_INCREMENT 40
#define LOCATION_MODE 0
#define LOCATION_LOW_TEMP 1
#define LOCATION_HIGH_TEMP 2
#define LOCATION_MAX_BRIGHNESS 3
#define LOCATION_MIN_AMBIENT 4
int readFromLocationWithDefaultValue(int location, int default_value) {
  int value = EEPROM.read(location);
  int crc_value = EEPROM.read(location + CRC_INCREMENT);
  if (value == (crc_value - MAGIC_NUMBER)) { // Check if valid entry, return value
    return value;
  } else { // CRC Failure, return default value
    return default_value;
  }

}

int writeToLocationWithValue(int location, int value) {
  EEPROM.write(location, value); //Write Value
  EEPROM.write(location + CRC_INCREMENT, value + MAGIC_NUMBER); //Write CRC version
}
6

You can do it that way, but I thought of something like this (although there are thousends of other ways and it depends on your use case - e.g. when/how often do you rewrite the config data, only sinlge items or all at once, ā€¦)

Untested, since it only should show the idea :wink:

#define CFG_BASE 0

struct {
  int     magicNumber = 0xC0FFEE; // this should change when struct changes
  uint8_t mode;
  int8_t  tmpLow;
  int8_t  tmpHigh;
  uint8_t brightness;
  uint8_t ambient;
  uint8_t padding[] = { 0, 0, 0 };  // fill to four byte boundaries
                                    // otherwise consider "#pragma pack(1)"
  int     checkSum = 0;
} cfgData;

bool readCfg()
{
  uint8_t* _data     = &cfgData;
  int      _checkSum = 0;

  for (int i = 0; i < sizeof(cfgData); i++)
  {
    _data[i] = EEPROM.read(CFG_BASE + i);
    if (i < sizeof(cfgData) - sizeof(_checkSum))
      _chkSum += (_data[i] << i);
  }

  return (_chkSum == cfgData.checkSum);
}

int writeCfg()
{
  uint8_t* _data = &cfgData;
  cfgData.checkSum = 0;

  for (int i = 0; i < sizeof(cfgData); i++)
  {
    EEPROM.write(CFG_BASE + i, _data[i]);
    if (i < sizeof(cfgData) - sizeof(cfgData.checkSum))
      cfgData.checkSum += (_data[i] << i);
  }

  return cfgData.checkSum;
}
7

I recently posted some code to do just this. It doesnā€™t use a fancy CRC, but seems to work pretty well. It checks to see if memory location 0 has 137 to determine whether it has been initialized or not.

Check out the setup() function for reading and initializing the values in EEPROM and the fnRouter() function for updating the EEPROM based on a cloud function call.

1 Like
8

And not forgetting the relatively new EEPROM get and set methods, so you can store any type of object in eeprom without having to break it down into bytes first.

2 Likes
9

@mdma: Excellent. I had missed that. Comes in very handy. In that context: What does the EEPROM.update(ā€¦) do in contrast to EEPROM.write()?

10

Okay, got it. It writes only, if the value is different. Thatā€™s neat!

1 Like
11

Iā€™m surprised the docs are not up to date. I know the docs have been written, but the changes probably still as a closed PR after the docs structure was radically changed. Iā€™ll look into getting the new docs for EEPROM published.

2 Likes
12

Sounds great, thanks!

13

Would these new functions be available in the current Spark Core firmware? Any chance of a quick example of the new functions and how to best use them? :slight_smile:

14

I am not sure if they are (yet) available for the Core. But the usage is very simple:

int value = 0;
EEPROM.get(12, value);

Will read an integer from address 12.

EEPROM.put(12, value);

Will write it. And it automatically adapts to the type, basically doing a kind of memcpy to the EEPROM.

15

@ScruffR, what do you think about this implementation? The main goal was not to have firmware updates that add settings clobber old settings. It does require that all new settings be added to the end of the struct, but thatā€™s a small concession, I think.

What are your thoughts?

#define SETTINGS_START 0

struct {
  uint32_t version; // Number of bytes in the rest of the struct
  uint32_t snooze;  // snooze length in ms
  float timezone;   // this should be self explanitory
  uint32_t checkSum = 0;
} settings;

void setup() {
  // Read configuration from Flash
  if(!readSettings()) {
    settings.snooze   = 10 * 60 * 1000; // 10 minutes
    settings.timezone = -6.0;           // CST
  }
  Time.zone(settings.timezone);
  // use settings.snooze here
}

bool readSettings() {
  uint8_t offset = sizeof(settings.version);
  uint8_t *data = (uint8_t*) &settings;
  uint32_t checkSum = 0;
  uint32_t size;

  EEPROM.get(SETTINGS_START, size);

  if(size > EEPROM.length()) return false; // bad uninitialized catch
  
  for(uint8_t i = 0; i < size; i++) {
    uint8_t index = offset + i;

    data[index] = EEPROM.read(SETTINGS_START + index);

    if(index < settings.version - sizeof(settings.checkSum)) {
      checkSum += (data[index] << i);
    }
  }

  return (checkSum == settings.checkSum);
}

void writeSettings() {
  uint8_t offset = sizeof(settings.version);
  uint8_t *data = (uint8_t*) &settings;
  uint32_t size = sizeof(settings) - offset;

  settings.checkSum = 0;

  EEPROM.put(SETTINGS_START, size);
  
  for(uint8_t i = 0; i < size; i++) {
    uint8_t index = offset + i;

    EEPROM.update(SETTINGS_START + index, data[index]);

    if(index < size - sizeof(settings.checkSum)) {
      settings.checkSum += (data[index] << i);
    }
  }
}
16

Looks good to me, but I havenā€™t really tested it - just read the code :wink:
One note:
You might want to add some range checking, since virtual EEPROM only can store up to 100 byte.
So depending on your growth rate of settings, you might sooner or later hit that limit.

1 Like
17

I guess I forgot to mention that Iā€™m using this code on a Photon. I havenā€™t tested it either, because I finished it so late last night and Dev gave me ā€œbuild didnā€™t produce binaryā€ but compiling locally does so Iā€™m a bit confused by that. lol

Thanks for looking at it, @ScruffR. I appreciate it!

18

It builds fine on Particle Build, so Iā€™m convinced itā€™s one of Devā€™s jokes on users :wink:

1 Like
19

I found this topic useful, and it pointed me in the right direction. Here is a tested CRC8 checksum implementation (often used for this purpose) that others may find useful. I use it with EEPROM.get(0,p) and EEPROM.put(0,p) to retain a list of phone number data, in this instance.

// nb. max. allowed size is 2047 bytes in EEPROM (flash emulated)
#define MAX_EEPROM 2047

#define PL_EEPROM_ADDRESS 0
#define PL_MAX_COUNT 8
#define PL_MAGIC 0xDABBAD00

struct pnStruct{     // telephone number list data
	uint8_t role;    // 0 master number, 1 executive, 2 status (gets unsolicited status)
	char phone[15];  // +614nnnnnnnn/04nnnnnnnn
};

// nb. must be a multiple of 4 bytes (32bit data space), so use spare or padding to ensure this or crc will fail
struct retainedStruct{
	uint32_t magic=PL_MAGIC;    	//4 bytes
	uint16_t  spare;				//2 byte
	uint8_t  count;				  	//1 byte
	pnStruct List[PL_MAX_COUNT];	//16*8=128 bytes
	uint8_t checkSum;		   		//1 byte
};

// Local copy of EEPROM data, retrieved/stored using EEPROM.get/put in calling code
retainedStruct p;


// CRC8 linear generator
// Call with data object (eg. byte, string, struct, array) and initial CRC value
// Successive calls calls will also accumulate valid final CRC if last value is fed in
// Based on examples: http://www.ccsinfo.com/forum/viewtopic.php?t=26264
uint8_t crc8(uint8_t *data, unsigned int sizeofdata, uint8_t crc){
    uint8_t i;
    for (int j=0; j < sizeofdata; j++){
        // Log.trace("crc8: data[%d]: %u", j, data[j]);
        i=(data[j] ^ crc) & 0xff;
        crc=0;
        if(i &  0x1) crc ^= 0x5e;
        if(i &  0x2) crc ^= 0xbc;
        if(i &  0x4) crc ^= 0x61;
        if(i &  0x8) crc ^= 0xc2;
        if(i & 0x10) crc ^= 0x9d;
        if(i & 0x20) crc ^= 0x23;
        if(i & 0x40) crc ^= 0x46;
        if(i & 0x80) crc ^= 0x8c;
    }
    return crc;
}

// calculate a checksum to confirm persistent data is valid
bool isValidChecksum(retainedStruct* p){
    Log.trace("isVldCkSum: sizeof(p): %d", sizeof(*p));

    uint8_t crc=crc8((uint8_t*) p, sizeof(*p)-1, 0xFF);

    Log.trace("vldCkSum: %u", crc);
    return (crc == p->checkSum);
};

// calculate a new checksum for persistent data
void updateChecksum(retainedStruct* p){
    Log.trace("updCkSum: sizeof(p): %d", sizeof(*p));

    uint8_t  crc=crc8((uint8_t*) p, sizeof(*p)-1, 0xFF);

    Log.trace("updCkSum: %u", crc);
    p->checkSum=crc;
}
1 Like