Using EEPROM file on Argon

Yurrtel · March 18, 2020, 12:25am

Hi there, hope you are all well. I’m having trouble using the EEPROM file on the Argon. I’m trying to have it so that whenever connection drops between my Argon and a local python server, the Argon starts saving sensor data to the emulated EEPROM instead. However, upon reconnection, when the argon sends the struct containing the sensor data, the data isn’t being received correctly by the server.The server and all the rest of the code works fine. Here’s the code:

bool storedData;
int eepromaddr = 0;
.
.
.
typedef struct {
    long time_read;
    float tempr;
    float pressure;
    float humidity;
} sensorReading_t;

sensorReading_t reading;
.
.
.
void loop() {
    if (client.connected()) {
        while (storedData == true)
        {
            //read data
            EEPROM.get(eepromaddr, reading);
            if (client.available()) {
                char c = client.read();
                Serial.println(c);
            }
            client.write((uint8_t*)&reading, 16);
            if (eepromaddr == 0){
                EEPROM.clear();
                storedData = false;
            }
            else{
                eepromaddr -= 16;//decrement by 16 until address is 0 - or can multiple bytes be stored in on adddress???
            } 
        }
.
.
.
if (client.available()) {
            char c = client.read();
            Serial.println(c);
        } else{
            storedData = true;
            client.stop();
            Serial.println("connection dropped");
            time_t time = Time.now();
            reading.time_read = time;
            reading.tempr = bme280.getTemperature();
            reading.pressure = bme280.getPressure();
            reading.humidity = bme280.getHumidity();
            EEPROM.put(eepromaddr, reading);
            eepromaddr+=16; //increments address by 16 until EEPROM is empty
            if(client.connect(server, port)){
                Serial.println("reconnected");
            }
            else { 
                Serial.println("connection failed");
            }

On the server side it complains that the time isn’t valid to transform to UTC so I assume the data is being sent incorrectly, or saved incorrectly in the EEPROM. I clear the EEPROM in the setup so I don’t think there should be anything there? Am I using the EEPROM class correctly or this just my lack of C knowledge? I’m assuming the problem is with the addressing. Thanks in advance!

ScruffR · March 18, 2020, 1:30am

Before looking at your code, here is what you can/should do in such cases

double check the values you are going to write to EEPROM before putting it there
read back the just saved values and compare with what you found out before
check on the server side what exactly it receives (as individual bytes rather than multi byte numbers) and compare with what you sent

It’s not enough to know that something is wrong. You also need to know (and consequently tell) in what way the data is wrong.
This comparison may already provide some clues about the possible reason for the error.

BTW, when you want to save yourself some headache use sizeof(reading) instead of hardcoding 16 for your struct
You should also ensure that you never write more data to EEPROM than you have space. Currently your code doesn’t seem to do any boundary checks.

Yurrtel · March 18, 2020, 3:45pm

Okay I’ve implemented your advice concerning checking the data before and after saving to EEPROM. The data is fine before being stored but not after removing from memory. I’ll attach the serial monitor output when it disconnects and reconnects, and I’ve re-organised the code so that the writing to eeprom and upload to server are in separate functions , as shown below. What I’m thinking is: can I write a full struct to each address on the EEPROM? and my client.connected() always seems to return true, I have to rely on a tcp write error to notice disconnection.

void loop() {
    if (client.connected()) {
        if (storedData == true){
            writeToEEPROM();
        }
        //get and print sensor values
        time_t time = Time.now();
        reading.time_read = time;
        reading.tempr = bme280.getTemperature();
        reading.pressure = bme280.getPressure();
        reading.humidity = bme280.getHumidity();
        
        client.write((uint8_t*)&reading, 16);
        Serial.println("still connected");
        int err = client.getWriteError();
        if (err != 0) {
            Serial.println("disconnected due to write error");
            writeToEEPROM();
        } 
        delay(1000);
    }
    else {
        Serial.println("disconnected per client.connected");
        writeToEEPROM();
    }
    serialDebug();
}

void writeToEEPROM() {
    storedData = true;
    client.stop();
    Serial.println("connection dropped");
    time_t time = 1400647897;
    reading.time_read = time;
    reading.tempr = bme280.getTemperature();
    reading.pressure = bme280.getPressure();
    reading.humidity = bme280.getHumidity();
    EEPROM.put(eepromaddr, reading);
    Serial.print("reading before storing: ");
    Serial.println(reading.time_read);
    eepromaddr+=1;
    if(client.connect(server, port)){
        Serial.println("reconnected");
        uploadReadings();
    }
    else { 
        Serial.println("connection failed");
    }

}

void uploadReadings() {
    while (storedData == true)
        {
            //read data
            sensorReading_t saved_reading;
            saved_reading = EEPROM.get(eepromaddr, reading);
            Serial.print("time read: ");
            Serial.println(saved_reading.time_read);
            client.write((uint8_t*)&saved_reading, 16);
            if ((eepromaddr == 0)||(eepromaddr < 3000)){
                EEPROM.clear();
                storedData = false;
                eepromaddr = 0;
            }
            else{
                eepromaddr -= 1;
            } 
        }
}

(I fed the write to EEPROM a dummy time for simplicity)

serialoutput
The time read seems to be quite random, it might return signed longs aswell. The sensor values are similar. Thanks for looking at this!

ScruffR · March 18, 2020, 4:39pm

You are battling on multiple fronts and you seem to be unable to determine which is causing the issue.
So my next suggestions would be to entirely separate the EEPROM stuff from your TCP stuff - best in two independent test projects each laser focusing on the one issue.

I also don’t quite understand this line

saved_reading = EEPROM.get(eepromaddr, reading);

You are reading the data into reading but then check what you get in saved_reading.
You’d either read directly into saved_reading or check what you got in reading.
I don’t see the reason for the “double” assignment.

You are also incrementing eepromaddr by 1 and not the size of your stored struct (you had that correct in your original code) and then you are reading from an already incremented but not yet populated location in EEPROM.

And you are always clearing EEPROM in uploadReadings() as your eepromadr will virtually always be < 3000

BTW, instead of calling EEPROM.clear() you could use a separet EEPROM based struct that holds the first and last index of valid data and use that to keep track of the state in your “repository”.

shanevanj · March 18, 2020, 5:40pm

You could also look up and excellent library on github ‘edb’ - this manages a simple database in EEPROM or SD card and takes card of al to of the issues you may be facing here

Yurrtel · March 18, 2020, 7:46pm

Separating the TCP component and EEPROM component for testing is something you’d think I would have thought of haha. Okay so I tested the EEPROM writing and reading on its own and that’s where it’s going wrong. Here is the output next to my loop:

And here are my write and read functions:

void writeToEEPROM() {
    
    EEPROM.put(eepromaddr, reading);
    Serial.print("reading before storing: ");
    Serial.println(reading.time_read);
    eepromaddr+=1;
}

void readEEPROM() {
            EEPROM.get(eepromaddr, reading);
            Serial.print("time read: ");
            Serial.println(reading.time_read);
            //client.write((uint8_t*)&reading, 16);
            if ((eepromaddr == 0)||(eepromaddr > 3000)){
                EEPROM.clear();
                //storedData = false;
                eepromaddr = 0;
            }
            else{
                eepromaddr -= 1;
            } 
}

Can a 16 byte struct be stored at say address 0, then another one at address 1 or is every address 1 byte? Im assuming the former hence my incrementing and decrementing by 1. I didn’t quite understand your last message in the previous comment so I haven’t implemented that. Is it possible for me to use @shanevanj 's suggested library if I have no access to external EEPROM/sd card etc?

peekay123 · March 18, 2020, 8:03pm

@Yurrtel, one small item. I noticed that you declare time_read as a long when in fact, it should be an unsigned long or uint32_t.

Yurrtel · March 18, 2020, 8:19pm

Oh thanks I forgot to double check that. At least now I can reconnect without the timestamp converter on the python server closing the server when I put the TCP back in. I added in the humidity info aswell and the output looks like this:

It’s frustrating because the humidity was correctly read once.
Can anyone confirm that I’m using the EEPROM addressing correctly? Has anyone had trouble using the argon EEPROM file in the past? I couldn’t find anything similar when searching.

EDIT: Looking at that output closer I can see that the second to last humidity reading is actually the pressure reading in Pascals. Also I’m not writing ‘ovf’ anywhere so I’m not sure what that means.

ScruffR · March 18, 2020, 11:26pm

The documentation of EEPROM.put() should give you the answer to that
https://docs.particle.io/reference/device-os/firmware/photon/#put-

Let's assume you are writing 16 bytes at EEPROM.length()-1 (which is the last possible position) where would the extra 15 bytes go?
Does this answer the question?

Why would you? Based on what?
But even if it was, why not go with the suggested 16 byte increment (just for the fun of testing)?

And your first reading in (readEEPROM) will still be reading an unpopulated entry in EEPROM.
You write to 0, increment to 1 and then read from 1 - what do you expect to read?

Sure you can, but understanding the basics before using "black-box-magic" will help you more in the long run.

That means overflow and most likely comes from reading garbled data (due to bad addressing) into a float.

Yurrtel · March 19, 2020, 12:24am

Okay you’re right its obvious its a single byte per address so I changed it to increment and decrement by 16. I also stopped that extra increment from happening and its working now thanks to you. Thanks so much for your help now I should be able to figure out how to implement it into my original firmware. I’ll be back tomorrow when I inevitably can’t figure out whats going wrong with the TCP.

system · September 17, 2020, 12:24pm

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Application ceases during EEPROM write on Argon Firmware argon	7	492	May 18, 2019
Photon2 EEPROM persistent data Firmware	4	191	July 14, 2023
What happens to the UART connection of Argon while the device try to reconnect to WI-Fi? Hardware argon	6	481	March 16, 2022
Storing data using a struct in EEPROM General	5	2115	July 17, 2018
Help with EEPROM Troubleshooting	4	594	September 19, 2019

Using EEPROM file on Argon

Related topics