Best way to transfer 1k-2k from APP into EEprom?

Is this an app you'll be writing yourself? Do these OSs not also come with TCP transport libraries?
But sure you can break things up into chunks, but for binary data you'd need to encode it into something like Base64 or Base85.

@ScruffR, I should said I don’t have much experience with TCP so coding maybe time consuming. To save on time, I was thinking the fastest way to get this project done. I was thinking how to encode in the app and decode it in the device to make it useful and save space. What I want to do is store a 7 day schedule set by the user into the EEProm. I calculated the EEProm space, it can store the data if I limit each day 255 bytes which make it perfect for the limit on the particle function call. Do you know any example to encode a timestamp into two bytes? The timestamp is day+time (example: Mon+1:00). I have few more bytes reserve for other data. I am thinking timestamp+reserve bytes = 5 bytes per entry. I hope its not process hogging in the p1 module for decode when I need to use the store value.

The data in a function call can only be 63 characters. The data in an event (Particle.publish) can be 255 bytes.
Do you need 1 minute resolution on the time (or for instance, could you get by with 5 or 10 minute resolution)?

@Ric, Thanks for correcting me, I will be using Particle.publish. I will be using 30 minute each entry, 1:00, 1:30, 2:00, 2:30… Base on my calculation, only 30 minute interval will fit into the available EEPROM, since I will be adding 2-3 bytes after the set timestamp.

Base on my math for 30 minute interval, there is 48 entry per day x 7 day = 336 entry for a 7 day schedule. Each entry used 5 bytes, hoping 2 bytes for the timestamp and 3 for other data, 336 entry x 5 bytes = 1,680 bytes which can fit into EEProm ( 2,047 bytes). Tricky part is converting the timestamp into bytes.

You should be able to encode the timestamp into two ascii characters. The hour (0-23) takes 5 bits, and the minute (0 or 30) would only take 1. The day of the week, expressed as a number 0-6 would only take 3 bits. So, for the time in pseudo code,

timeValue = hour + (0 if on the hour, 32 if on the half hour)
asciiValue = timeValue + 33 (the + 33 gets you up to the first printable character)

For the day of the week, you just need to add 33 to the day number (0=Sunday, 1=Monday, etc.)

@Ric, from your pseudo code, it seem like the time take one ascii character. Please correct me if I am wrong, that mean I can leave the day of the week as 0-7 with out encoding and still give me 2 bytes. This will save some decoding on the device side. All I need is decode the time. Does this seem right?

Yeah, that’s true, you can just send the day byte as one of the characters, ‘0’ through ‘6’. That would be simpler.

@sheng, what is in “the other 3 bytes”? Are these ascii values?

@peekay123, the structure of one entry would look like “day,time,mode,temperature”. Example: Monday, 1:00, On, 75 F would be “1”,“ascii Char”,“1”,“75” = 5 bytes. I will have to map the ascii value to ascii char before I send to the device. Would this work or other way to accomplish this?

Yes this should work fine. You could cut it down to 4 bytes if that would be useful. To do that, you could encode the day using 3 bits as I first suggested, and use the 4th bit in that byte to encode the mode. If you wanted finer grain control over the time, you could use the 5th and 6th bits to encode the minute (giving you four values, so :00, :15, :30, and :45). In that case, you wouldn’t encode the minute in the hour byte.

If your range of temperatures isn’t greater than 94 degrees, you could even go down to 3 bytes, since the temperature could then be encoded in 1 printable ascii character.

@Ric, Thank you for the help and suggestion. :slight_smile:

Hi,

I put together a simple test case to make sure the EEprom works and stored after power loss.

Below is the handler for the Particle.subscribe:

char sunSched[240];
int dayflag;

  void schedHandler(const char *event, const char *data)
{
  Serial.println("String Data received");
  Serial.println (data);
  dayflag = String(data[0]).toInt();
  switch(dayflag)
  {
    case 1:
      Serial.println("Pre Stored value");
      Serial.println(sunSched);
      EEPROM.get(0, sunSched);
      Serial.println("test reset Stored Value");
      Serial.printlnf(sunSched);
      EEPROM.put(0, data);
      EEPROM.get(0, sunSched);
      Serial.println("After Stored Value");
      Serial.printlnf(sunSched);
      break;
    default:
      Serial.println("default");
      break;
  }

}

Output:

Opening serial monitor for com port: "COM3"
String Data received
1"175
Pre Stored value

test reset Stored Value
�g
After Stored Value
�g
publish success
String Data received
1"4566778889
Pre Stored value
�g
test reset Stored Value
�g
After Stored Value
�g
String Data received
1"4566778889
Pre Stored value
�g
test reset Stored Value
�g
After Stored Value
�g

It seem like the value is being store even after a power lose. Is there anyway to make the get value from the EEProm back to readable char? I hope I am doing the right way of using put and get. Thanks

It’s hard to evaluate your results without knowing what you had “pre-stored”, and what data you sent in the publication(s). The black diamond with the question mark is what you get when you try to print something not in the ascii character set.

The point here is that your received const char* data is not a char[240] and hence EEPROM.put() will not store the complete string but rather the pointer address, since data is a pointer.
Also, when putting a string into EEPROM you need to ensure you include the zero-terminator otherwise you’ll also get all non-zero bytes printed that follow your actual string.

The code in the first comment to this post might help explain how the EEPROM functions work with integers and strings, as it's not entirely obvious.

1 Like

@Ric, the test setup, device first power up this mean sunSched[240] is empty when init as shown below. The string that is Particle.publish by the app is 1"175 for the first run. test reset Stored Value show what happen when I rest the device with the reset button (to test if the data is retain in the EEprom). After Stored Value, test what is stored in the EEprom.

String Data received
1"175
Pre Stored value

test reset Stored Value
�g
After Stored Value
�g
publish succes

That’s why I referenced the post above. And why ScruffR said what he said. The way you are saving the string is not how to save a string in EEPROM. You need to specify the pointer to the data and the length of the data to get or put. You can’t just pass a pointer to a c-string, it doesn’t work that way.

2 Likes

@rickkas7, I actually red the referenced post before I make up the test case but I guess I didn’t fully understand it since I have not touch pointer for a long time. I am revisiting the referenced post and see if I can make more sense out of it. Could you help me out with this example code? If I am publishing a string from the IOS APP say 1"175 and getting to the subscribing handler const char *data. How I can store this in the EEprom? It seem like in the referenced post you convert the data back to String before you write to the EEprom.

If you simply want to store a single value that you are sending with a publish, then you can just copy data into sunSched, and use EEPROM.put to store it. There’s no need to convert data into a String object. Here is an example of one way to do that,

char sunSched[240];
int dayflag;

void setup() {
    Particle.subscribe("schedTest", schedHandler);
    Serial.begin(9600);
    delay(3000);
    EEPROM.get(0, sunSched);
    Serial.printlnf("sunSched (from setup) is: %s", sunSched);
    delay(3000);
}

void loop() {}

void schedHandler(const char *event, const char *data) {
    strcpy(sunSched, data); // be careful that data is not bigger than sunSched
    dayflag = atoi(data);
    switch(dayflag) {
        case 1:
            Serial.println("storing new value");
            EEPROM.put(0, sunSched);
            break;
        default:
            Serial.println("default");
            break;
    }
}

The first time you run this, the print out will be whatever happens to be in the EEPROM at the time, but it should reflect the value you send in the publish the next time you run it.

However, I suspect that you want to send more than one value at some point since you’re using a switch statement. In that case, you need to know where to store the next value. A simple way to do that would be to always send the same size data, so you can just increment the location in the eeprom where you store or read the data. We could give you more detailed advice if we knew what you actually intend to do.