Best way to transfer 1k-2k from APP into EEprom?

[sheng]

Hi,

I am trying to find the best way to transfer 1k-2k data from IOS/Android app into the p1 module EEprom. Could this be done using the cloud with out going to ftp/https? Particle.publish() can do 255 bytes and particle.function() args give 63 chars. Is there any other ways? Thanks.

[ScruffR]

Direct TCP would probably be the best suited way.

[sheng]

@ScruffR, unfortunately, I don’t have direct TCP option open in this project. The project only consist of ios/android app to Particle cloud to p1 module. The Particle cloud is the middle man. Since the function of Particle cloud is limited to 255 bytes per call, I think the best scenario is to break the App data into 255bytes chuck and do multi send. Please correct me if I am wrong. Thanks.

[ScruffR]

@sheng

The project only consist of ios/android app to Particle cloud to p1 module

Is this an app you'll be writing yourself? Do these OSs not also come with TCP transport libraries?
But sure you can break things up into chunks, but for binary data you'd need to encode it into something like Base64 or Base85.

[sheng]

@ScruffR, I should said I don’t have much experience with TCP so coding maybe time consuming. To save on time, I was thinking the fastest way to get this project done. I was thinking how to encode in the app and decode it in the device to make it useful and save space. What I want to do is store a 7 day schedule set by the user into the EEProm. I calculated the EEProm space, it can store the data if I limit each day 255 bytes which make it perfect for the limit on the particle function call. Do you know any example to encode a timestamp into two bytes? The timestamp is day+time (example: Mon+1:00). I have few more bytes reserve for other data. I am thinking timestamp+reserve bytes = 5 bytes per entry. I hope its not process hogging in the p1 module for decode when I need to use the store value.

[Ric]

@sheng

255 bytes which make it perfect for the limit on the particle function call

The data in a function call can only be 63 characters. The data in an event (Particle.publish) can be 255 bytes.
Do you need 1 minute resolution on the time (or for instance, could you get by with 5 or 10 minute resolution)?

[sheng]

@Ric, Thanks for correcting me, I will be using Particle.publish. I will be using 30 minute each entry, 1:00, 1:30, 2:00, 2:30… Base on my calculation, only 30 minute interval will fit into the available EEPROM, since I will be adding 2-3 bytes after the set timestamp.

[sheng]

Base on my math for 30 minute interval, there is 48 entry per day x 7 day = 336 entry for a 7 day schedule. Each entry used 5 bytes, hoping 2 bytes for the timestamp and 3 for other data, 336 entry x 5 bytes = 1,680 bytes which can fit into EEProm ( 2,047 bytes). Tricky part is converting the timestamp into bytes.

[Ric]

You should be able to encode the timestamp into two ascii characters. The hour (0-23) takes 5 bits, and the minute (0 or 30) would only take 1. The day of the week, expressed as a number 0-6 would only take 3 bits. So, for the time in pseudo code,

timeValue = hour + (0 if on the hour, 32 if on the half hour)
asciiValue = timeValue + 33 (the + 33 gets you up to the first printable character)

For the day of the week, you just need to add 33 to the day number (0=Sunday, 1=Monday, etc.)

[sheng]

@Ric, from your pseudo code, it seem like the time take one ascii character. Please correct me if I am wrong, that mean I can leave the day of the week as 0-7 with out encoding and still give me 2 bytes. This will save some decoding on the device side. All I need is decode the time. Does this seem right?

[Ric]

Yeah, that’s true, you can just send the day byte as one of the characters, ‘0’ through ‘6’. That would be simpler.

[peekay123]

@sheng, what is in “the other 3 bytes”? Are these ascii values?

[sheng]

@peekay123, the structure of one entry would look like “day,time,mode,temperature”. Example: Monday, 1:00, On, 75 F would be “1”,“ascii Char”,“1”,“75” = 5 bytes. I will have to map the ascii value to ascii char before I send to the device. Would this work or other way to accomplish this?

[Ric]

Yes this should work fine. You could cut it down to 4 bytes if that would be useful. To do that, you could encode the day using 3 bits as I first suggested, and use the 4th bit in that byte to encode the mode. If you wanted finer grain control over the time, you could use the 5th and 6th bits to encode the minute (giving you four values, so :00, :15, :30, and :45). In that case, you wouldn’t encode the minute in the hour byte.

If your range of temperatures isn’t greater than 94 degrees, you could even go down to 3 bytes, since the temperature could then be encoded in 1 printable ascii character.

[sheng]

@Ric, Thank you for the help and suggestion.

[sheng]

Hi,

I put together a simple test case to make sure the EEprom works and stored after power loss.

Below is the handler for the Particle.subscribe:

char sunSched[240];
int dayflag;

  void schedHandler(const char *event, const char *data)
{
  Serial.println("String Data received");
  Serial.println (data);
  dayflag = String(data[0]).toInt();
  switch(dayflag)
  {
    case 1:
      Serial.println("Pre Stored value");
      Serial.println(sunSched);
      EEPROM.get(0, sunSched);
      Serial.println("test reset Stored Value");
      Serial.printlnf(sunSched);
      EEPROM.put(0, data);
      EEPROM.get(0, sunSched);
      Serial.println("After Stored Value");
      Serial.printlnf(sunSched);
      break;
    default:
      Serial.println("default");
      break;
  }

}

Output:

Opening serial monitor for com port: "COM3"
String Data received
1"175
Pre Stored value

test reset Stored Value
�g
After Stored Value
�g
publish success
String Data received
1"4566778889
Pre Stored value
�g
test reset Stored Value
�g
After Stored Value
�g
String Data received
1"4566778889
Pre Stored value
�g
test reset Stored Value
�g
After Stored Value
�g

It seem like the value is being store even after a power lose. Is there anyway to make the get value from the EEProm back to readable char? I hope I am doing the right way of using put and get. Thanks

[Ric]

It’s hard to evaluate your results without knowing what you had “pre-stored”, and what data you sent in the publication(s). The black diamond with the question mark is what you get when you try to print something not in the ascii character set.

[ScruffR]

The point here is that your received const char* data is not a char[240] and hence EEPROM.put() will not store the complete string but rather the pointer address, since data is a pointer.
Also, when putting a string into EEPROM you need to ensure you include the zero-terminator otherwise you’ll also get all non-zero bytes printed that follow your actual string.

[rickkas7]

The code in the first comment to this post might help explain how the EEPROM functions work with integers and strings, as it's not entirely obvious.

Need explanation for EEPROM usage (generally and on Photon)

I’m lacking some basic understanding on how the EEPROM on the Photon can be used. As I understand there are 2047 bytes of date accessible as permanent storage. Every byte can be addressed by an address as an integer. What can I store in one byte under one address? One byte is 8 bit I guess - so can I store a number from 0 to 255? If correct what do I do if I want to store bigger numbers or strings with more than one character. Can I store characters? If I read/write access the storage address a…

[sheng]

@Ric, the test setup, device first power up this mean sunSched[240] is empty when init as shown below. The string that is Particle.publish by the app is 1"175 for the first run. test reset Stored Value show what happen when I rest the device with the reset button (to test if the data is retain in the EEprom). After Stored Value, test what is stored in the EEprom.

String Data received
1"175
Pre Stored value

test reset Stored Value
�g
After Stored Value
�g
publish succes