Method to reduce data operations | Publish an array of JSON

Firmware
4

@joel Thanks for letting me know. After thinking about it some more… I think I’m better off with a simpler approach and not use the file system. Rather than writing files to the file system I decided to just continue to append a JSON to an Array using the very nice library JsonParserGeneratorRK. I just declare the JSON Writer to be global.

Here’s the current snippets of code that I put together.

Before Setup - Create a Global Object:

//Define Buffer for JSON Array and Initialize it
JsonWriterStatic<1024> JSONArray;

//Start with a max Event Data size of 256. Update it once the cellular modem is on and connected. 
int maxEventDataSize = 256;

In Setup() Initialize the start of the Array:

  //start an array object
  JSONArray.startArray();

Each time I want to add data to the JSON array: (Typically every 5 minu

//Create a new locally scoped JSON object to Insert into the Array
JsonWriterStatic<256> jw;
          {
            JsonWriterAutoObject obj(&jw);
            jw.insertKeyValue("Datetime", Time.now());
            jw.insertKeyValue("Sensor1", 123);
            jw.insertKeyValue("Sensor2", 456);
            jw.insertKeyValue("Sensor3", 789);
          }

          //If adding the new JSON object extends the array beyond the max bytes of a Publish event, close the array, publish it via PublishQueue, clear it and start another array. 8 bytes are left as spare for Particle pre/ or post data. 
          if ((maxEventDataSize - int(JSONArray.getOffset()) - int(jw.getOffset())) < 8 ){
            JSONArray.finishObjectOrArray();
            publishQueue.publish("DataArray", JSONArray.getBuffer(), 60, PRIVATE, NO_ACK);
            JSONArray.init();
            JSONArray.startArray();
          }

          //Add new data to the JSON Array
          JSONArray.insertCheckSeparator();
          JSONArray.insertJson(jw.getBuffer());

          //If adding the new data extends the array beyond the max bytes of a Publish event, close the array, publish it, clear it and start another array 
          //If if the data came in during an irregular time period publish the data immediately. 
          if (!rptWindow || (maxEventDataSize - int(JSONArray.getOffset()) - int(jw.getOffset())) < 8){
            JSONArray.finishObjectOrArray();
            publishQueue.publish("DataArray", JSONArray.getBuffer(), 60, PRIVATE, NO_ACK);
            JSONArray.init();
            JSONArray.startArray();
          }

When I’m done with all sensor readings for this period (i.e. every 20 or 60 minutes), publish the Current JSON Array to the Cloud:

            //If there is data in the JSON Array: End the Array, Publish it, clear the array and start a new JSON array for new data:
            if (JSONArray.getOffset()>4){
              JSONArray.finishObjectOrArray();
              publishQueue.publish("DataArray", JSONArray.getBuffer(), 60, PRIVATE, NO_ACK);
              JSONArray.init();
              JSONArray.startArray();
            }

            //Updated the Max Event Data Size if Needed. This can not be done in setup() as the cellular modem must be ON. The value is first initialized to 256. This should allow updating it once. 
            if (maxEventDataSize <= 256) {
              maxEventDataSize = Particle.maxEventDataSize();
            }
                       
            //Wait until all Events in the PublishQueue are sent successfully. Wait up to 30 seconds
            int i = 0;
            while (publishQueue.getNumEvents() > 0 and i <= 30) {
                  softDelay(1000);
                  i = i+1;
            }

The result in the Particle Console is identical as my earlier example. It’s an array of JSON where each member is sensor readings. In most cases, these are sensor readings spaced 5 minutes apart in time and then published at either a 20 minute frequency or 60 minute frequency.

Overall, this JSON Array seems like a cleaner approach then using the Flash File System. I won’t have any lasting affects on the flash this way either. Only downside I see is lost data if the device resets but that should be rare and is not critical for my application.

1 Like