DeviceOS 2.0 and JSONBufferWriter (built-in)

The below code has been very stable on OS 1.5.2, however 2.0 causes a panic and restart when the JSONBufferWriter is called from a function invoked by a Timer. The result is the same with the JSONBufferWriter code placed directly in the postSensorData function or as its detailed below.

It may be my poorly written code or maybe a problem with 2.0, any help would be appreciated.

Thanks.

// This #include statement was automatically added by the Particle IDE.
#include <RunningMedian2.h>
#include <Adafruit_BME280_RK.h>
#include "PMS7003-Particle-Sensor-Serial.h"

PMS7003Serial<USARTSerial> pms7003(Serial1, D2);
Adafruit_BME280 bm1;

Timer readSensors(10000, readSensorData);
Timer postData(60000, postSensorData);

char jsonBuf[256];
float temp;
float humid;
unsigned long pm1_0, pm2_5, pm10;
unsigned long readCount;

RunningMedian temp10sec = RunningMedian(3);
RunningMedian humid10sec = RunningMedian(3);
RunningMedian pm1_0_10sec = RunningMedian(3);
RunningMedian pm2_5_10sec = RunningMedian(3);
RunningMedian pm10_10sec = RunningMedian(3);
RunningMedian temp_10sec = RunningMedian(3);
RunningMedian humid_10sec = RunningMedian(3);

RunningMedian temp60sec = RunningMedian(6);
RunningMedian humid60sec = RunningMedian(6);
RunningMedian pm1_0_60sec = RunningMedian(6);
RunningMedian pm2_5_60sec = RunningMedian(6);
RunningMedian pm10_60sec = RunningMedian(6);
RunningMedian temp_60sec = RunningMedian(3);
RunningMedian humid_60sec = RunningMedian(3);

void setup() {

  Serial.begin();
  
  bm1.begin(0x76);
  
  readSensors.start();
  postData.start();
  
  Particle.variable("getData", createResponse );
  Particle.variable("readCount", readCount );

}

void loop() {

}

void readSensorData() {

    int loopCnt = 0;
    do {
        
        loopCnt++;
        pms7003.Read();
        pm1_0_10sec.add( pms7003.GetData(pms7003.pm1_0) );
        pm2_5_10sec.add( pms7003.GetData(pms7003.pm2_5) );
        pm10_10sec.add( pms7003.GetData(pms7003.pm10) );

        temp_10sec.add( bm1.readTemperature() );
        humid_10sec.add( bm1.readHumidity() );


    } while (loopCnt < 3 );    
    
    pm1_0_60sec.add( pm1_0_10sec.getMedian() );
    pm2_5_60sec.add( pm2_5_10sec.getMedian() );
    pm10_60sec.add( pm10_10sec.getMedian() );
    temp_60sec.add( temp_10sec.getMedian() );
    humid_60sec.add( humid_10sec.getMedian() );

    readCount++;
}

String createResponse() {

  memset(jsonBuf, 0, sizeof(jsonBuf));
  JSONBufferWriter writer(jsonBuf, sizeof(jsonBuf) -1);

  writer.beginObject();

    writer.name("pm1_0").value( word( pm1_0_60sec.getAverage() ) );
    writer.name("pm2_5").value( word( pm2_5_60sec.getAverage() ) );
    writer.name("pm10").value( word( pm10_60sec.getAverage() ) );

    writer.name("um_3").value( pms7003.GetData(pms7003.count0_3um) );
    writer.name("um_5").value( pms7003.GetData(pms7003.count0_5um) );
    writer.name("um1").value( pms7003.GetData(pms7003.count1um) );
    writer.name("um2_5").value( pms7003.GetData(pms7003.count2_5um) );
    writer.name("um5").value( pms7003.GetData(pms7003.count5um) );
    writer.name("um10").value( pms7003.GetData(pms7003.count10um) );

    writer.name("temp").value( float( round( (temp_60sec.getAverage() *10) ) /10));
    writer.name("humid").value( float( round( (humid_60sec.getAverage() *10) ) /10));

  writer.endObject();
  
  return writer.buffer();
}

void postSensorData() {

  Particle.publish("emon-data", createResponse() , PRIVATE);

}



The reason for the crash could be the limited stack size for SW timers.
Although I wouldn’t have thought that this would behave differently between 1.5.2 and 2.0.0.

However, I’d suggest, you move all the JSON stuff into postSensorData() to avoid one function call and an extra return value to be placed on the stack and maybe use a global JSONBufferWriter instance instead of spinning one up locally - you are already using a global buffer for it anyhow.

BTW, your RunningMedian objects should better be created like this

RunningMedian temp10sec(3); // don't do it via '= RunningMedian(3)`

to avoid creating a temporary generic object which would then be replaced the actually desired one.

1 Like

Consolidating into postSensorData hasnt changed the behaviour.

Interestingly the local JSONBufferWriter is due to when it was declared globally it caused panics under OS1.5.2, making it global under OS2.0 doesnt cause panics but the JSONBufferWriter issue remains.

I will need to think through next steps. Thanks for your help.

Can you tell us what SOS panic you see? (SOS + x red blinks)
You are also calling lots of pms7003.xxx() functions which may also not play well with the limited stack size.

It also appears that Particle.publish() does not play well with being called from a timer callback as it keeps causing cloud connection losses - a fact that is (somewhat) documented in the Software Timers reference but seems undocumented in the Particle.publish() reference (where it probably should be too).

A workaround for all the above would be to move the logic into a function that’s called from loop() guarded by a flag that can be set via your timer.

I also see that JSONBufferWriter cannot be reused and hence has to be created locally (which is odd IMHO - @rickkas7?).

1 Like

Thanks @ScruffR for your insights. The SOS panic is a single flash - a hard fault, what ever that means. Happy to continue to work through if it helps overall.

I like your suggestion to simply set a flag with the timer and do everything else in the loop, that certinaly gives me an elegant solution.

1 Like

I have changed the code to a timer set flag and now call postSensorData within loop. This works sucessfully under OS 2.0. Happy to try variants if it helps to understand why.

// This #include statement was automatically added by the Particle IDE.
#include <RunningMedian2.h>
#include <Adafruit_BME280_RK.h>
#include "PMS7003-Particle-Sensor-Serial.h"
#include "sensorData.h"

PMS7003Serial<USARTSerial> pms7003(Serial1, D2);
Adafruit_BME280 bm1;
SensorData mySensorData;

Timer readSensors(10000, readSensorData);
Timer postData(60000, setPublishFlag);

char jsonBuf[256];
float temp;
bool publishData = false;
float humid;
unsigned long pm1_0, pm2_5, pm10;
unsigned long readCount;

RunningMedian temp10sec(3);
RunningMedian humid10sec(3);
RunningMedian pm1_0_10sec(3);
RunningMedian pm2_5_10sec(3);
RunningMedian pm10_10sec(3);
RunningMedian temp_10sec(3);
RunningMedian humid_10sec(3);

RunningMedian temp60sec(6);
RunningMedian humid60sec(6);
RunningMedian pm1_0_60sec(6);
RunningMedian pm2_5_60sec(6);
RunningMedian pm10_60sec(6);
RunningMedian temp_60sec(6);
RunningMedian humid_60sec(6);

void setup() {

  Serial.begin();
  
  bm1.begin(0x76);
  
  readSensors.start();
  postData.start();
  
  Particle.variable("getData", createResponse );
  Particle.variable("readCount", readCount );

}

void loop() {

  while (publishData) {
    
    postSensorData();  
    publishData = false;
  }

}

void readSensorData() {

    int loopCnt = 0;
    do {
        
        loopCnt++;
        pms7003.Read();
        pm1_0_10sec.add( pms7003.GetData(pms7003.pm1_0) );
        pm2_5_10sec.add( pms7003.GetData(pms7003.pm2_5) );
        pm10_10sec.add( pms7003.GetData(pms7003.pm10) );

        temp_10sec.add( bm1.readTemperature() );
        humid_10sec.add( bm1.readHumidity() );


    } while (loopCnt < 4 );    
    
    pm1_0_60sec.add( pm1_0_10sec.getMedian() );
    pm2_5_60sec.add( pm2_5_10sec.getMedian() );
    pm10_60sec.add( pm10_10sec.getMedian() );
    temp_60sec.add( temp_10sec.getMedian() );
    humid_60sec.add( humid_10sec.getMedian() );

    readCount++;
}

void setPublishFlag() {
    publishData = true;
}

String createResponse() {

  JSONBufferWriter writer(jsonBuf, sizeof(jsonBuf) -1);
  memset(jsonBuf, 0, sizeof(jsonBuf));

  writer.beginObject();

    writer.name("pm1_0").value( word( pm1_0_60sec.getAverage() ) );
    writer.name("pm2_5").value( word( pm2_5_60sec.getAverage() ) );
    writer.name("pm10").value( word( pm10_60sec.getAverage() ) );

    writer.name("um_3").value( pms7003.GetData(pms7003.count0_3um) );
    writer.name("um_5").value( pms7003.GetData(pms7003.count0_5um) );
    writer.name("um1").value( pms7003.GetData(pms7003.count1um) );
    writer.name("um2_5").value( pms7003.GetData(pms7003.count2_5um) );
    writer.name("um5").value( pms7003.GetData(pms7003.count5um) );
    writer.name("um10").value( pms7003.GetData(pms7003.count10um) );

    writer.name("temp").value( float( round( (temp_60sec.getAverage() *10) ) /10));
    writer.name("humid").value( float( round( (humid_60sec.getAverage() *10) ) /10));

  writer.endObject();
  
  return writer.buffer();
}

void postSensorData() {

  Particle.publish("emon-data", createResponse() , PRIVATE);

}