Memory Model of Electron and Dynamic Memory Allocation in Spark Firmware

My main question, is this:

Is the Spark Firmware memory model the same as the mbed memory model?

I seem to be getting some type of hard fault on my Electron, only without the associated red SOS blinking. The LED just stays RED and my program no longer executes.

This occurs after roughly 20 hours of run time.

A push of the reset button and the Electron resets itself and starts working like nothing bad happened.

This smells like a memory leak to me. I don’t use dynamic memory allocation anywhere in my program (that I know of) so I don’t think it could be from my code.

I’ve heard that the String class uses dynamic memory allocation and can therefore fragment RAM, resulting in symptoms similar to a memory leak. Since I am registering cloud functions using Particle.function(), which take a String as a parameter, is it possible that the spark firmware is causing a memory leak resulting in MCU hanging after 20 hours?

Here are my cloud function callbacks and class constructor for cloud stuff:

    JazaControl::JazaControl(){};

    //Ensure this function is called before Particle.connect() in setup().
    //This will save data and prevent  these functions/variables from being re-registered.
    void JazaControl::begin(){

      // int* targBalance = &packManager.chargeBalance;
      Particle.variable("chargeBal", packManager.chargeBalance);
      //Particle.function("addBalance", addBalance);
      //Particle.function("lockOut", lockOut);

      Particle.function("addBalance", &JazaControl::addBalance, this);
      Particle.function("lockOut", &JazaControl::lockOut, this);

      // Particle.function("addBalance", [this](const String& command)->int{return addBalance(command); });
      // Particle.function("lockOut", [this](const String& command)->int{return lockOut(command); });

    }
    int JazaControl::addBalance(String command){

      char inputStr[10];
      int credit = 0;
      char buffer[50];

      if(command.length()>0){
        command.toCharArray(inputStr,10);
        credit = atoi(inputStr);

        balanceAdded = credit;
        packManager.chargeBalance += credit;

        //Confirm topUp was recieved by publishing to cloud
        snprintf(buffer, PUBLISH_BUFFER_SIZE, "Amount: %d, New Balance: %d", credit, packManager.chargeBalance); //NOTE Probably this will need to have "Amount:" and "New Balance:" removed for Norex compliance
        Particle.publish("topUp", buffer);
        myLog.warn("Added charge credit from cloud!");
        jazaSD.updateConfigurationFile();
        jazaUI.jazaCloudBalanceAddedCallback(credit);
        return packManager.chargeBalance;
      }
      else{
        myLog.error("Invalid function input from Particle cloud!");
        return 0;
      }
    }

    //This function is used to lock the system from the UI and prevent any new charges from being initiated
    int JazaControl::lockOut(String command){

       myLog.warn("Remote cloud lockout callback! -->lockOut()");
       myLog.info("lockOut command is %d characters long -->lockOut()", command.length());
       //Copies the lockout message to the char array for future reference
       command.toCharArray(lastLockoutMessage, 64);
       lastLockoutMessage[command.length()] = '\0';

       myLog.info("Lockout message char array is: \"%s\"", lastLockoutMessage);
       myLog.info("Lockout message String is:");
       myLog.info(command);
       //Add null terminating character to our char array version

       //Find which character the reason part of the message starts at
       short commaIndex = command.indexOf(',');

       //Check that reason was given
       if(commaIndex < 0){
          myLog.warn("No reason for lockout message given! -->lockOut()");
          //There was no comma in command, must not have been a reason supplied!
          lastLockoutReason = defaultLockoutReason;
       }
       else if (commaIndex < 60){
          myLog.info("Comma found at index %d of command, treating rest of command as a lockout reason message! -->lockOut()", commaIndex);
          //Set lockout reason pointer to just past the comma
          lastLockoutReason = lastLockoutMessage + commaIndex + 1;
          //Check if there is just a comma with nothing after it as the last character
          if(lastLockoutReason[0] == '\0'){
             lastLockoutReason = defaultLockoutReason;
          }
       }


       myLog.warn("Lockout - reason for change in lockout state is: %s -->lockOut()", lastLockoutReason);

       if(command.indexOf("unlock") == 0){
          lockSystem = false;
          myLog.warn("The JazaHub was just unlocked remotely -->lockOut");
          if( jazaUI.jazaCloudLockoutCallback(lastLockoutReason) ){
             Particle.publish("lockOut Status", "unlocked");
             return 1;
          }
       }
       else if(command.indexOf("lock") == 0){

          lockSystem = true;
          myLog.warn("The JazaHub was just locked remotely -->lockOut");
          if( jazaUI.jazaCloudLockoutCallback(lastLockoutReason) ){
             Particle.publish("lockOut Status", "locked");
             return 2;
          }
       }
       //Return -1 if none of above returned
       return 3;
    }

I added a Serial.println("System RAM free: %d bytes", System.freeMemory() ); To the end of my setup() function and it says I have 65304 bytes free once all my class instances are declared and initialized. I can’t imagine I’m using 65 kB of the stack inside my loop() function calls, especially since I’m not using dynamic memory allocation or recursion, and I have all of my application’s loop() activities segmented into 6 separate calls as shown below (allows stack to be cleared up in between each call).

    void loop(){
       //Debugging and test mode chores
       showLineFeedChars();
       checkTestModes();

       //Take care of chores
       canStack.update();
       packManager.update();
       dataManager.update();
       jazaUI.update();
       jazaPublish.update();
       networkManager.update();

    }

The only other thing I can think of is that I am making liberal use of the logging API that the recent Particle firmware releases have provided throughout my project. I have defined 16 logging categories, each corresponding to a different Logger instance myLog that is statically defined at the top of various cpp files in my project. Something like:

    //Logging objects
    SerialLogHandler logHandler(LOG_LEVEL_INFO,  //Default logging level for non-application messages
       {
          { "app", LOG_LEVEL_WARN },             //Logging level for logs from any application .cpp file not specified below

          { "app.main", LOG_LEVEL_TRACE},         //Logging level for logs from "main.cpp"

          { "app.MAX11610", LOG_LEVEL_WARN},          //Logging level for logs from "MAX11610.cpp" log category
          { "app.jazaAFE", LOG_LEVEL_TRACE},      //Logging level for logs from "jazaAFE.cpp" log category
          { "app.systemData", LOG_LEVEL_WARN},   //Logging level for logs from "systemData.cpp" log category
          { "app.DataManager", LOG_LEVEL_WARN},  //Logging level for logs from "DataManager.cpp" log category

          { "app.jazaHubChannel",    LOG_LEVEL_TRACE},    //Logging level for logs from "jazaHubChannel.cpp" code
          { "app.jazaPackManager", LOG_LEVEL_TRACE},  //Logging level for logs from "jazaPackManager.cpp" log category

          { "app.CM_CAN", LOG_LEVEL_INFO},     //Logging level for logs from "CM_CAN.cpp" log category
          { "app.jazaCAN", LOG_LEVEL_WARN},   //Logging level for logs from "jazaCAN.cpp" log category

          { "app.userInterface", LOG_LEVEL_TRACE},        //Logging level for logs from "userInterface.cpp" log category
          { "app.ST7565",         LOG_LEVEL_WARN},

          {"app.jazaPublish", LOG_LEVEL_ALL},
          {"app.jazaControl", LOG_LEVEL_TRACE},
          {"app.network",   LOG_LEVEL_TRACE},
          {"app.jazaCloud", LOG_LEVEL_ALL},
          {"app.jazaSD", LOG_LEVEL_ALL}
       }
    );

    static Logger myLog("app.main");  //Logger object used in this "main.cpp" file

and then in other jazaControl.cpp file…

static Logger myLog("app.jazaControl");

and in jazaPackManager.cpp file…

static Logger myLog("app.jazaPackManager");

I would say that in the course of a typical iteration through loop() my project makes about 70-100 calls to up to 16 myLog Logger objects. Is that a possible cause of the behaviour I’m seeing?

What kind of heap usage does the spark firmware typically consume, and what are the data types/function calls that I should be weary of in the Particle Firmware API?

Is chargeBalance a fully static member?
How big is char lastLockoutMessage[]?
Maybe explicitly limiting the length in this might also be good lastLockoutMessage[command.length()] = '\0';

lastLockoutMessage[] is 65 bytes long.

chargeBalance is a public member of class jazaPackManager. I only have 1 instance of jazaPackManager, and have made it available as a global utility to all code that includes jazaPackManager.h as shown below…

jazaPackManager.h…

class jazaPackManager{

public:

int chargeBalance

};

extern jazaPackManager packManager;

and then in jazaPackManager.cpp

jazaPackManager packManager;

Is there a need to use the static keyword when declaring a class member variable to be exposed via particle.variable? I am able to query the value of this variable from Particle-CLI at will, so it seems to work, at least when the program first starts running!

If your class only allows for one instance (and the use of Particle.function() and Particle.variable() with static names suggests so), I’d guard against multi instantiation and only use static members.

And as for your supposed memory leak, you may not only look at System.freeMemory() after setup() but also keep whatch how it behaves over time.

You may also need to add some safe guards for some edge cases like here

        command.toCharArray(inputStr,10);

If your command is longer than 10 char your inputStr will not be terminated.

I’d personally prefer sizeof(buffer) instead a constant that may well diverge from the buffer size during code reworks

snprintf(buffer, PUBLISH_BUFFER_SIZE, "Amount: %d, New Balance: %d", credit, packManager.chargeBalance);

Just for the superstitious (like myself when I can’t find my problems :blush:) you could add extra precautions here

 lastLockoutMessage[constrain(command.length(), 0, sizeof(lastLockMessage)-1] = '\0';

Since not all code paths in lockOut(String command) seem to set lastLockoutReason, I’d propose to initialise that variable on entry like this

lastLockoutReason = defaultLockoutReason;

and only set it otherwise when you got a valid reason.
Otherwise any subsequent visit to that function without a correct reason may end up with this pointer pointing to an invalidated location causing troubles.

BTW, what are the last log entries before your system stalls?

Hey thanks for the tips, I will try and put a bit more work into safeguarding char array stuff, and will make sure to declare the chargeManager variable as static.

The last log entries before the system stalls are user interface operations printing bitmaps and char arrays to an LCD, followed by the very last log entry which is just a single character that I print to the serial monitor periodically to show that the app is still running (a ‘|’ character).

One side note on this periodic character printing is that even though I am using myLog.trace('|'); , the output to the serial monitor just shows the character itself, with no logging category or timestamp. For example, here are the last log entries before the app crashes:

0070070827 [app.userInterface] TRACE: Dealing with the unplug event on channel 4 that just happened -->jazaPackUnplugCallback()
0070070828 [app.userInterface] TRACE: Clearing infoString due to unplug event! -->jazaPackUnplugCallback()
0070070829 [app.userInterface] TRACE: Entering displayCenteredString()

0070070837 [app.userInterface] TRACE: Updating LCD ports section -->updateUI_LCD()
0070070844 [app.userInterface] TRACE: Updating LCD info section -->updateUI_LCD()
0070070844 [app.userInterface] TRACE: Entering displayCenteredString()

|

Note the lack of timestamp/logging category on the ‘|’ print.