Safe way to build strings without memory fragmentation

Firmware
1

I have an Electron that I am using as a datalogger but it keeps being unable to publish data to the cloud after about 4 days. It is running with system threading enabled.

Right now I am using a function to build and return the string that I want to publish, but I believe it may be causing a memory fragmentation issue when the Electron is run for more than 4 days continuously. I was under the impression that this would not result in memory fragmentation since all of the dynamically allocated strings that are implicitly created when String::format() is called get released when the function exits. Sure there is a big memory gap in the heap that dataEntry is sitting on top of at the end, but dataEntry gets released in short order once the function returns and the built string gets written to the cloud.

Here is my function as it stands:

String getStringForCloud){

  //Stores the line string to be published to the Cloud
  String dataEntry;
    
  dataEntry += String(Time.now()) + ',';  //Add the UNIX time stamp

  //Add pv system data points
  dataEntry += String::format("%6.2f,", pvDat.avgVoltage);
  dataEntry += String::format("%6.2f,", pvDat.avgCurrent);
  dataEntry += String::format("%6.2f,", pvDat.avgPower);
  dataEntry += String::format("%6.2f,", pvDat.energy);

  //Add BESS system data points
  dataEntry += String::format("%6.2f,", bessDat.avgVoltage);
  dataEntry += String::format("%6.2f,", bessDat.avgCurrent);
  dataEntry += String::format("%6.2f,", bessDat.avgPower);
  dataEntry += String::format("%6.2f,", bessDat.energy);

  //Add MPPT system data points
  dataEntry += String::format("%6.2f,", mpptDat.avgPower);
  dataEntry += String::format("%6.2f,", mpptDat.energy);

  //Add JazaPack system data points
  dataEntry += String::format("%6.2f,", inverterDat.avgPower);
  dataEntry += String::format("%6.2f,", inverterDat.energy);

  //Add 12 V Load system data points
  dataEntry += String::format("%6.2f,", loads12vDat.avgPower);
  dataEntry += String::format("%6.2f,", loads12vDat.energy);

  //Add Controller Load system data points
  dataEntry += String::format("%6.2f,", controllerDat.avgPower);
  dataEntry += String::format("%6.2f,", controllerDat.energy);

  return dataEntry;

}

  • Is there a better/safer way to build this string?

  • Would calling dataEntry.reserve(200) at the beginning of this function be a way to safeguard against heap fragmentation?

  • What about all the little String objects that are allocated as I build up dataEntry ? Since I have system threading enabled, is it possible that the system firmware will dynamically allocate heap memory on top of these short-lived string building blocks before this function exits and frees them back to the heap, thus causing fragmentation? If this is the case, would it be better to simply replace the above code with a single String::format() call? Like this:

    dataEntry += String::format("%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,%6.2f,", pvDat.avgVoltage, pvDat.avgCurrent, pvDat.avgPower, pvDat.energy, bessDat.avgVoltage, bessDat.avgCurrent, bessDat.avgPower, bessDat.energy, mpptDat.avgPower, mpptDat.energy, inverterDat.avgPower, inverterDat.energy, loads12vDat.avgPower, loads12vDat.energy, controllerDat.avgPower, controllerDat.energy);

2

As I think others have said to you in other threads, I would allocate a global char array and then just snprintf() into that.

1 Like
3

Nope, since it's not global, so it will still be released on exit. So if another variable (e.g. String) grabs part of that and doesn't release it, your next String of 200 won't fit into the same spot and other 200 have to be reserved. It also doesn't help against creating/releasing all the temporary substrings.

I agree with @bko and repeat what's said in the other thread.
Since you have a format string already use it with snprintf() instead of String::format().

4

Thanks for the response guys. What about the format string itself? Do I need to declare the format string as a global variable as well to avoid memory fragmentation? Otherwise, it is being dynamically allocated every time I make a call to snprintf(), right?

5

That’s a string literal which won’t be copied to RAM since snprintf() uses a const char* as format string.
This way that string is only present in flash.

6

Ah okay. So in other words, the format string literal resides in the 1 MB of application flash, and when I pass that string literal to snprint() it simply allocates memory for a char pointer on the stack which points to that static address where the string literal is in flash.

That makes sense.

1 Like