Memory leak for unknown reason

I have a small example program here that has a memory bug that I cannot figure out. What it does: loads up a buffer with small string chunks, converts that large buffer back into a string. Then it creates a bunch of objects that are only wrappers for small chunks of buffer. It does this repetitively, and I don’t allocate any new memory after the setup(), yet the memory goes down slowly until it crashes. Any help appreciated!

#main.cpp

#include "application.h" //needed when compiling spark locally

#include <string>
#include <unordered_map>
#include "dummyclass.h"

using namespace std;
SYSTEM_MODE(MANUAL);

char* buffer;
unordered_map<int, DummyClass*> store;
string alphabet;
unsigned char alphabet_range;
unsigned char state;
int num_chars;

static const unsigned char STATE_INIT = 0;
static const unsigned char STATE_LOAD_BUFFER = 1;
static const unsigned char STATE_PREP_FOR_DESERIALIZE = 2;
static const unsigned char STATE_FAKE_DESERIALIZE = 3;
static const unsigned char STATE_FINISH_RESTART = 4;


bool delete_objects()
{
    Serial.println("deleting objects in 'store'");
    for(auto iter = store.begin(); iter != store.end(); iter++)
    {
        // Serial.println("\nDeleting the following element: \n");
        // Serial.print("Key: ");
        // Serial.println(iter->first);
        // delay(1000);
        // Serial.print("Value: ");
        // Serial.println(iter->second->ShowMeWhatYouGot());
        // delay(1000);

        delete iter->second;
        iter->second = nullptr;
    } 
    store.clear();

    if(store.empty())
        return true;
    else
        return false;
}


void setup()
{
    Serial.begin(9600);
    Serial1.begin(38400);

    Serial.println("Main:: Entered setup()");
    Serial.println("Main:: Waiting...");
    delay(2000);

    buffer = new char[9000];
    alphabet = string("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789~!@#$^&*()_-?/><[]{}|");
    alphabet_range = alphabet.length() - 1;
    state = STATE_INIT;
    num_chars = 0;
}


void loop()
{
    switch(state){
        case STATE_INIT: {

            Serial.println("\nin STATE_INIT");
            Serial.println("initializing buffer..");

            strcpy(buffer, "");
            state = STATE_LOAD_BUFFER;
            delay(1000);
            break;

        }
        case STATE_LOAD_BUFFER: {

            Serial.println("\nin STATE_LOAD_BUFFER");

            if(num_chars < 6000){
                string chunk;
                for(char i = 0; i < 200; i++){
                    int index = rand() % alphabet_range;
                    chunk.append(alphabet.substr(index, 1));
                    // strcat(buffer, alphabet.substring(index, index + 1));
                    num_chars++;
                }
                Serial.println("Constructed chunk: \n");
                Serial.println(chunk.c_str());
                Serial.println("\nadding chunk to buff");
                strcat(buffer, chunk.c_str());
            }
            else{
                num_chars = 0;
                state = STATE_PREP_FOR_DESERIALIZE;
            }
            delay(500);
            break;

        }
        case STATE_PREP_FOR_DESERIALIZE: {

            Serial.println("\nin STATE_PREP_FOR_DESERIALIZE");
            Serial.println("full buffer: \n");
            Serial.println(buffer);

            Serial.print("\nFree Memory: ");
            Serial.println(System.freeMemory());

            Serial.println("\nAttempting to delete current object set...");
            delay(500);
            if(delete_objects())
                Serial.println("_delete_objects succeeded");
            else {
                Serial.println("_delete_objects failed");
                break;
            }

            state = STATE_FAKE_DESERIALIZE;
            delay(1000);
            break;

        }
        case STATE_FAKE_DESERIALIZE: {

            Serial.println("\nin STATE_FAKE_DESERIALIZE");
            string buff_string(buffer);
            if(buff_string.length() == 0){
                Serial.println("Main:: EMPTY STRING CONVERTED FROM BUFFER");
            }

            int index = 0;
            int key = 1;
            while(index < buff_string.length())
            {
                int amount = (rand() % 50) + 5;
                DummyClass* dcp = new DummyClass(buff_string.substr(index, amount));
                store[key] = dcp;
                index += amount;
                key++;

                Serial.println("Created new DummyClass object");
                Serial.print("Free Memory: ");
                Serial.println(System.freeMemory());
            }
            state = STATE_FINISH_RESTART;
            delay(1000);
            break;

        }
        case STATE_FINISH_RESTART: {

            state = STATE_INIT;
            break;

        }
    }

}

#dummyclass.h

using namespace std;

class DummyClass {
    private:
        char* _container;

    public:
        DummyClass(){
        }

        DummyClass(string input){
            _container = new char[input.length()];
            strcpy(_container, input.c_str());
        }

        ~DummyClass(){
            delete _container;
            _container = nullptr;
        }

        char* ShowMeWhatYouGot(){
            return _container;
        }
};

Given that the memory leak is most likely in your application code, it may be easier to debug that on your computer rather than on the hardware. Can you compile your code for Visual Studio / Xcode and fix the memory growth there? The tools for finding memory leaks are much better on desktop.

1 Like

How do you know you have a memory leak? Do you see free memory go to 0 and other failures?

Keep in mind that System.freeMemory() is the high-water mark for the heap, and is the minimum amount of free memory.

yeah, I know that it only goes down, but I don’t see why it should be going down at all? And when it does reach 0 the system crashes.

I’m not versed in either of those tools. Is there an alternative?

                string chunk;
                for(char i = 0; i < 200; i++){
                    int index = rand() % alphabet_range;
                    chunk.append(alphabet.substr(index, 1));
                    // strcat(buffer, alphabet.substring(index, index + 1));
                    num_chars++;
                }

This loop looks suspect to me. You are depending on the string append method to grow chunk as needed, but you know you are going to run that loop 200 times. Why not use the string reserve method to just allocate that much space? I bet that this chews up a lot of memory with each new char you append calling realloc, potentially fragmenting memory.

2 Likes

As @bko pointed out, you might not, but the String class does - and so might other constructs like unordered_map (would need to look into the implementation of std::unordered_map to be sure, but I wouldn’t be surprised if it did)

Can you explain in a bit more detail why this causes a memory leak? Are there other memory leaks that might be ‘built in’ to the language as a result of using some standard method calls?

It turns out that the leak is actually caused in the object allocation; By making the _container variable only the size of the string input, I didn’t leave room for the null terminating character. This somehow causes a leak… though I’m not sure why.

1 Like

Try this link on how fragmentation happens:

By not leaving enough room for the string’s null terminator, and then doing a strcpy into the allocated memory, you were not necessarily leaking memory, but your were certainly corrupting your heap. The write of the null character trashed whatever was in that memory location that didn’t belong to you. Depending on what was there and its state this could have been a slow problem to develop, or it could have been an immediate crash. In any case, this is not a leak in the traditional sense.

5 Likes

Such is the wonder of low-level programming!