PROGMEM - major feature requirement

Spark Team. At present, Spark does not have a PROGMEM variable modifier like arduino:

The PROGMEM keyword is a variable modifier, it should be used only with the datatypes defined in pgmspace.h. It tells the compiler "put this information into flash memory", instead of into SRAM, where it would normally go.

This creates a situation where const and initialized variables data is stored in both flash and RAM, using precious memory resources. It is crucial that this capability be added soon as important libraries and programs will be near impossible to port (eg u8glib). At present, I am unable to run my code on my Spark as I have hit the 16KB boundary on RAM usage, in large part due to this problem. Can you please let the community know when this will be "attacked" in one of your sprints?

2 Likes

I did some other testing before and found const to work great for ensuring the data structure you are defining is in FLASH and not RAM. I can’t find my post with test code at the moment. Not sure if static is needed, but might be a good idea to add it just to be explicit about things.

A follow up question would be, can you think of a way to automatically redefine it so that it works for the STM32… I haven’t been able to. That is, create a macro called PROGMEM that does something to give us static const in the right place.

Thanks BDub! I can’t definitively say that const works since my compile RAM sizes seem to say otherwise. I have tried static const and the compiler barks back at me. I will have to do more testing but something is not making sense. Part of the problem is the product of the compile is the core AND the user code and it does not disaggregate the portions of flash and RAM allocated to each. I has become clear, nonetheless that there is a 16KB RAM “wall” after which programs will no run.

If anything BDub, we need a clear understanding of how NOT to have stuff go to RAM and what resources are unequivocally left for the user. This is mega important for new users and advanced user planning a port or new code.

3 Likes

I have a pull request in around this issue. Please take a look and if you think it best solves this problem comment! here (and there!) https://github.com/spark/core-firmware/pull/187

1 Like

jjrosent, have you actually tried to compile your test code? In the web IDE it produces errors! You may want to reconsider your issue :smile:

We got it all figured out! Remember it wont work in the web ide because my proposed changes arent in there yet :slight_smile:

Has anyone found a resolution to this issue yet?

This pull request is in for both local builds and the web based IDE or CLI.

Are you having a specific problem?

These AVR specific macros do nothing on Spark since the right thing already happens generally speaking.

Honestly I probably wont have a ram issue currently on the core with my code, but I get caried away pretty fast so i like to explictly put things I know to be constant in program flash. Bit confused as what is differant about the STM32. Still going to run out of ram if I throw enough at it right?

Basically what I’m trying to figure out is the semantics of variable assignment for the spark. The language in this thread kinda goes back and forth about it. Will defining all the progmem arrays as “static const” get things done?

Yes, in my experience static const arrays are only in flash, not in RAM. You would think that the keyword static is not required since that is the default, but for some of my stuff it seemed to be required and it generally doesn’t hurt.

1 Like

Sorry to dig up this really old thread but I wasn’t able to find an answer to a related problem. @peekay123, I’m trying to use your u8glib library with the spark but as you noted in the first post, some of the arrays are being stored in RAM and that causes the core to run out of space. According to other threads, declaring the large arrays as static const should place them in Flash but that’s not what I’m seeing when compiling;

Before incl. Large Array:

text	data	bss	dec	hex
76096	1252	11912	89260	15

In a nutshell:
Flash used	77348 / 110592	69.9 %
RAM used	13164 / 20480	64.3 %

After incl. Large Array

text	data	bss	dec	hex
76144	1256	17832	95232	17400

In a nutshell:
Flash used	77400 / 110592	70.0 %
RAM used	19088 / 20480	93.2 

Are there any workarounds for this now? Searched through a few threads but wasn’t able to find anything.

I’m not sure why you see this behaviour, but this little test sketch does suggest it works as expected

//uint8_t yy[2048] = "This is a rather large array";
//const uint8_t yy[2048] = "This is a rather large array";
static const uint8_t yy[2048] = "This is a rather large array";
uint8_t xx[2048];

void setup()
{
    xx[0] = 0;
}

void loop()
{
    for(int i = 0; i < 2048; i++)
      xx[i] = yy[i];
}

/*
uint8_t yy[2048] = "This is a rather large array";

Output of arm-none-eabi-size:

text	data	bss	dec	hex
75800	3272	13904	92976	16

In a nutshell:
Flash used	79072 / 110592	71.5 %
RAM used	17176 / 20480	83.9 %

-------------------------------------
const uint8_t yy[2048] = "This is a rather large array";

Output of arm-none-eabi-size:

text	data	bss	dec	hex
77848	1224	13904	92976	16

In a nutshell:
Flash used	79072 / 110592	71.5 %
RAM used	15128 / 20480	73.9 %

-------------------------------------

static const uint8_t yy[2048] = "This is a rather large array";

Output of arm-none-eabi-size:

text	data	bss	dec	hex
77848	1224	13904	92976	16

In a nutshell:
Flash used	79072 / 110592	71.5 %
RAM used	15128 / 20480	73.9 %
*/

What are the alterations between your Before and After results?

1 Like

@ScruffR, @emr, you just need to declare “const” and “static” is not required except to limit the scope of the declaration :smile:

I thought that too until I tried to have my included library have its constants in flash. I could not force them to be in flash without static in that case. As I said above, static is the default in C so it should not matter but it seemed to. Maybe the way you are compiling changes it somehow. I will try to test it again.

1 Like

@peekay123, as my attached results show in my test case it didn’t make any difference what so ever, but since @bko had mentioned his personal experiance in an earlier post, I just added it as a test case anyway.

I would actually love to see a code where it does make a difference to get to the bottom of how and why - just out of curiosity :wink:

Pretty strange. const data are placed in .rodata (read only data) sections, and our linker script places these in sections of the flash memory. Only non-const data goes into the RAM data section.

If someone can send me some code to compile I’m happy to investigate.

1 Like

Thanks everyone. Declaring const actually worked. There was more than one array that needed to be declared as const on my code…not sure how I missed that!

1 Like

@emr, it would be great if you shared your u8glib code once its ported! :wink: