Memory profiling


#15

@peekay123 thanks for helping out! I did not flash it to the device yet. The complete output during compilation is here:
https://dl.dropboxusercontent.com/u/20621/compilation_process.txt


#16

@jenschr, try flashing your code!


#17

I flashed it now, and surely enough - the 6Kb .bin file did not contain the same as the one that is 101Kb :wink:
It’s basically flashed with nothing according to the output of “arm-none-eabi-size -A”.

What I should see is a listing of all the classes used in my project and how much they each take up. Right?


#18

Can you provide a list of files and the directory structure? Seems like not everything is being linked in.


#19

Looking at line 3 in the output file (with the “rm -f”) I can see .o files for all the classes in my project, so I’m pretty sure it’s all there. All these files have both a header (.h) and a body (.cpp) file. The files that are not listed as being in the “src” folder are in the “lib” folder. “lib” and “src” are both folders on the root level in my project directory (standard Particle project setup).

These are the files listed and I can’t find any of them in the output from arm-none-eabi-size:

/myProject/src/Fans.o
/myProject/src/Fire.o
/myProject/src/InputManager.o
/myProject/src/Internetwork.o
/myProject/src/Motors.o
/myProject/src/ScreenManager.o
/myProject/src/Model/PtHundred.o
/myProject/src/Model/Process.o
/myProject/src/Model/ValueObjects.o
/myProject/src/Screens/BaseScreen.o
/myProject/src/Screens/DebugScreen.o
/myProject/src/Screens/PidScreen.o
/myProject/src/Screens/RootScreen.o
/myProject/src/Screens/SettingsScreen.o
/myProject/src/Screens/StartupScreen.o
/myProject/src/Fans.o
/myProject/src/Fire.o
/myProject/src/InputManager.o
/myProject/src/Internetwork.o
/myProject/src/Motors.o
/myProject/src/ScreenManager.o
/myProject/src/Model/PtHundred.o
/myProject/src/Model/Process.o
/myProject/src/Model/ValueObjects.o
/myProject/src/Screens/BaseScreen.o
/myProject/src/Screens/DebugScreen.o
/myProject/src/Screens/PidScreen.o
/myProject/src/Screens/RootScreen.o
/myProject/src/Screens/SettingsScreen.o
/myProject/src/Screens/StartupScreen.o
/myProject/src/Fans.o
/myProject/src/Fire.o
/myProject/src/InputManager.o
/myProject/src/Internetwork.o
/myProject/src/Motors.o
/myProject/src/ScreenManager.o
/myProject/src/Model/PtHundred.o
/myProject/src/Model/Process.o
/myProject/src/Model/ValueObjects.o
/myProject/src/Screens/BaseScreen.o
/myProject/src/Screens/DebugScreen.o
/myProject/src/Screens/PidScreen.o
/myProject/src/Screens/RootScreen.o
/myProject/src/Screens/SettingsScreen.o
/myProject/src/Screens/StartupScreen.o
/myProject/src/Fans.o
/myProject/src/Fire.o
/myProject/src/InputManager.o
/myProject/src/Internetwork.o
/myProject/src/Motors.o
/myProject/src/ScreenManager.o
/myProject/src/Model/PtHundred.o
/myProject/src/Model/Process.o
/myProject/src/Model/ValueObjects.o
/myProject/src/Screens/BaseScreen.o
/myProject/src/Screens/DebugScreen.o
/myProject/src/Screens/PidScreen.o
/myProject/src/Screens/RootScreen.o
/myProject/src/Screens/SettingsScreen.o
/myProject/src/Screens/StartupScreen.o
/myProject/src/Fans.o
/myProject/src/Fire.o
/myProject/src/InputManager.o
/myProject/src/Internetwork.o
/myProject/src/Motors.o
/myProject/src/ScreenManager.o
/myProject/src/Model/PtHundred.o
/myProject/src/Model/Process.o
/myProject/src/Model/ValueObjects.o
/myProject/src/Screens/BaseScreen.o
/myProject/src/Screens/DebugScreen.o
/myProject/src/Screens/PidScreen.o
/myProject/src/Screens/RootScreen.o
/myProject/src/Screens/SettingsScreen.o
/myProject/src/Screens/StartupScreen.o
/myProject/AccelStepperSpark/src/AccelStepper.o
/myProject/AccelStepperSpark/src/AccelStepperSpark.o
/myProject/DACx311/src/DACx311.o
/myProject/GSL1680/src/GSL1680.o
/myProject/MAX6639/src/max6639.o
/myProject/MCP3424/src/MCP3424.o
/myProject/MQTT/src/MQTT.o
/myProject/PID/src/PID_v1.o
/myProject/RA8875/src/RA8875.o
/myProject/RA8875/src/fonts/GothamXL16.o
/myProject/RA8875/src/fonts/GothamXL32.o
/myProject/SparkIntervalTimer/src/SparkIntervalTimer.o
/myProject/TCA6408/src/TCA6408.o  
/myProject/src/Fans.o.d
/myProject/src/Fire.o.d
/myProject/src/InputManager.o.d
/myProject/src/Internetwork.o.d
/myProject/src/Motors.o.d
/myProject/src/ScreenManager.o.d
/myProject/src/Model/PtHundred.o.d
/myProject/src/Model/Process.o.d
/myProject/src/Model/ValueObjects.o.d
/myProject/src/Screens/BaseScreen.o.d
/myProject/src/Screens/DebugScreen.o.d
/myProject/src/Screens/PidScreen.o.d
/myProject/src/Screens/RootScreen.o.d
/myProject/src/Screens/SettingsScreen.o.d
/myProject/src/Screens/StartupScreen.o.d
/myProject/src/Fans.o.d
/myProject/src/Fire.o.d
/myProject/src/InputManager.o.d
/myProject/src/Internetwork.o.d
/myProject/src/Motors.o.d
/myProject/src/ScreenManager.o.d
/myProject/src/Model/PtHundred.o.d
/myProject/src/Model/Process.o.d
/myProject/src/Model/ValueObjects.o.d
/myProject/src/Screens/BaseScreen.o.d
/myProject/src/Screens/DebugScreen.o.d
/myProject/src/Screens/PidScreen.o.d
/myProject/src/Screens/RootScreen.o.d
/myProject/src/Screens/SettingsScreen.o.d
/myProject/src/Screens/StartupScreen.o.d
/myProject/src/Fans.o.d
/myProject/src/Fire.o.d
/myProject/src/InputManager.o.d
/myProject/src/Internetwork.o.d
/myProject/src/Motors.o.d
/myProject/src/ScreenManager.o.d
/myProject/src/Model/PtHundred.o.d
/myProject/src/Model/Process.o.d
/myProject/src/Model/ValueObjects.o.d
/myProject/src/Screens/BaseScreen.o.d
/myProject/src/Screens/DebugScreen.o.d
/myProject/src/Screens/PidScreen.o.d
/myProject/src/Screens/RootScreen.o.d
/myProject/src/Screens/SettingsScreen.o.d
/myProject/src/Screens/StartupScreen.o.d
/myProject/src/Fans.o.d
/myProject/src/Fire.o.d
/myProject/src/InputManager.o.d
/myProject/src/Internetwork.o.d
/myProject/src/Motors.o.d
/myProject/src/ScreenManager.o.d
/myProject/src/Model/PtHundred.o.d
/myProject/src/Model/Process.o.d
/myProject/src/Model/ValueObjects.o.d
/myProject/src/Screens/BaseScreen.o.d
/myProject/src/Screens/DebugScreen.o.d
/myProject/src/Screens/PidScreen.o.d
/myProject/src/Screens/RootScreen.o.d
/myProject/src/Screens/SettingsScreen.o.d
/myProject/src/Screens/StartupScreen.o.d
/myProject/src/Fans.o.d
/myProject/src/Fire.o.d
/myProject/src/InputManager.o.d
/myProject/src/Internetwork.o.d
/myProject/src/Motors.o.d
/myProject/src/ScreenManager.o.d
/myProject/src/Model/PtHundred.o.d
/myProject/src/Model/Process.o.d
/myProject/src/Model/ValueObjects.o.d
/myProject/src/Screens/BaseScreen.o.d
/myProject/src/Screens/DebugScreen.o.d
/myProject/src/Screens/PidScreen.o.d
/myProject/src/Screens/RootScreen.o.d
/myProject/src/Screens/SettingsScreen.o.d
/myProject/src/Screens/StartupScreen.o.d
/myProject/AccelStepperSpark/src/AccelStepper.o.d
/myProject/AccelStepperSpark/src/AccelStepperSpark.o.d
/myProject/DACx311/src/DACx311.o.d
/myProject/GSL1680/src/GSL1680.o.d
/myProject/MAX6639/src/max6639.o.d
/myProject/MCP3424/src/MCP3424.o.d
/myProject/MQTT/src/MQTT.o.d
/myProject/PID/src/PID_v1.o.d
/myProject/RA8875/src/RA8875.o.d
/myProject/RA8875/src/fonts/GothamXL16.o.d
/myProject/RA8875/src/fonts/GothamXL32.o.d
/myProject/SparkIntervalTimer/src/SparkIntervalTimer.o.d
/myProject/TCA6408/src/TCA6408.o.d 
/myProject/libuser.a

#20

@jenschr, does you code make calls to the libraries? The object files are always produced but the linker will only bring in the code that is used in the final .bin file.


#21

That’s probably my problem? I cannot see any trace of myProject.ino in the above output. This is the file that is my entry-point and that links to all the others.

Did I miss something? Do I need to call this file something specific? Or do I have to specify what file has the entry-point?


#22

@jenschr, when compiling locally (local toolchain), the compiler won’t “see” an .ino file. You need to rename it to .cpp!


#23

Oh my… You’d almost think that this was so important that you’d mention it in https://github.com/spark/firmware/blob/develop/docs/gettingstarted.md or any of the other “how to build locally” guides by Particle?

But Thanks a lot @peekay123! I now get a load of error messages (since the local build does not work with the Particle project structure) but those I should be able to handle :smiley:


#24

One thing though - how does it figure out what .cpp file is the entry-point?


#25

@jenschr, the one with ‘setup()’ and ‘loop()’ :wink:


#26

Added a pull request to the readme.md now with the two most important things that stumped me when compiling locally.

Now I just need to understand how to dig deeper into this. I’ve found this to be a brilliant command:

arm-none-eabi-nm -C -S --size-sort …/myOut/coffeeroaster2.elf

This lists all the methods and how much space they take up in memory. It’s a great tool to locate what is taking up RAM and where you could save/optimise.

Any pointers to good articles on analysing memory from a Particle standpoint @peekay123?


#27

@jenschr, this nugget is great as well. Feed it any arm gcc .map file and it produces a memory usage analysis.

http://danieleff.com/stm32/map_analizer/


#28

That’s brilliant! Looking at other threads, it’s very confusing how much memory you ACTUALLY have (since some of these threads go back to the CORE). There is however an official link that points to the memory output from the Web IDE and this claims that the Photon has:

110592 bytes for program
20480 bytes for RAM

In my case it should mean that since I get this output from compile:

text data bss dec hex filename
107468 2124 12484 122076 1dcdc /workspace/target/workspace.elf

The text & data columns adds up to the amount of Flash used (out of 110592 available):
107468 + 2124 = 109592 (99% Flash used)

The data + bss columns adds up to the amount of RAM used (out of 20480 available):
2124 + 12484 = 14608 (71% RAM used)

That would explain some interesting behavior in my app this week? :blush:
Time to start using that extra 1Mb of storage in the P1 modules!


#29

@jenschr, the available memory thing is an ongoing beef I have with Particle. They still haven’t fix that. The Photon has near about 128KB of user flash and at least 60KB of RAM (if not 80KB). The numbers given were for the old Core which had 128KB - bootloader of flash and 20KB of RAM.


#30

Right. And the P1 has the exact same specs + that extra Flash chip, right?


#31

@jenschr, yes. However, I still don’t like the lack of documentation on that external chip.


#32

Some days ago I decapped a P1 and had a look inside. The P1 Flash Memory chip is called MX25L8006e and it’s from the company Macronix. There’s good documentation on their site and I got that added to the P1 datasheet so others should now find that info more easily. It works right out of the box, so I should be able to offload several things into this memory.

As you may know, @cermak has completed my initial attempts at porting WolfSSL and I have a colleague (more skilled than I am) that is working on a complete AWS IoT setup with TLS 1.2 taking it the last steps. It is however more critical than ever that we get to know the numbers for this as we’re spending 118Kb Flash already (Full TLS stack from WolfSSL + MQTT client). I’ve asked for help in a different thread, but thought I’d mention it here as well in case you had som new details.


#33

the flashee library works with the external memory on the P1, plus you can read/write to it using DFU via the -a 2 flag.


#34

Ohhh… That could be very useful @mdma! Do you have a link to an example of how I could write to the flash in DFU mode?