Library for logging to SD card: SdCardLogHandlerRK


#1

The official location and documentation for this library is: https://github.com/rickkas7/SdCardLogHandlerRK.

It’s in the Particle community libraries as: SdCardLogHandlerRK.

You’ll need a SD card reader, presumably a Micro SD card reader. Make sure you get one that’s compatible with the 3.3V logic levels used on the Photon and Electron. Some readers designed for the Arduino expect the 5V logic levels used by the original Arduino. I use this one from Sparkfun but there are others.

electron

This library uses the Logging API that was added in system firmware 0.6.0.

For example:

Log.trace("Low level debugging message");
Log.info("This is info message");
Log.warn("This is warning message");
Log.error("This is error message");

Log.info("System version: %s", System.version().c_str());

It does not remap Serial, so if you’re using Serial.print for logging, you should switch to using Log.info instead.

The library creates a directory (default: “logs”) at the top level of the SD card and stores the log files in that. Each log file is a .txt file 6-digit number beginning with 000001.txt.

The default log file is 1 MB (1000000 bytes), but you can reconfigure this. The actual size will be slightly larger than that, as the log is rotated when it exceeds the limit, and the file will always contain a whole log entry.

When rotated, the default is to keep 10 log files, but this is configurable. The oldest is deleted when the maximum number is reached.

By default, SdCardLogHandler writes to Serial as well, like SerialLogHandler. This can be reconfigured.

This is the example program:

#include "Particle.h"

#include "SdFat.h"
#include "SdCardLogHandlerRK.h"

SYSTEM_THREAD(ENABLED);

const int SD_CHIP_SELECT = A2;
SdFat sd;

SdCardLogHandler logHandler(sd, SD_CHIP_SELECT, SPI_FULL_SPEED);

size_t counter = 0;

void setup() {
	Serial.begin(9600);
}

void loop() {
	Log.info("testing counter=%d", counter++);
	delay(1000);
}

The full browsable API documentation is here.


How to print SD Card Success / Fail to OLED
Need Suggestion for external Cell and GPS antennas locations
Adafruit SSD1306 [SOLVED]
#2

Using this library couldn’t be simpler. Thank you Rick for the efforts.
I have a question related to repurposing its use to data logging. Can I use this API concurrently with another instance of the logger object?
If I want to use logger over Serial as usual:
SerialLogHandler logHandler(115200, LOG_LEVEL_WARN, {
{ "app", LOG_LEVEL_INFO },
{ "app.custom", LOG_LEVEL_INFO }
});`

and then add an instance of your object to log geo-location data for example:

SdCardLogHandler logHandler2(sd, SD_CHIP_SELECT, SPI_FULL_SPEED);

How do I direct data to each logger?

Log.info("GNSS fix age=%d ms", gnss_last_fix_ms - millis()); --> I want this to go over Serial

Log.info("{\"s_id\": %d, \"spd\": %f,\"lat\": %f,\"lon\": %f}, session_id, speed_mph, lat, lon); --> I want this written to an SD file


#3

Thanks! Yes, you can have multiple log handlers. Here’s an example that uses both Serial and SD card loggers, and you choose which to log to using the categories feature.

#include "Particle.h"

#include "SdFat.h"
#include "SdCardLogHandlerRK.h"

SYSTEM_THREAD(ENABLED);

const int SD_CHIP_SELECT = A2;
SdFat sd;

// Only log errors and app.sd info level log messages to the SD card
SdCardLogHandler sdLogHandler(sd, SD_CHIP_SELECT, SPI_FULL_SPEED, LOG_LEVEL_ERROR, {
	{ "app.sd", LOG_LEVEL_INFO }
});
// Turn off serial logging in the SdCardLogHandler to avoid logging twice
STARTUP(sdLogHandler.withNoSerialLogging());

// Use LogToSD.info() instead of Log.info(), etc. to write to the SD card
Logger LogToSD("app.sd");

// Log everything to Serial
SerialLogHandler serialLogHandler;


size_t counter = 0;

void setup() {
	Serial.begin(9600);
}

void loop() {
	Log.info("testing counter=%d", counter++);
	LogToSD.info("write to SD counter=%d", counter++);

	delay(1000);
}


#4

Great! thank you.


#5

Thank you thank you thank you! You just saved me a crap load of work and frustration. I was going to tackle SD card logging this week for a urgent project.


#6

I am looking for some newbie help. I have hit a wall with what I assume is a simple task. I have been able to log data to an Sd card with the help of the above post. What I can’t seem to do is get multiple entries in one line with a comma separator.

I would like my data to post as Data + Comma + Time Stamp. This is so I can export to excel.

I can log them as separate lines but can’t seem to get the right code correct to get them on one line.

Here is my part of the code.

Serial.print(Time.timeStr());
Serial.print(",");
Serial.printf(“Flow: %.1f”, flow);

Log.info(“Flow: %.1f”, flow); // Had one line with flow: Example: INFO: Flow: -0.1
Log.info(Time.timeStr()); // Next line had time and date. Example: INFO: Thu May 24 22:13:07 2018
delay(6000);

Sorry in advance if this is obvious. I have been trying for 3 days now and just can’t seem to get it right.
Cheers,
Tom


#7

It’s easy, but I wouldn’t say obvious:

Log.info(“%s,Flow: %.1f”, Time.timeStr().c_str(), flow); 

#8

Thank you so much sir. I will try it now.

BTW your example and code could not be easier to follow. As I stated I am a complete rookie at all this. But I was able to log your test to the sd card on the first try. Then for some miracle was able to add it to my 4-20 ma code and log that as well… Then I got stuck trying to be fancy. lol.


#9

In the event that someone stumbles across this question this is the line of code I used.

Log.info(",%s, %.1f", Time.timeStr().c_str(), flow);

This gives me the following on my SD Card.

0004291877 [app] INFO: ,Tue May 29 14:25:32 2018, 0.1

I was able to add a comma before the %s at the beginning of the code. This lets me separate the default initial [app] INFO: from the date time, and data. This way I can import the data to excel using comma separated format.

This may not be the correct way of doing this but I used Ricks code and trial and error until I got it. Now I just need to learn how to convert a JSON string to a Log.info string, I may need a few bottles of wine for that.

Cheers


#10

Happy Friday Particle People.

Would anyone following this thread mind posting a sniped of the code they use to serial print the debug error messages and status messages they use with this library?

I know it says use Serial.printlnf at the top of the .cpp file for debug and I have been trying to get it done by trial and error but just cant seem to get it. The long term goal here is to print the status of the SD Card to my OLED screen on my project. If I can print the message to serial then I know how to print it to my OLED… but that’s about it.

Here are the goal messages I want to print, or a version of the following.

Writing to SD Success
Writing to SD Failed
No Card Found,
Or Card Ejected.
Wow your really handsome… ok that last one is a joke.

I figure I can convert the debug message but just can’t seem to get the code correct to print them to serial.

Any help is greatly appreciated.

Thank you,
Tom


#11

I released version 0.0.5 of SdCardLogHandler. The new version updates to a version of SdFat that is compatible with mesh devices. It also refactors the code into two classes.

The new SdCardPrintHandler works just like the log handler, but isn’t a log handler. You can use it to print arbitrary text data to rotating log files of a maximum size, without getting log data mixed in. It’s a subclass of the Print class, so all of the overloads of print, println, printf, and printlnf can all be used to write to your file.

The data is written to the card whenever a \n in encountered (or the buffer is full) so as long as you write lines of less than 128 bytes, you won’t run into a situation where a line gets split across multiple files.


#12

I’m attempting to use this library for the first time. I’m also attempting to use the Particle Workbench for Visual Studio Code for the first time. That said, I’m not able to get the simplest of the examples to compile.

The compiler first complains about:
SdCardLogHandler logHandler2(sd, SD_CHIP_SELECT, SPI_FULL_SPEED);
error: argument list for class template “SdCardLogHandler” is missing

changing the line to read:
SdCardLogHandler<2048> logHandler2(sd, SD_CHIP_SELECT, SPI_FULL_SPEED);
satisfies the error above, but then it complains:
identifier “SPI_FULL_SPEED” is undefined

I’m happy to post more code but I wanted to first find out if this is a tool problem or a code problem.

Any input will be appreciated!


#13

Probably worth noting that using a Particle: Cloud Compile results in a successful compile and the bin file is downloaded. This leads me in the direction of a tool problem?


#14

It shouldn’t be required but you could try adding #include "SdFat.h"; in your code since this is the library where SPI_FULL_SPEED is defined.
Also double check whether SdFat library is also downloaded into your lib folder. This should happen automatically due to the dependency defined in SdCardLogHandlerRK but better make sure.

What I also always do when downloading a library to Workbench is to remove the examples folders from all the libraries in the lib folder - this might be just superstition, but in the past local builds did have troubles with the examples being present.
Another thing to actually have parity between cloud build and local build is to comment out the dependencies in the project.properties file. This ensures that the cloud build also uses your local copy of the library and not the cloud based version - that’s especially important if you decide to modify the library for any reason.


#15

Thanks for the input. It turns out that it compiles successfully (using OS 0.7.0, 0.6.4 fails - more on that later). However, the Problem window shows 1 - “identified SPI_FULL_SPEED” is undefined. And the inteli-sense is confused:

As for OS 0.6.4 I get the following build errors:

../build/target/system/platform-10/\libsystem.a(system_network.o): In function `HAL_NET_notify_can_shutdown':
C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\system/src/system_network_internal.h:550: multiple definition of `HAL_NET_notify_can_shutdown'
../build/target/hal/platform-10/\libhal.a(cellular_hal.o):C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\hal/src/electron/cellular_hal.cpp:41: first defined here
../build/target/system/platform-10/\libsystem.a(system_network.o): In function `HAL_NET_notify_connected':
C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\system/src/system_network.cpp:69: multiple definition of `HAL_NET_notify_connected'
../build/target/hal/platform-10/\libhal.a(cellular_hal.o):C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\hal/src/electron/cellular_hal.cpp:20: first defined here
../build/target/system/platform-10/\libsystem.a(system_network.o): In function `HAL_NET_notify_dhcp':
C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\system/src/system_network.cpp:84: multiple definition of `HAL_NET_notify_dhcp'
../build/target/hal/platform-10/\libhal.a(cellular_hal.o):C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\hal/src/electron/cellular_hal.cpp:34: first defined here
../build/target/system/platform-10/\libsystem.a(system_network.o): In function `HAL_NET_notify_disconnected':
C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\system/src/system_network.cpp:74: multiple definition of `HAL_NET_notify_disconnected'
../build/target/hal/platform-10/\libhal.a(cellular_hal.o):C:\Users\norma\.particle\toolchains\deviceOS\0.6.4\firmware-0.6.4\hal/src/electron/cellular_hal.cpp:27: first defined here
collect2.exe: error: ld returned 1 exit status
make[1]: *** [../build/module.mk:225: c:/Dev_Area/FT/ITFT-SOTERIA/ITFT-SOTERIA/embedded/eval/HelloWorld/target/HelloWorld.elf] Error 1
make[1]: Leaving directory '/cygdrive/c/Users/norma/.particle/toolchains/deviceOS/0.6.4/firmware-0.6.4/main'
make: *** [C:\Users\norma\.particle\toolchains\buildscripts\1.2.0\Makefile:46: compile-debug] Error 2
The terminal process terminated with exit code: 2

#16

Try make clean-all and build again.


#17

I’ve tried that but it doesn’t help.


#18

I’m not sure why your program isn’t compiling in workbench, but the 1-simple example can be built on the Electron for both 0.6.4 and 0.7.0 using the Particle CLI compiler:

Electron 0.6.4

MacPro:SdCardLogHandlerRK rickk$ particle compile electron examples/1-simple/ --target 0.6.4 --saveTo firmware.bin

Compiling code for electron
Targeting version: 0.6.4

Including:
    examples/1-simple/SdCardLogExample.cpp
    library.properties
    src/RingBuffer.h
    src/SdCardLogHandlerRK.cpp
    src/SdCardLogHandlerRK.h
attempting to compile firmware 
downloading binary from: /v1/binaries/5c7850e25476ab035a798ad1
saving to: firmware.bin
Memory use: 
   text	   data	    bss	    dec	    hex	filename
  26620	    196	   5480	  32296	   7e28	/workspace/target/workspace.elf

Compile succeeded.

Electron 0.7.0

MacPro:SdCardLogHandlerRK rickk$ particle compile electron examples/1-simple/ --target 0.7.0 --saveTo firmware.bin

Compiling code for electron
Targeting version: 0.7.0

Including:
    examples/1-simple/SdCardLogExample.cpp
    library.properties
    src/RingBuffer.h
    src/SdCardLogHandlerRK.cpp
    src/SdCardLogHandlerRK.h
attempting to compile firmware 
downloading binary from: /v1/binaries/5c7850fc32e923407e861ce5
saving to: firmware.bin
Memory use: 
   text	   data	    bss	    dec	    hex	filename
  24844	    196	   5208	  30248	   7628	/workspace/target/workspace.elf

Compile succeeded.

#19

Found the problem. As noted earlier, the application compiles in the cloud and even compiles locally. The IDE just reports that it can’t find the reference. I noticed that I had #include “Particle.h” in my ino file. When I remove that the IDE is happy.


#20

Hello @rickkas7

I have been trying to understand the 3 topics i found on the forum regarding the use of a micro SD card with the particle photon. Aside from this one, the other two are:



(In the above disccusion the Trymefirst program is mentioned which is used to test if the SD card and photon are communicating which i do understand)

I am still relatively new to particle and this will be the first time that i am using an SD card. I am struggling to determine which to use for the purpose of logging weather data to an SD card between the serialLogHandler and SdCardLogHandlerRK methods. Aside from the syntax changes what the difference between these two and which would be the best suited for my purpose? I am sorry if this is a stupid question, if it is then i don’t quite understand how to work with SD cards. In that case can you please send me a few links that i can read to better understand this. Thanks in advance.