Logging Data to an SD card (Argon)

maxhakk · July 8, 2019, 7:28pm

Hi,

I am trying to create a 10 sensor data logger using a Particle Argon. All of my hardware is working, but I am having trouble logging to an SD card using the argon. I have been using the sdFat and SdCardLogHandlerRK libraries. I included my code at the bottom. I am nominally able to log to my micro-SD card, but I am having two problems:

I cannot get it to create a new file. As it stands, the script that I have writes in a folder it creates on the SD card called logs in a file called "000001.txt". If I reset the Argon, it will just continue writing in that file right after the last line written on it. In an ideal world, I would like to create a new file for each time the logger is reset. I have tried a few things from the examples in the sdFat library (some of which you can see commented out), but none of them have worked. Does anyone have a way of doing this? Do I need a new library?
I have these antecedents before each line that I log onto the SD card like "0000000441 [app.sd] INFO: " (see picture). While I can deal with them later in data processing, it would be a bit easier if I could get rid of them now. Does anyone have a fix for that? Thank you!

This image should show you both the antecedents mentioned in problem 2 and the fact that the log restarts after the argon has been reset.

Here is my Code

// This #include statement was automatically added by the Particle IDE.

#include <SdCardLogHandlerRK.h>
// This #include statement was automatically added by the Particle IDE.
#include <SdFat.h>
int read_pin = A5;
int sig_pin_0 = D3;
int sig_pin_1 = D4;
int sig_pin_2 = D5;
int sig_pin_3 = D6;
int enable_pin = D2;
int val_1_temp[10];
int val_2_temp[10];
int val_3_temp[10];
int val_4_temp[10];
int val_5_temp[10];
int val_6_temp[10];
int val_7_temp[10];
int val_8_temp[10];
int val_9_temp[10];
int val_10_temp[10];
int val_1_print = 0;
int val_2_print = 0;
int val_3_print = 0;
int val_4_print = 0;
int val_5_print = 0;
int val_6_print = 0;
int val_7_print = 0;
int val_8_print = 0;
int val_9_print = 0;
int val_10_print = 0;
String val_1_string;
String val_2_string;
String val_3_string;
String val_4_string;
String val_5_string;
String val_6_string;
String val_7_string;
String val_8_string;
String val_9_string;
String val_10_string;
String value_string;
String str_sep = ", ";
int start_time = 0;
int current_time = 0;
int jj = 0;
int flag_1 = 0;
char name[24];
#define threshold 50
#define resistance 10000
#define num_itter 10
#define gain 2
SYSTEM_THREAD(ENABLED);
const int SD_CHIP_SELECT = D8;
SdFat sd;
// Only log errors and app.sd info level log messages to the SD card
// API change in 0.1.0: You must specify the size of the log handler buffer and call both logHandler.setup() and logHandler.loop()!
SdCardLogHandler<2048> sdLogHandler(sd, SD_CHIP_SELECT, SPI_FULL_SPEED, LOG_LEVEL_ERROR, {
	{ "app.sd", LOG_LEVEL_INFO }
});
// Turn off serial logging in the SdCardLogHandler to avoid logging twice
STARTUP(sdLogHandler.withNoSerialLogging());
// Use LogToSD.info() instead of Log.info(), etc. to write to the SD card
Logger LogToSD("app.sd");
// Log everything to Serial
SerialLogHandler serialLogHandler;
void publish_readings()
{
    current_time = current_time + num_itter;
    val_1_print = 0;
    val_2_print = 0;
    val_3_print = 0;
    val_4_print = 0;
    val_5_print = 0;
    val_6_print = 0;
    val_7_print = 0;
    val_8_print = 0;
    val_9_print = 0;
    val_10_print = 0;
    //Averaging readings for the past minute
    for(int ii = 1; ii <= (num_itter - 1); ii++){
        val_1_print = val_1_print + val_1_temp[ii];
        val_2_print = val_2_print + val_2_temp[ii];
        val_3_print = val_3_print + val_3_temp[ii];
        val_4_print = val_4_print + val_4_temp[ii];
        val_5_print = val_5_print + val_5_temp[ii];
        val_6_print = val_6_print + val_6_temp[ii];
        val_7_print = val_7_print + val_7_temp[ii];
        val_8_print = val_8_print + val_8_temp[ii];
        val_9_print = val_9_print + val_9_temp[ii];
        val_10_print = val_10_print + val_10_temp[ii];
    }
    val_1_print = val_1_print / num_itter;
    val_2_print = val_2_print / num_itter;
    val_3_print = val_3_print / num_itter;
    val_4_print = val_4_print / num_itter;
    val_5_print = val_5_print / num_itter;
    val_6_print = val_6_print / num_itter;
    val_7_print = val_7_print / num_itter;
    val_8_print = val_8_print / num_itter;
    val_9_print = val_9_print / num_itter;
    val_10_print = val_10_print / num_itter;
    if(val_1_print < threshold){
        // Removing low voltage values (open circuit)
        val_1_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_1_print = ((4095 * gain - val_1_print) * resistance) / val_1_print;
        val_1_string = String(val_1_print);
    }
    if(val_2_print < threshold){
        // Removing low voltage values (open circuit)
        val_2_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_2_print = ((4095 * gain - val_2_print) * resistance) / val_2_print;
        val_2_string = String(val_2_print);
    }
    if(val_3_print < threshold){
        // Removing low voltage values (open circuit)
        val_3_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_3_print = ((4095 * gain - val_3_print) * resistance) / val_3_print;
        val_3_string = String(val_3_print);
    }
    if(val_4_print < threshold){
        // Removing low voltage values (open circuit)
        val_4_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_4_print = ((4095 * gain - val_4_print) * resistance) / val_4_print;
        val_4_string = String(val_4_print);
    }
    if(val_5_print < threshold){
        // Removing low voltage values (open circuit)
        val_5_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_5_print = ((4095 * gain - val_5_print) * resistance) / val_5_print;
        val_5_string = String(val_5_print);
    }
    if(val_6_print < threshold){
        // Removing low voltage values (open circuit)
        val_6_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_6_print = ((4095 * gain - val_6_print) * resistance) / val_6_print;
        val_6_string = String(val_6_print);
    }
    if(val_7_print < threshold){
        // Removing low voltage values (open circuit)
        val_7_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_7_print = ((4095 * gain - val_7_print) * resistance) / val_7_print;
        val_7_string = String(val_7_print);
    }
    if(val_8_print < threshold){
        // Removing low voltage values (open circuit)
        val_8_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_8_print = ((4095 * gain - val_8_print) * resistance) / val_8_print;
        val_8_string = String(val_8_print);
    }
    if(val_9_print < threshold){
        // Removing low voltage values (open circuit)
        val_9_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_9_print = ((4095 * gain - val_9_print) * resistance) / val_9_print;
        val_9_string = String(val_9_print);
    }
    if(val_10_print < threshold){
        // Removing low voltage values (open circuit)
        val_10_string = "NaN";
    }
    else{
        // Converting to Resistance 
        val_10_print = ((4095 * gain - val_10_print) * resistance) / val_10_print;
        val_10_string = String(val_10_print);
    }
    value_string = String(current_time) + str_sep + val_1_string + str_sep + val_2_string + str_sep + val_3_string + str_sep + val_4_string + str_sep + val_5_string + 
    str_sep + val_6_string + str_sep + val_7_string + str_sep + val_8_string + str_sep + val_9_string + str_sep + val_10_string;
    jj = 0;
    flag_1 = 1;
}
void take_readings(){
    digitalWrite(enable_pin, LOW);
    digitalWrite(sig_pin_0, LOW);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, LOW);
    val_1_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, HIGH);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, LOW);
    val_2_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, LOW);
    digitalWrite(sig_pin_1, HIGH);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, LOW);
    val_3_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, HIGH);
    digitalWrite(sig_pin_1, HIGH);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, LOW);
    val_4_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, LOW);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, HIGH);
    digitalWrite(sig_pin_3, LOW);
    val_5_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, HIGH);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, HIGH);
    digitalWrite(sig_pin_3, LOW);
    val_6_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, LOW);
    digitalWrite(sig_pin_1, HIGH);
    digitalWrite(sig_pin_2, HIGH);
    digitalWrite(sig_pin_3, LOW);
    val_7_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, HIGH);
    digitalWrite(sig_pin_1, HIGH);
    digitalWrite(sig_pin_2, HIGH);
    digitalWrite(sig_pin_3, LOW);
    val_8_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, LOW);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, HIGH);
    val_9_temp[jj] = analogRead(read_pin);
    digitalWrite(sig_pin_0, HIGH);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, HIGH);
    val_10_temp[jj] = analogRead(read_pin);
    digitalWrite(enable_pin, HIGH);
    jj = jj + 1;
}
Timer timer_1(10000, publish_readings);
Timer timer_2(1000, take_readings);
void setup() {
    //Configuring Pins
    pinMode(read_pin, INPUT);
    pinMode(sig_pin_0, OUTPUT);
    pinMode(sig_pin_1, OUTPUT);
    pinMode(sig_pin_2, OUTPUT);
    pinMode(sig_pin_3, OUTPUT);
    pinMode(enable_pin, OUTPUT);
    pinMode(D7, OUTPUT);
    digitalWrite(sig_pin_0, LOW);
    digitalWrite(sig_pin_1, LOW);
    digitalWrite(sig_pin_2, LOW);
    digitalWrite(sig_pin_3, LOW);
    digitalWrite(enable_pin, HIGH);
    digitalWrite(D7, LOW);
    //enabling serial communications
    Serial.begin(9600);
    // getting the year
    char year[4];
    String year_str = String(Time.year());
    year_str.toCharArray(year, 4);
    // getting the month
    char month[2];
    int month_raw = Time.month();
    String month_str = String(month_raw);
    if (month_raw < 10){
        char temp_1[1];
        String temp_2 = "00";
        temp_2.toCharArray(month, 2);
        month_str.toCharArray(temp_1, 1);
        month[1] = temp_1[0];
    }
    else{
        month_str.toCharArray(month, 2);
    }
    // getting the day
    char day[2];
    int day_raw = Time.day();
    String day_str = String(day_raw);
    if(day_raw < 10){
        char temp_3[1];
        String temp_4 = "00";
        temp_4.toCharArray(day, 2);
        day_str.toCharArray(temp_3, 1);
        day[1] = temp_3[0];
    }
    else{
        day_str.toCharArray(day, 2);
    }
    //generating the file name
    String name_str = "NSF_TEST_00000000_00.txt";
    name_str.toCharArray(name, 24);
    name[9] = year[0];
    name[10] = year[1];
    name[11] = year[2];
    name[12] = year[3];
    name[13] = month[0];
    name[14] = month[1];
    name[15] = day[0];
    name[16] = day[1];
    for (uint8_t i = 0; i < 100; i++) {
        name[18] = i/10 + '0';
        name[19] = i%10 + '0';
        if (sd.exists(name)) {
            continue;
        }
        // logfile.open(name);
        break;
    }
    //get current date time
    start_time = Time.now();
    current_time = start_time;
    timer_1.start();
    timer_2.start();
    // You must call logHandler.setup() from setup()!
	sdLogHandler.setup();
    String col_names = "UNIX_Datetime_UTC, Probe_1_Resistance_Ohms, Probe_2_Resistance_Ohms, Probe_3_Resistance_Ohms, Probe_4_Resistance_Ohms, Probe_5_Resistance_Ohms, Probe_6_Resistance_Ohms, Probe_7_Resistance_Ohms, Probe_8_Resistance_Ohms, Probe_9_Resistance_Ohms, Probe_10_Resistance_Ohms";
   // Log.openLogFile(name);
    Log.info(col_names);
    LogToSD.info(col_names);
    //Serial.println(col_names);
}
void loop() {
    // You must call logHandler.loop() from loop()!
	sdLogHandler.loop();
	if(flag_1 == 1){
	    flag_1 = 0;
	    Log.info(value_string);
		LogToSD.info(value_string);
	}
}

ScruffR · July 8, 2019, 8:59pm

These "antecedents" are the timestamps (in millis) and the origin of the log statement.
If you are using the standard Log functions these are supposed to be there.

If you don't want them there you should use the SdCardPrintHanlder.

I'd also rework take_readings() into something less spaghetti-cody
You are already using arrays for val_X_temp[] why not use a two-dimensional array to rid yourself of the individual variable types for each X?
A similar thing goes for val_X_print, sig_pin_X, val_X_string.

This

maxhakk:

    for(int ii = 1; ii <= (num_itter - 1); ii++){
        val_1_print = val_1_print + val_1_temp[ii];
        val_2_print = val_2_print + val_2_temp[ii];
        val_3_print = val_3_print + val_3_temp[ii];
        val_4_print = val_4_print + val_4_temp[ii];
        val_5_print = val_5_print + val_5_temp[ii];
        val_6_print = val_6_print + val_6_temp[ii];
        val_7_print = val_7_print + val_7_temp[ii];
        val_8_print = val_8_print + val_8_temp[ii];
        val_9_print = val_9_print + val_9_temp[ii];
        val_10_print = val_10_print + val_10_temp[ii];
    }
    val_1_print = val_1_print / num_itter;
    val_2_print = val_2_print / num_itter;
    val_3_print = val_3_print / num_itter;
    val_4_print = val_4_print / num_itter;
    val_5_print = val_5_print / num_itter;
    val_6_print = val_6_print / num_itter;
    val_7_print = val_7_print / num_itter;
    val_8_print = val_8_print / num_itter;
    val_9_print = val_9_print / num_itter;
    val_10_print = val_10_print / num_itter;

could look like this

    for (int v=1; v < nVal; v++) { 
      for(int ii = 1; ii <= (num_itter - 1); ii++) 
        val_print[v] += val_temp[v][ii];

      val_print[v] /= num_itter;
    }

You are building your name string rather complicated.
Have a look at Time.format()

About the naming of the files after a reset, maybe @rickkas7 can help.

maxhakk · July 10, 2019, 3:14pm

Thanks for those suggestions. They helped a lot! I’m not the most familiar with C coding, especially for dealing with arrays. I have always been much more on the hardware side, and the data processing that I have done has been much more in R.

The SdCardPrintHandler helped a lot.

Topic		Replies	Views
[SDFat] Logging data from sensor to sd card Libraries	10	1887	July 15, 2019
Library for logging to SD card: SdCardLogHandlerRK Libraries	51	8010	January 24, 2021
Adafruit AdaLogger SD card and RTC not wroking with argon SDfat Getting Started argon	1	621	November 14, 2020
SD Data Logger - Problems At Higher Speed Firmware argon	6	629	December 2, 2020
Particle Argon Saving to micro SD Firmware	10	950	March 17, 2021

Logging Data to an SD card (Argon)

Related topics