Why are there gaps in the timeline?

As I’ve noted in a number of different posts in this community, I am using a Photon with some sensors to record environmental information by sending that info to ThingSpeak every 5 minutes.

I am generally happy with it working now but it could be better. Every now and then, there are gaps in the timeline. That is, instead of 5 minutes between values, there are 10 min and even 15 min gaps. So, sometimes the values do not get published to ThingSpeak.


#include "HttpClient.h"
#include "HTU21D/HTU21D.h"

#include "ThingspeakAPIKey.h"

#define PREVENT_SLEEP D7
#define LIGHT_PIN A0

SYSTEM_MODE(SEMI_AUTOMATIC);

char publish_str[40];
unsigned long sleeptime = (60 * 5); // specify seconds and not milliseconds! // Send data every 5 minutes.

TCPClient client;

HTU21D htu = HTU21D();

unsigned long starttime;

void setup() {
    starttime = millis();
    
    pinMode(PREVENT_SLEEP, INPUT);
    pinMode(LIGHT_PIN, INPUT);
    
    Serial.begin(115200);

    while(!htu.begin()){
        Serial.println("Waiting for HTU21D to start.");
        delay(100);
    }
    Serial.println("HTU21D OK");

    // ThingsSpeak restriction: Can only publish every 15 seconds.
    if (sleeptime < 15)
        sleeptime = 15;
}

void loop() {
    Serial.println("===============================================================================================");

    if(!Spark.connected())
        ConnectPhoton();
    
    if (!WiFi.ready())
        ConnectWiFi();
    
    Serial.print("Local IP: ");
    Serial.println(WiFi.localIP());
    
    ThingSpeakUpdate();

    delay(500); // things seem to be happier with this delay

    unsigned long currentmillis = millis();
    unsigned long actualsleeptime = (sleeptime * 1000) - (currentmillis - starttime);
    
    if (actualsleeptime > sleeptime*1000)
        actualsleeptime = sleeptime*1000;
    
    Serial.print("Actual sleep time is ");
    Serial.print((int)actualsleeptime/1000);
    Serial.println(" seconds.");

    delay(2000); // Wait for serial to finish
    actualsleeptime = actualsleeptime - 2;

    if (digitalRead(PREVENT_SLEEP) != HIGH) {
        Serial.println("Going to sleep now.");
        System.sleep(SLEEP_MODE_DEEP, (int)(actualsleeptime/1000)); // When Photon wakes up, it will start with setup() again. // Sleep time is in seconds and not milliseconds!
    }
    else {
        Serial.println("D7 is high, prevent deep sleep.");
        delay(actualsleeptime);
        
        starttime = millis();
    }
}

void ConnectWiFi() {
    WiFi.connect();
    while(!WiFi.connecting())
    {
        delay(100);
        Serial.println("Waiting for WiFi.connect.");
    }
}

void ConnectPhoton() {
    Spark.connect();
    while(!Spark.connected())
    {
        delay(100);
        Serial.println("Waiting for Spark.connected.");
    }
}

float GetTemperature() {
    return htu.readTemperature();
}

float GetHumidity() {
    return htu.readHumidity();
}

int GetLight() {
    return map(analogRead(LIGHT_PIN), 0, 4095, 0, 100);
}

void ThingSpeakUpdate() {
    float datatmp = GetTemperature();
    float datahum = GetHumidity();
    int datalight = GetLight();
    
    Serial.println("ThingSpeak querystring: field1=" + String(datahum) + "&field2=" + String(datatmp) + "&field3=" + String(datalight));
    Serial.println("...Connecting to Thingspeak");

    // Connecting and sending data to Thingspeak
    if (client.connect("api.thingspeak.com", 80)) {
        Serial.println("...Connection succesful, updating datastreams");

        client.print("POST /update HTTP/1.1\n");
        client.print("Host: api.thingspeak.com\n");
        client.print("Connection: close\n");
        client.print("X-THINGSPEAKAPIKEY: " + String(writeAPIKey) + "\n");
        client.print("Content-Type: application/x-www-form-urlencoded\n");
        client.print("Content-Length: ");
        client.print(("field1=" + String(datahum) + "&field2=" + String(datatmp) + "&field3=" + String(datalight)).length());
        client.print("\n\n");
        client.println("field1=" + String(datahum) + "&field2=" + String(datatmp) + "&field3=" + String(datalight)); //the "ln" is important here.

        // This delay is pivitol without it the TCP client will often close before the data is fully sent
        delay(300);
        
        Serial.println("...Thingspeak update sent.");
    }
    else {
        // Failed to connect to Thingspeak
        Serial.println("...Connection failed!!!!!!");
    }

    if (!client.connected()) {
        client.stop();
    }
    client.flush();
    client.stop();
    Serial.println("...Connection Stop");
}

Can somebody see where I might improve this code to more reliably get a gapless timeline?

Thanks for any and all help. :smile:

P.s. Should I have posted the code on Github and just liked to it?

Have you tried running your code with the Thingspeak call removed, and just watching serial to see if it is consistent? You could quickly code up something that tells you how much time has passed between each measurement as a replacement for the Thingspeak post function. That should give you a clue about where the inconsistency is coming from… it could be an flaky Internet connection.

1 Like

Thank you @Awake.

Yes, I always get results when to the serial port when I pull D7 high.

But, I don’t know if it is because of flakey internet connect or if there was just not enough time to publish the results before the Photon goes to sleep… or something different altogether.

I am just too inexperienced to really know what is going on. I say this because I noticed that the Photon will go to sleep even when not all data has been output to the serial monitor. That is why I added this line… which seems to be enough time for the data to be sent to the serial monitor:

delay(2000); // Wait for serial to finish

Maybe it is a matter of waiting for the response before I put the Photon to sleep.

Again, thanks for your input. :smile:

@EricBrian , don’t you want to connect WiFi first, then the spark cloud? And use while(WiFi.connecting()), not while(!WiFi.connecting()), in your ConnectWiFi function?
Anyway, that’s what I do and seems to work for me.

Since you are using the HttpClient library, you can both send and receive. What you need to do is confirm that there was a valid update by checking the return value of the Thingspeak post. If it is ZERO, the post failed and you should try again.

Your Thingspeak function call does not make this easy, since it returns ‘Void’ rather than a status. So you need to start by being able to check for a return value.

You are already using the HttpClient library, maybe you should experiment sending data directly through it by using the Thingspeak API http://community.thingspeak.com/documentation/api/ , even if as a good learning experience. It took me a few hours to get familiar with the library and the API, but it was worth it.

1 Like