Processing HTTP response (ThingSpeak read last value)

I have a ThingSpeak channel, that is capturing a simple integer value in one field. The values might be 0, 1 or 2.

I only need the last known value from my channel, which I can get using the following url: https://thingspeak.com/channels/24587/fields/1/last

I’ve tried to use TCPClient to get this value, without success. I’ve followed the documentation but I don’t get back the single integer that I am expecting.

// Think speak channel information
String _channelID = "24587";

// TCP socket initialize
TCPClient _client;

// thingspeak IP
byte _ip[] = { 184, 106, 153, 149 };

void setup() {
    Serial.begin(9600);
    delay(10000);
    Serial.println("===Starting===");
}

void loop() {

    if (Spark.connected()){
        Serial.println("==Loop====================");

        // fetch the last value from ThingSpeak
        ThingSpeakReadLastValue();

        delay(15000);
    } 
}



void ThingSpeakReadLastValue()
{
    Serial.println("...Connecting to Thingspeak");
     
    // Connecting and sending data to Thingspeak
    if(_client.connect(_ip, 80))
    {
        Serial.println("...Connection succesful");
         
        // fetch the last value from ThingSpeak
        // https://thingspeak.com/channels/24587/fields/1/last
         
        // GET /channels/24587/fields/1/last HTTP/1.1
        // Host: thingspeak.com
        // Cache-Control: no-cache
         
        _client.println("GET /channels/" + _channelID + "/fields/1/last HTTP/1.1");
        _client.println("Host: api.thingspeak.com");
        _client.println("Content-Length: 0");
        _client.println();
     
        // This delay is pivitol without it the TCP client will often close before the data is fully sent
        delay(200);
        
        // read the values returned by the get
        Serial.println("Received value: ");
        
        char c = _client.read();
        Serial.println(c);

        Serial.println("Thingspeak value received.");
    }
    else{
        // Failed to connect to Thingspeak
        Serial.println("Unable to connect to Thingspeak.");
    }
     
    if(!_client.connected()){
        _client.stop();
    }
    _client.flush();
    _client.stop();
}

Thanks in advance.

If the Server uses HTTP for communication please Note!
You are sending HEADERS to the Server (like Content-Length and HOST). You will also receive HEADERS from the Server. So the answer from the Server (on TCP Level) may be something like :

HTTP/1.1 200 OK
Connection: close
Date: Mon, 02 Feb 2015 14:14:27 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Cache-Control: private
Content-Type: text/xml; charset=iso-8859-1
Content-Length: 31

This is your first Real DATA

So to get the Answer from the Server you have to consume all the headers (until the first EMPTY HEADER) and then the answer starts!

Maybe another Problem: you are using HTTPS (at least in your Sample URL).
Is this URL available as plain HTTP?
The Spark is not HTTPS capable

I think the Server doesn’t support HTTP. You will only get this

HTTP/1.1 301 Moved Permanently
Server: nginx/1.7.5
Date: Mon, 02 Feb 2015 14:30:19 GMT
Content-Type: text/html
Content-Length: 184
Connection: keep-alive
Location: https://thingspeak.com/channels/24587/fields/1/last

as answer. :frowning:

1 Like

Thanks @softmeter.

Do you have any example on how to "consume the headers"?

UPDATE:
I did just find this on their documentation:

HTTP Headers

If you would like to reduce the number of HTTP headers sent by our application, please add the parameter headers=false to any HTTP request.

API keys can optionally be sent via HTTP headers by setting the header name to THINGSPEAKAPIKEY and the header value to the API key.

I'll try that tonight.

@softmeter

The ThingSpeak API does support HTTP (or Regular URL as they call it: API Reference - MATLAB & Simulink).
Can HTTP be forced from TCPClient?

(The HTTPS in my example is only a comment, I don't specify to use SSL anywhere in the TCPClient specifically)

Normaly load all TCPData into a String and look for the first occurrences of 0x0d 0x0d (which marks the begin of “real data”.

The HTTPClient Library from the Community library Section of the IDE is a good starting point (worked for me).

Do you have any example on how I cab load all TCP Data into a string. All examples I've tried (including HTTPClient) using client.read() aren't working for me :blush:

UPDATE:
Actually, I just reviewed the HTTPClient code again and I haven't tried that for a few days, I'll give that a whirl again too. I'll let you know how that goes.

You can read the Bytes in a Buffer (buffer char[200]).
Then terminate the Buffer with a Zero Byte (0x00).
Now you can construct the string using the buffer as an constructor like

 char buffer[200];
 ....
 String string(buffer);

[Update]
You should also use the mentioned attribute from thingspeak to reduce the amount of Headers for the buffer!

I haven’t had the appetite for taking on a DFUtil installation, instead I’ve used the ThingSpeak IP number to GET info from my channel… further problems abound…

My channel field GET is: http://184.106.153.149/channels/24587/fields/1/last

This is currently resulting a simple “2” when requested via a browser (or POSTMAN) but on my Core the result is a 400:

> ============================
> Application>.Start of Loop.
> HttpClient>.Connecting to IP: 184.106.153.149:80
> HttpClient>.Start of HTTP Request.
> GET channels/24587/field/1.json HTTP/1.0
> Connection: close
> Accept: */*
> HttpClient>.End of HTTP Request.

> HttpClient>.Receiving TCP transaction of 128 bytes.
> HTTP/1.1 400 Bad Request
> Server: nginx/1.7.5
> Date: Mon, 09 Feb 2015 20:54:50 GMT
> Content-Type: text/html
> Content-Length: 172
> Connection: close

> <html>
> <head><title>400 Bad Request</title></head>
> <body bgcolor="white">
> <center><h1>400 Bad Request</h1></center>
> <hr><center>nginx/1.7.5</center>
> </body>
> </html>

> HttpClient>.End of TCP transaction.
> HttpClient>.Error: Timeout while reading response.

> HttpClient>.End of HTTP Response (5423ms).
> HttpClient>.Status Code: 400
> ----------------------------------Application>.Response status: 400
> ----------------------------------Application>.HTTP Response Body: <html>
> <head><title>400 Bad Request</title></head>
> <body bgcolor="white">
> <center><h1>400 Bad Request</h1></center>
> <hr><center>nginx/1.7.5</center>
> </body>
> </html>
> ============================

My SparkCore sketch uses the HttpClient from the Spark library:

//============================
// This #include statement was automatically added by the Spark IDE.
#include "HttpClient/HttpClient.h"

/**
* Declaring the variables.
*/
unsigned int nextTime = 0;    // Next time to contact the server
HttpClient http;

// Headers currently need to be set at init, useful for API keys etc.
http_header_t headers[] = {
    //  { "Content-Type", "application/json" },
    //  { "Accept" , "application/json" },
    { "Accept" , "*/*"},
    { NULL, NULL } // NOTE: Always terminate headers will NULL
};

http_request_t request;
http_response_t response;

// thing speak api IP
IPAddress server(184,106,153,149);

void setup() {
    Serial.begin(9600);
}

void loop() {
    if (nextTime > millis()) {
        return;
    }

    Serial.println();
    Serial.println("============================");
    Serial.println("Application>\tStart of Loop.");
    // Request path and body can be set at runtime or at setup.
    request.ip = server;
    //request.hostname = "www.timeapi.org";
    request.port = 80;
    //request.path = "channels/24587/fields/1/last";
    request.path = "channels/24587/field/1.json";

    // The library also supports sending a body with your request:
    //request.body = "{\"key\":\"value\"}";

    // Get request
    http.get(request, response, headers);
    
    Serial.println("----------------------------------");
    Serial.print("Application>\tResponse status: ");
    Serial.println(response.status);

    Serial.println("----------------------------------");
    Serial.print("Application>\tHTTP Response Body: ");
    Serial.println(response.body);

    nextTime = millis() + 10000;
}
//============================

Try adding a leading / at the start of the URL as per the line below.

request.path = “/channels/24587/field/1.json”;

1 Like

Yes! That was it!

// Argggghhhh!

Thank you!