HttpClient - connection failed (ubidots)

Hello,

Three weeks ago I saw the new “smart planter” from parrot (CES 2015) and decided to do something like that with the Spark. I bought the sensors: DHT22, photodetector and moisture and a small pump (like the one in the aquarium). The Spark read the parameters and decide when to water the plants (when there is low value of water in the sand). In the end, for reading all the parameters timeline I am sending the data to “ubidots”. It all worked fine till two days ago, when the HttpClinet start to fail.
Although I changed a little bit the code, I didn’t touch the part of the “message and sending code”. I tried to: check again the code, do reset, factory reset and burn again and after all that, try to go back to the basic and burn the HttpClient example application and it still doesn’t work. What can I do?

Hi @itayd100,

Last month we discussed in this thread about the stability of the Spark when sending continuous HTTP requests. Following the suggestions there, I re-wrote the code to connect the Spark to Ubidots (below). Basically, it uses the function “sprintf” instead of concatenating strings to setup the HTTP request body, which does a better handling of the internal memory allocation.

#include "HttpClient/HttpClient.h"

/**
* Declaring the variables.
*/
HttpClient http;

#define VARIABLE_ID "xxxxxxxxxx"
#define TOKEN "xxxxxxxxxxxxxxxxxxxxxx"

float voltage = 0.0;
char resultstr[64];

// Headers currently need to be set at init, useful for API keys etc.
http_header_t headers[] = {
      { "Content-Type", "application/json" },
      { "X-Auth-Token" , TOKEN },
    { NULL, NULL } // NOTE: Always terminate headers will NULL
};

http_request_t request;
http_response_t response;

void setup() {
    pinMode(A0, INPUT);
    request.hostname = "things.ubidots.com";
    request.port = 80;
    //Serial.begin(9600);
}

void loop() {

    // Get variable

    voltage = analogRead(A0);

    // Send to Ubidots

    request.path = "/api/v1.6/variables/"VARIABLE_ID"/values";
    sprintf(resultstr, "{\"value\":%.4f}",voltage);
    request.body = resultstr;

    http.post(request, response, headers);

    //Serial.println(response.status);

    //Serial.println(response.body);

    delay(1000);
}

Has it stopped completely? whats the core doing? a red SOS? or is it just not connecting?

there is a nice tool here you can use to test the cc3k is still working correctly… let us know how you get on

Hey @aguspg & @Hootie81,

First, thanks for the help!
Sec, for your question Hootie81, the core connected to the Spark cloud and I can post or get the core from the cloud. There is no red light or something just fail to client request. I try to change the code like aguspg suggested. The connection continues to fail. After that I burned and tried the tool of @mtnscott and still fail when trying to "Test DNS lookup". This is what I got....

DNS Test App v1.0
Written by Scott Piette October 12, 2014

SSID :
Local IP : 0.0.0.0
Mac : 00:00:00:00:00:00
Subnet mask : 0.0.0.0
Gateway IP : 0.0.0.0
DNS IP : 0.0.0.0

Enter
[0] Print IP/DNS settings
[1] WiFi Enable / Disable
--> WiFi Enable / Disable

WiFi - enabling

SSID :
Local IP : 0.0.0.0
Mac : 00:00:00:00:00:00
Subnet mask : 0.0.0.0
Gateway IP : 0.0.0.0
DNS IP : 0.0.0.0

Enter
[0] Print IP/DNS settings
[1] WiFi Enable / Disable
[2] WiFi Clear Credentials
[3] WiFi Set Credentials
[4] WiFi Connect / Disconnect
[5] Set WiFi to DHCP
[6] Set WiFi to static IP/DNS
--> WiFi Connect / Disconnect

WiFi - connecting
Waiting for WiFi to connect ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
SSID : DAGAN 2.4
Local IP : 10.0.0.10
Mac : 08:00:28:6e:42:19
Subnet mask : 255.255.255.0
Gateway IP : 10.0.0.138
DNS IP : 76.83.0.0

Enter
[0] Print IP/DNS settings
[1] WiFi Enable / Disable
[2] WiFi Clear Credentials
[3] WiFi Set Credentials
[4] WiFi Connect / Disconnect
[5] Set WiFi to DHCP
[6] Set WiFi to static IP/DNS
[7] Connect / Disconnect Cloud
[8] Test DNS lookup
--> Test DNS

Connect to data.sparkfun.com -
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Starting client.connect
Retry [10] Connect unsuccessful.

Enter
[0] Print IP/DNS settings
[1] WiFi Enable / Disable
[2] WiFi Clear Credentials
[3] WiFi Set Credentials
[4] WiFi Connect / Disconnect
[5] Set WiFi to DHCP
[6] Set WiFi to static IP/DNS
[7] Connect / Disconnect Cloud
[8] Test DNS lookup
-->

What is wrong?

It’s a thing that happens in the cc3k chip and we don’t know what causes it. It gets a strange value stuck in it that stops the core being able to look for any servers IP address from a dns server. That’s the 76.83.0.0 you see in dns IP

If you do a quick search on the forum for DNS you should see a few people that have fixed it by doing a cc3k update, sometimes it takes 4 times in a row to get it working. You have to flash tinker after too.

@itayd100
Yes, you should try updating to the latest CC3000 update.

If you use the spark-cli, and place your core in dfu-mode ( https://github.com/spark/spark-cli )
spark flash --usb cc3000

This fixed two of my cores.

1 Like

Thanks everyone!! (@Hootie81, @mtnscott, @aguspg )

I did the update like you said and the core stuck in DFU mode, I have the ST-link then I erase and burn everything and now it’s working! (the “Test DNS lookup” back to normal DNS Server).

Thanks

2 Likes

I think the stuck in dfu mode is because the factory reset gets overwritten by the cc3k patch. So you could have flashed the factory tinker and it would have worked

Hi, I’ve been having what sounds like the same problem described by @itayd100.

I have not been able to make a stable connection to Ubidots. I’ve run the example Spark code from the Ubidots website. I’ve also run the exact code above posted by @aguspg (of course using my own variable id and token). It always ends the same way. The Spark will accept the flashed firmware and begin breathing cyan. Shortly after it starts I get a SOS with a panic code or 1 flash (hard fault) followed by some green flashes, and eventually back to breathing cyan. The cycle then repeats a pretty regular intervals.

Some of the data gets through to Ubidots but not as expected based on various Delay() values I’ve tried. For example I set the delay to 1 minute (60000 ms) and the Ubidots timestamps shows data received at anywhere between 2 and 10 minutes. Most timestamps around 2 minutes apart. All the while the Spark is going through the SOS cycle.

I’ve run the DNS Test tool code, setting my own gateway, DNS server, etc. info and the tool ran without problem. It was able to ping sparkfun without problem. I tried it at lest 10 times.

I’ve also flashed the CC3000 update by running “spark flash --usb cc3000”. I ran this a few times and each time it flashed fine and without complaint. I’m a little confused about the difference between the Deep Update and the CC3000 update but for what its worth the Deep Update has been done as well.

I’m out of ideas. Any suggestions out there?

Hi @twfinmv

Just one quick thing--you are not using capital-D Delay() are you? You should use lowercase-d delay() instead. The capital-D version is an internal function that can break the cloud.

Hi @bko,

Thanks for the correction. I was actually using Delay in one of my test files (I’ve fixed it now). I hope I haven’t caused grief in the cloud!

My other test file is literally a cut and paste from from the above code from @aguspg and it correctly uses a little d delay.

1 Like

I should have said "break the cloud connection on your core." You can't easily hurt the cloud!

Is the problem there only when you try to post to Ubidots or is it a more general problem?

I have run all the examples in the EXAMPLES section of the the Spark documentation without problem. No issues with the POST or GET as written in the examples. My core has run with these simple examples for long periods without any obvious problems, dropouts or SOS. I guess there could have been problems when I wasn’t watching but none that I’ve seen.

Hi @twfinmv,

I think I was able to replicate your error; I took a Spark that I know has had DNS problems, then ran the DNS test tool by @mtnscott, then it didn’t pass the test, then I typed the “Fix DNS” option and I was able to ping the data.sparkfun.com address.

So far so good, but then I ran the example I had written above and the HTTP request didn’t come through (“HTTP failed”). So instead of dealing with the CC3000 DNS issue, I used this other code which had been discussed here before.

It uses some kind of internal DNS server inside the Spark, which can be seen in this line:

IPAddress dnsServerIP(8,8,8,8);

Here’s the code that is sending data as I write:

// These #include statements were automatically added by the Spark IDE.
#include "dnsclient/dnsclient.h"
#include "HttpClient/HttpClient.h"

/**
* Declaring the variables.
*/
HttpClient http;

#define VARIABLE_ID "54eb85fb76254xxx756158d4"
#define TOKEN "6ZImUxxxGhPWecutlSnMd8ayOpjRps"

float voltage = 0.0;
char resultstr[64];

// Headers currently need to be set at init, useful for API keys etc.
http_header_t headers[] = {
      { "Host", "things.ubidots.com" },           // Declare the host here as well when your Spark has problems with DNS resolution
      { "Content-Type", "application/json" },
      { "X-Auth-Token" , TOKEN },
    { NULL, NULL } // NOTE: Always terminate headers will NULL
};

http_request_t request;
http_response_t response;

//This are additional declarations in case your Spark is not supporting DNS resolution
IPAddress dnsServerIP(8,8,8,8);
IPAddress remote_addr;
DNSClient dns;
char serverName[] = "things.ubidots.com";

void setup() {
    pinMode(A0, INPUT);
    request.port = 80;
    dns.begin(dnsServerIP);
    dns.getHostByName(serverName, remote_addr);
    request.ip = remote_addr;
    request.path = "/api/v1.6/variables/"VARIABLE_ID"/values";
    Serial.begin(9600);
}

void loop() {

    // Get variable

    voltage = analogRead(A0);

    // Send to Ubidots

    sprintf(resultstr, "{\"value\":%.4f}",voltage);
    request.body = resultstr;
    http.post(request, response, headers);

    Serial.println(response.status);
    Serial.println(response.body);

    delay(1000);
}

Hope this helps!

1 Like

Thanks @aguspg. I will try it tonight.

Quick question though, When you re-created the failure did the symptoms include the SOS panic code to breathing cyan cycle? That is what I experienced.

Thanks for the advice.

Mmm - actually not. It only returned the “HTTP Failed” message, but no panic code.

@aguspg, when I run the code you just posted (with my own variable id and token) I get interesting results

With the delay set to 1000 ms I get the following serial output. The first time I ran it it locked out my core and I had to do a factory reset. The data on Ubidots was timestamped mostly at 2 and 3 second intervals but I did notice one timestamp that was 27 seconds later. The core was mostly breathing cyan but occasionally flashing cyan. No SOS however. The second time I ran it (after factory reset) it stuck to mostly 2-3 second update cycle, seemed more stable, and remained connected to the cloud.

HttpClient>.Connecting to IP: 50.23.124.68:80
HttpClient>.Start of HTTP Request.
POST /api/v1.6/variables/54f11aa276254256995acac3/values HTTP/1.0
Connection: close
Content-Length: 19
Host: things.ubidots.com
Content-Type: application/json
X-Auth-Token: p7D0xQvnjMfIfTdH2VrIumk7RhE7xQ1sSmOwjEm6q4calNCB76QQVYof5VC8
{"value":2127.0000}
HttpClient>.End of HTTP Request.

HttpClient>.Receiving TCP transaction of 128 bytes.
HTTP/1.1 201 CREATED
Server: nginx
Date: Sat, 28 Feb 2015 02:17:32 GMT
Content-Type: application/json
Connection: close
Vary: Accept
Location: http://things.ubidots.com/api/v1.6/values/54f1253c762542677675f71e
Allow: GET, POST, HEAD, OPTIONS

{"url": "http://things.ubidots.com/api/v1.6/values/54f1253c762542677675f71e", "value": 2127.0, "timestamp": 1425089852624, "context": {}, "created_at": "2015-02-28T02:17:32.624"}
HttpClient>.End of TCP transaction.
HttpClient>.End of HTTP Response (734ms).
HttpClient>.Status Code: 201
201
{"url": "http://things.ubidots.com/api/v1.6/values/54f1253c762542677675f71e", "value": 2127.0, "timestamp": 1425089852624, "context": {}, "created_at": "2015-02-28T02:17:32.624"}

With the delay set to 60000 ms and no other change to the code I get the following output. The first time I tried this I got regular timestamps around 62-63 seconds apart and the core remain connected for the few minutes I let it run. The second time I tried it the core was locked out almost immediately but later (2-3 minutes) resumed connection to the cloud.

HttpClient>.Connection failed.
-1

HttpClient>.Connecting to IP: 50.23.124.68:80
HttpClient>.Start of HTTP Request.
POST /api/v1.6/variables/54f11aa276254256995acac3/values HTTP/1.0
Connection: close
Content-Length: 19
Host: things.ubidots.com
Content-Type: application/json
X-Auth-Token: p7D0xQvnjMfIfTdH2VrIumk7RhE7xQ1sSmOwjEm6q4calNCB76QQVYof5VC8
{"value":2106.0000}
HttpClient>.End of HTTP Request.

HttpClient>.Error: Timeout while reading response.

HttpClient>.End of HTTP Response (5056ms).
HttpClient>.Status Code: 
HttpClient>.Error: Can't find HTTP response body.
-1

Still seems pretty unstable to me but definitely better.

Hey guys, just wanted to quickly drop a note that i played around with the testing of CC3000 patch and it improved the connections by a lot.

But it need some help here since there’s limited time for me to sit down and do serious testing.

1.) Create a gist of code that will be able to upload/post without any delays etc (treat it like the core has no issues)

2.) Provides steps on how i can create an account + view that data is being uploaded.

If you can do me a little favour, i will be able to test out whether things get better with the patch :wink:

@twfinmv I left it running without a delay and saw that it also resets every 2-3 minutes. I don’t know why it becomes unstable, it might be related to heap fragmentation, the good news is that it automatically starts working again after reset.

To me it looks like the body part is coming in after the connection is closed which will cause the hard fault. Try increasing the timeout and maybe have a look at the thread “faster http client” for an updated version I wrote a few weeks ago. I found that the print method in the old version caused some delays that really slowed down the servers response for some reason… and sometimes there was no response… Now with the much faster write method I have much more stable connections