@rac146, Yes it can be nice to have !
Here it isā¦ I added a getHeaderInfo function to the HttpClient. Basically it begins reading the response until it finds 1. Content Length and 2. Start of response body. Once the start of the response body is found it will return the content length as an int. You can then use this content length to read the rest of the response and know when the response body is fully read. Just a disclaimer, my C++ skills arenāt the best and Iām sure there are better ways to do this ;).
int HttpClient::getHeaderInfo()
{
unsigned int bufferPosition = 0;
bool contentLengthFound = false;
bool contentLengthComplete = false;
char contentLength[10];
unsigned int contentLengthCount = 0;
unsigned int contentLengthInt = -1;
bool complete = false;
char bufferHeader[512];
do {
if(client.available())
{
//get next char
char c = client.read();
bufferHeader[bufferPosition] = c;
bufferPosition++;
//2
//if we know the content length header is being read then the next characters
//are the actual numbers of the content length.
//once we reach a newline, we have the entire content length.
if(contentLengthFound && c != '\n')
{
contentLength[contentLengthCount] = c;
contentLengthCount++;
}
else if(contentLengthFound && c == '\n')
{
contentLengthFound = false;
contentLengthComplete = true;
String raw_contentLength(contentLength);
contentLengthInt = atoi(raw_contentLength.c_str());
}
//1
//we're looking for the content length first
//every time we get a character, convert entire buffer to String
//and look for 'Content-Length: '
if(!contentLengthFound)
{
String raw_response(bufferHeader);
int contentLengthPosition = raw_response.indexOf("Content-Length: ");
if (!contentLengthFound && contentLengthPosition != -1) {
contentLengthFound = true;
}
}
//3
//once the content length is read completetly we want to find the
//response body. This is represented by '\r\n\r\n'
if(contentLengthComplete)
{
String raw_response(bufferHeader);
int bodyPos = raw_response.indexOf("\r\n\r\n");
if (bodyPos != -1) {
complete = true;
}
}
}
} while(!complete && client.connected());
memset(&bufferHeader[0], 0, sizeof(bufferHeader));
return contentLengthInt;
}
You want to make that getHeaderInfo call at line 162 of the existing HttpClient.cpp. Now in the while loop (starting at line 190), create a count to compare the response body to the content length:
(Pseudocode)
int currentResponseCount = 0;
while (client.available()) {
char c = client.read();
buffer[bufferPosition] = c;
currentResponseCount++;
if(currentResponseCount == contentLengthCount)
{
buffer[bufferPosition] = '\0';
client.stop();
//this will stop the loop
error = true;
}
else
{
bufferPosition++;
}
}Thanks @rac146 Iāll take a look a this this weekend ! @JumpMaster do you know when the next firmware is supposed to be available ?
No idea and the recent commits have reintroduced the bug.
Iāve nearly finished porting the Arduino http client library which is more sophisticated then the one weāre using. Iām also combining the previous changes to use client.write rather than client.print. Will have a version on GitHub later tonight (UK time) if testing goes well.
2 Likes
@JumpMaster I just want to say thank you for taking over and helping Particle with this rather important (fundamental?) library. You work is greatly appreciated!
1 Like
Here is the ported Arduino library. I havenāt been able to do a massive amount of testing but HTTP GET with IP or hostname seem to work.
Feedback good or bad is welcome! 
Nice job thanks !
If you are porting Arduino library to Spark, maybe you could be interesting in that one: http://drcharlesbell.blogspot.ca/2013/04/introducing-mysql-connectorarduino_6.html
I used this library to communicate with MySQL database and it was working pretty well.
@JumpMaster I just the library that you ported and the response time is pretty good ! Great Work ! However, I tried to get a long string of data and I couldnāt get it all.
String:
[{"1":"00aeff","2":"00aeff","3":"00d9ff","4":"00d9ff","5":"03fabc","6":"03fabc","7":"00fab3","8":"00fab3","9":"00ffb7","10":"00ffb7","11": "00ff3c","12": "00ff3c","13":"00ff33","14":"00ff33","15":"00ff1e","16":"00ff1e","17":"15ff00","18":"15ff00","19":"55ff00","20":"55ff00","21":"1","22":"1","23":"0","24":"0","25":"100","26":"0","27":"3000","28":"1","29":"2","30":"2"}]
String receive:
[{"1":"00aeff","2":"00aeff","3":"00d9ff","4":"00d9ff","5":"03fabc","6":"03fabc","7":"00fab3","8":"00fab3","9":"00ffb7Is the website something thatās public so I can test? How long did it say the Content length was?
Iāve updated the .ino file with a workaround for when no Content-Length is supplied. I believe all websites should when HTTP/1.0 is requested but not all do.
Working pretty well right now, Thanks !
Works great for me. Just doing a simple, non-SSL GET, but itās very fast. And I like that I can do my own pseudo-asynchronous handling of the data as it comes in if I want. Thanks for your work!
Hello,
Has someone test it with a continuous sketch?
I am using the HttpClient of the Particle Libraries but I when I have sent 45 packets it restarted. other times (when it passes this criticial number could achieve 2175 packets (in the best case).
I need a stability http get whith no restarts. In Arduino YUN I do not have this problem, it is very easy and stable but lots of problems with Spark and Particle.
Which version are you building with? i believe there is a stability issue with 0.4.6, going back to 0.4.5 may help until the next version comes out
Hello Hootie81 I have 0.4.6. I use the particle build Web IDE to flash the code on the Photons. I do not know how to downgrade the firmware, maybe with the particle CLI ? Thanks
In the web IDE you can select which version you are building with, just click the arrow next to the device you want to flash and select the version.
1 Like
Thanks, I have changed a data type of the string to send and it seems it works.
With the changes I have increase (until about 7000 packets in best case) but I am having same troubles.
With 0.4.5, 1 packet and them, crash (red light).
It seems to be something related with memory, in the http sending process it has to be some memory which does not free later.
I will look for something related with it.
I am having this same problem with http lib.. and particle electron. I am retriving only small amount of data...
If you found better lib or solution please help.
This is my question on this forum