TCP Not Receiving Whole Response

Troubleshooting
1

I’m trying to get data from an Enphase Energy device which gets stats from my solar collectors. The device is on my LAN, and if I type 10.0.1.2 into a browser, I get a web page from the device. I’ve tried using TCP, and also the HttpClient library to connect to it from my Photon. They both connect, but only the sketch using the HttpClient library returns any data. However, it only returns a small amount, and not the part I need. Here is the code I’m using,

#include "HttpClient/HttpClient.h"

unsigned int nextTime = 0;    // Next time to contact the server
HttpClient http;
http_header_t headers[] = {};
http_request_t request;
http_response_t response;



void setup() {
    Serial.begin(9600);
    delay(3000);
}



void loop() {
    if (nextTime > millis()) {
        return;
    }

    Serial.println();
    Serial.println("Application>\tStart of Loop.");
    
    request.port = 80;
    request.ip = {10,0,1,2};
    request.path = "/production";

    http.get(request, response, headers);
    Serial.print("Application>\tResponse status: ");
    Serial.println(response.status);

    Serial.print("Application>\tHTTP Response Body: ");
    Serial.println(response.body);

    nextTime = millis() + 10000;
}

The comments in the HttpClint.cpp file would seem to indicate that it should continue to read if all the data can’t be gotten in the first read, but it doesn’t seem be doing that.

2

Have you tried view-source or had a look at the communication logs in your browser?

If there is a redirect or IFRAMES or some other interaction between the host and the browser you might not get the desired response on first request.

3

The HttpClient library (0.0.5) that’s at the top of the list of public libraries in the Web IDE appears to only support 1023 bytes of data in the HTTP response. The rest is discarded. This declaration in HttpClient.h determines the buffer size:
char buffer[1024];

4

Yeah, I saw that buffer size, and tried to change it, but I ran into a problem I can’t understand. I created a new app, added .h and .cpp files to it through the + button, and copied and pasted the code from the library into those files. Unfortunately, I always get an HTTP response error of 400 when I run that app. I’ve checked it over very carefully, and I can’t see any differences between the apps, and yet one works (to a point), and one doesn’t. The code in the .ino files of both apps is identical, other than the #include which is just “HttpClient.h” in the app with the copied files, and “HttpClient/HttpClient.h” in the app with the library.

5

Yeah, I looked at the source in Safari, and I didn’t see any redirects or IFRAMES. I think the problem was the buffer size in the library as rickkas7 pointed out; now why I couldn’t change that, I don’t know. In any case, it’s now a moot point. I got the TCP client to work properly with he following code,

void setup() {
    startMillis = millis();
    Serial.begin(9600);
    delay(3000);
    Serial.println("connecting...");

    if (client.connect(server, 80)) {
        Serial.println("connected");
        client.print("GET /production HTTP/1.0\r\n");
        client.println("Content-Length: 0");
        client.println();
        }else{
            Serial.println("connection failed");
        }
}

I didn’t originally have the client print statements for the “Content-Length” or the final println(), both of which seem to be necessary. I don’t know whether these are always necessary, or this is specific for my particular server.

1 Like