[Solved] Issues with string / char array length, JSON parsing

General
1

This might be me sucking at C++, but I’m stumped here.

First issue: If I make a request via HttpClient, the response.body only gets ~500 characters, even though the JSON I’m fetching is more like ~1500 characters long.

Second issue: I convert that to a char array, and I only get 10 characters, regardless of what I set the length to. (Unless I set it to less than 10, anyways.)

Third issue: if I hard code the JSON string, the conversion to a char array works as expected and I can parse the JSON… but I still don’t get the value I’m expecting, so I’m just buggered all around.

Here’s my complete code, except I removed the base64’d username and password from the Authorization header:

    #include "HttpClient/HttpClient.h"
    #include "SparkJson/SparkJson.h"
    #include "Adafruit_mfGFX/Adafruit_mfGFX.h"
    #include "Adafruit_SSD1351_Photon/Adafruit_SSD1351_Photon.h"

    // OLED - You can use any (4 or) 5 pins
    #define sclk A3
    #define mosi A5
    #define dc   D7
    #define cs   A2
    #define rst  D5

    // Color definitions
    #define    BLACK           0x0000
    #define    BLUE            0x001F
    #define    RED             0xF800
    #define    GREEN           0x07E0
    #define CYAN            0x07FF
    #define MAGENTA         0xF81F
    #define YELLOW          0xFFE0  
    #define WHITE           0xFFFF


    // Option 1: Hardware SPI - uses some analog pins, but much faster
    Adafruit_SSD1351 tft = Adafruit_SSD1351(cs, dc, rst);

    // Option 2: Software SPI - use any pins but a little slower
    //Adafruit_SSD1351 tft = Adafruit_SSD1351(cs, dc, mosi, sclk, rst);  


    HttpClient http;

    // Headers currently need to be set at init, useful for API keys etc.
    http_header_t headers[] = {
        //  { "Content-Type", "application/json" },
        { "Authorization", "Basic <snip>" }, // base64encode(username:password)
        { "Accept" , "application/json" },
        { NULL, NULL } // NOTE: Always terminate headers will NULL
    };

    http_request_t request;
    http_response_t response;


    void setup(void) {

        tft.begin();

        tft.fillScreen(BLACK);
        tft.setCursor(0,0);
        tft.setTextSize(0);
        tft.setTextColor(WHITE);

        // http://twcservice.mybluemix.net/api/weather/v2/observations/current?units=e&geocode=40.0388067%2C-84.3428471&language=en-US
        request.hostname = "twcservice.mybluemix.net";
        request.port = 80;
        request.path = "/api/weather/v2/observations/current?units=e&geocode=40.0388067%2C-84.3428471&language=en-US";
        //request.body = "{\"key\":\"value\"}";
        http.get(request, response, headers);

        tft.print("Status: ");
        tft.println(response.status);

        //response.body = "{\"metadata\":{\"language\":\"en-US\",\"transaction_id\":\"1460309558940:-660053703\",\"version\":\"1\",\"latitude\":40.04,\"longitude\":-84.34,\"units\":\"e\",\"expire_time_gmt\":1460310025,\"status_code\":200},\"observation\":{\"class\":\"observation\",\"expire_time_gmt\":1460310025,\"obs_time\":1460307900,\"obs_time_local\":\"2016-04-10T13:05:00-0400\",\"wdir\":160,\"icon_code\":26,\"icon_extd\":2690,\"sunrise\":\"2016-04-10T07:06:23-0400\",\"sunset\":\"2016-04-10T20:11:13-0400\",\"day_ind\":\"D\",\"uv_index\":3,\"uv_warning\":0,\"wxman\":\"wx4400\",\"obs_qualifier_code\":null,\"ptend_code\":2,\"dow\":\"Sunday\",\"wdir_cardinal\":\"SSE\",\"uv_desc\":\"Moderate\",\"phrase_12char\":\"Cloudy/Wind\",\"phrase_22char\":\"Cloudy/Wind\",\"phrase_32char\":\"Cloudy/Wind\",\"ptend_desc\":\"Falling\",\"sky_cover\":\"Cloudy\",\"clds\":\"OVC\",\"obs_qualifier_severity\":null,\"vocal_key\":\"OT48:OX2690\",\"imperial\":{\"wspd\":20,\"gust\":24,\"vis\":10.000,\"mslp\":1018.7,\"altimeter\":30.09,\"temp\":48,\"dewpt\":27,\"rh\":43,\"wc\":41,\"hi\":48,\"temp_change_24hour\":11,\"temp_max_24hour\":44,\"temp_min_24hour\":27,\"pchange\":-0.10,\"feels_like\":41,\"snow_1hour\":0.0,\"snow_6hour\":0.0,\"snow_24hour\":0.0,\"snow_mtd\":1.2,\"snow_season\":21.3,\"snow_ytd\":21.3,\"snow_2day\":0.7,\"snow_3day\":0.8,\"snow_7day\":0.9,\"ceiling\":7400,\"precip_1hour\":0.00,\"precip_6hour\":0.00,\"precip_24hour\":0.00,\"precip_mtd\":1.10,\"precip_ytd\":11.76,\"precip_2day\":0.29,\"precip_3day\":0.30,\"precip_7day\":0.96,\"obs_qualifier_100char\":null,\"obs_qualifier_50char\":null,\"obs_qualifier_32char\":null}}}";

        int str_len = response.body.length() + 1; 
        tft.print("Length: "); tft.println(str_len);
        char char_array[str_len];
        response.body.toCharArray(char_array, str_len);

        DynamicJsonBuffer jsonBuffer;
        JsonObject& root = jsonBuffer.parseObject(char_array);

        if (root.success()) {
            tft.print("Current Temperature: "); 
            const char* temp = root["observation"]["imperial"]["temp"];
            tft.println(temp);
        } else {
            tft.setTextColor(RED);
            tft.println("parseObject() failed");
    
            tft.setTextColor(GREEN);
            uint32_t freemem = System.freeMemory();
            tft.print("Free memory: "); 
            tft.print(freemem/1024); 
            tft.println("k");
    
            tft.setTextColor(WHITE);
            tft.println(char_array);
    
            tft.setTextColor(CYAN);
            tft.println(response.body);
        }
    }

    void loop() {

    }

The actual printout on the screen is something like

Status: 200
Length: 501
parseObject() failed
Free memory: 59k
{"metadata
{"metadata":{"language":"en-US","transaction_id":"1460309558940:-660053703","version":"1","latitude":40.04,"longitude":-84.34,"units":"e","expire_time_gmt":1460310025,"status_code":200},"observation":{"class":"observation","expire_

Or, with the hard coded response.body:

Status: 200
Length: 1423
Temp:

Finally, here’s an example of the JSON response I’m trying to parse:

{"metadata":{"language":"en-US","transaction_id":"1460309558940:-660053703","version":"1","latitude":40.04,"longitude":-84.34,"units":"e","expire_time_gmt":1460310025,"status_code":200},"observation":{"class":"observation","expire_time_gmt":1460310025,"obs_time":1460307900,"obs_time_local":"2016-04-10T13:05:00-0400","wdir":160,"icon_code":26,"icon_extd":2690,"sunrise":"2016-04-10T07:06:23-0400","sunset":"2016-04-10T20:11:13-0400","day_ind":"D","uv_index":3,"uv_warning":0,"wxman":"wx4400","obs_qualifier_code":null,"ptend_code":2,"dow":"Sunday","wdir_cardinal":"SSE","uv_desc":"Moderate","phrase_12char":"Cloudy/Wind","phrase_22char":"Cloudy/Wind","phrase_32char":"Cloudy/Wind","ptend_desc":"Falling","sky_cover":"Cloudy","clds":"OVC","obs_qualifier_severity":null,"vocal_key":"OT48:OX2690","imperial":{"wspd":20,"gust":24,"vis":10.000,"mslp":1018.7,"altimeter":30.09,"temp":48,"dewpt":27,"rh":43,"wc":41,"hi":48,"temp_change_24hour":11,"temp_max_24hour":44,"temp_min_24hour":27,"pchange":-0.10,"feels_like":41,"snow_1hour":0.0,"snow_6hour":0.0,"snow_24hour":0.0,"snow_mtd":1.2,"snow_season":21.3,"snow_ytd":21.3,"snow_2day":0.7,"snow_3day":0.8,"snow_7day":0.9,"ceiling":7400,"precip_1hour":0.00,"precip_6hour":0.00,"precip_24hour":0.00,"precip_mtd":1.10,"precip_ytd":11.76,"precip_2day":0.29,"precip_3day":0.30,"precip_7day":0.96,"obs_qualifier_100char":null,"obs_qualifier_50char":null,"obs_qualifier_32char":null}}}

Any idea what’s going on?

2

1: It’s a limitation of the HttpClient. You’ll need to copy it into your project and change the constant that determines the maximum received data size.

3: The temperature field is number, so you need to read it as one. The JSON library doesn’t convert between strings and numbers automatically:

if (root.success()) {
		int temp= root["observation"]["imperial"]["temp"];
		Serial.printlnf("Current temp %d", temp);
	}
1 Like
3

Yep, that solved all three. I’m not sure exactly why, but copying the lib and boosting the buffer size fixed both #1 and #2. (Well, I get why it fixed #1, but I still don’t understand #2…)

And then #3 was just a brainfart…

Thanks!

1 Like