# Long JSON Strings **URL:** \ **Category:** Troubleshooting\ **Created:** [November 7, 2018, 7:37pm UTC](https://community.particle.io/t/long-json-strings/45059 "2018-11-07T19:37:09Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 7, 2018, 7:37pm UTC](https://community.particle.io/t/long-json-strings/45059/1 "2018-11-07T19:37:09Z")
Is there an easy way to parse long JSON strings? Many of the pages I am using have over 5K characters which are read into the Photon in many chunks from the webhook. Once I have the complete string, it looks like I can easily use SparkJSON to parse the string, but is building a string of over 5K characters feasible, or is there a better method? ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 7, 2018, 7:39pm UTC](https://community.particle.io/t/long-json-strings/45059/2 "2018-11-07T19:39:59Z")
Do you actually need all of the response? The Particle Cloud has the ability to extract the desired fields and only pass on what you need. Have a look at [custom response template](https://docs.particle.io/reference/device-cloud/webhooks/#responsetemplate) ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 7, 2018, 8:01pm UTC](https://community.particle.io/t/long-json-strings/45059/3 "2018-11-07T20:01:19Z")
Cool. Thanks! I have used simple response templates like the following. `"responseTemplate": "{{#data}}{{attributes.arrival_time}}{{/data}}"` I assume the following example from the documents is similar, but how do I access “lat” and “lng” in my code? ```auto "responseTemplate": { "lat": "{{{results.0.location.lat}}}", "lng": "{{{results.0.location.lng}}}" } ``` ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 7, 2018, 8:08pm UTC](https://community.particle.io/t/long-json-strings/45059/4 "2018-11-07T20:08:43Z")
Once you get the reduced JSON response onto your device you can use any JSON parser library you like (like [this](https://build.particle.io/libs/JsonParserGeneratorRK/0.0.5)) or you just parse the string yourself. ---
**Author:** ![peekay123](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/peekay123/32/810_2.png) [@peekay123](https://community.particle.io/u/peekay123)\ **Post date:** [November 7, 2018, 8:22pm UTC](https://community.particle.io/t/long-json-strings/45059/5 "2018-11-07T20:22:10Z")
You can also use @rickkas7’s great [mustache tester](http://rickkas7.github.io/mustache/) to develop your response template. It has saved me numerous hours of testing! ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 7, 2018, 8:27pm UTC](https://community.particle.io/t/long-json-strings/45059/6 "2018-11-07T20:27:38Z")
I am okay with using JSON parsers, I just don’t know how to access the specific sub-strings using the variables “lat” and “lng” as suggested in the example from the document. When I subscribe to a webhook, I have only received a single string, but the example (included below) implies that you can get two or more separate sub-strings identified by “lat” and “lng” in the example below. Just don’t know how to reference these sub-strings in code. Sorry if this is dumb question. Also, thanks for the link to the mustache tester. ---
**Author:** ![peekay123](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/peekay123/32/810_2.png) [@peekay123](https://community.particle.io/u/peekay123)\ **Post date:** [November 7, 2018, 8:51pm UTC](https://community.particle.io/t/long-json-strings/45059/7 "2018-11-07T20:51:16Z")
@stk, here is an example using openweatherapi that I use for my weather station. A sample call gets a 7-day forecast: [https://samples.openweathermap.org/data/2.5/forecast/daily?id=524901&appid=b1b15e88fa797225412429c1c50c122a1](https://samples.openweathermap.org/data/2.5/forecast/daily?id=524901&appid=b1b15e88fa797225412429c1c50c122a1) A webhook based on this API would produce this raw JSON data: ```auto {"cod":"200","message":0,"city":{"geoname_id":524901,"name":"Moscow","lat":55.7522,"lon":37.6156,"country":"RU","iso2":"RU","type":"city","population":0},"cnt":7,"list":[{"dt":1485766800,"temp":{"day":262.65,"min":261.41,"max":262.65,"night":261.41,"eve":262.65,"morn":262.65},"pressure":1024.53,"humidity":76,"weather":[{"id":800,"main":"Clear","description":"sky is clear","icon":"01d"}],"speed":4.57,"deg":225,"clouds":0,"snow":0.01},{"dt":1485853200,"temp":{"day":262.31,"min":260.98,"max":265.44,"night":265.44,"eve":264.18,"morn":261.46},"pressure":1018.1,"humidity":91,"weather":[{"id":600,"main":"Snow","description":"light snow","icon":"13d"}],"speed":4.1,"deg":249,"clouds":88,"snow":1.44},{"dt":1485939600,"temp":{"day":270.27,"min":266.9,"max":270.59,"night":268.06,"eve":269.66,"morn":266.9},"pressure":1010.85,"humidity":92,"weather":[{"id":600,"main":"Snow","description":"light snow","icon":"13d"}],"speed":4.53,"deg":298,"clouds":64,"snow":0.92},{"dt":1486026000,"temp":{"day":263.46,"min":255.19,"max":264.02,"night":255.59,"eve":259.68,"morn":263.38},"pressure":1019.32,"humidity":84,"weather":[{"id":800,"main":"Clear","description":"sky is clear","icon":"01d"}],"speed":3.06,"deg":344,"clouds":0},{"dt":1486112400,"temp":{"day":265.69,"min":256.55,"max":266,"night":256.55,"eve":260.09,"morn":266},"pressure":1012.2,"humidity":0,"weather":[{"id":600,"main":"Snow","description":"light snow","icon":"13d"}],"speed":7.35,"deg":24,"clouds":45,"snow":0.21},{"dt":1486198800,"temp":{"day":259.95,"min":254.73,"max":259.95,"night":257.13,"eve":254.73,"morn":257.02},"pressure":1029.5,"humidity":0,"weather":[{"id":800,"main":"Clear","description":"sky is clear","icon":"01d"}],"speed":2.6,"deg":331,"clouds":29},{"dt":1486285200,"temp":{"day":263.13,"min":259.11,"max":263.13,"night":262.01,"eve":261.32,"morn":259.11},"pressure":1023.21,"humidity":0,"weather":[{"id":600,"main":"Snow","description":"light snow","icon":"13d"}],"speed":5.33,"deg":234,"clouds":46,"snow":0.04}]} ``` For my application, I only need the values for dt, temp.day, temp.min, humidity and the weather icon text. Using @rickkas7’s mustache tester, I developed the following mustache filter for the Response Template: ```auto {{#list}}{{dt}}~{{temp.day}}~{{temp.min}}~{{humidity}}{{#weather}}~{{icon}}{{/weather}}/{{/list}} ``` Now, using this template, the webhook returns: ```auto 1485766800~262.65~261.41~76~01d/1485853200~262.31~260.98~91~13d/1485939600~270.27~266.9~92~13d/1486026000~263.46~255.19~84~01d/1486112400~265.69~256.55~0~13d/1486198800~259.95~254.73~0~01d/1486285200~263.13~259.11~0~13d/ ``` The tildes (~) are used to separate values and the slashes (/) are used to separate days. These were specifically included in the mustache filter. I can now parse this much smaller string using `strtok()` along with `atoi()`, `atof()` and `strncpy()` to get all the data into a daily data structure which I then use to display on an epaper display. ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 7, 2018, 10:10pm UTC](https://community.particle.io/t/long-json-strings/45059/8 "2018-11-07T22:10:25Z")
I have definitely used and parsed response templates like those described above. My basic question is, if the following is my response template, what does my firmware look like? ```auto "responseTemplate": { "lat": "{{{results.0.location.lat}}}", "lng": "{{{results.0.location.lng}}}" } ``` ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 8, 2018, 7:02am UTC](https://community.particle.io/t/long-json-strings/45059/9 "2018-11-08T07:02:58Z")
The library I linked you to features an example that illustrates the use > **[Particle Web IDE](https://build.particle.io/libs/JsonParserGeneratorRK/0.0.5/tab/example/1-parser-JsonParserGeneratorRK.cpp)** > > Write and flash code to your Particle devices > [@](#): > > ```auto > parser1.clear(); > parser1.addString(test2); > if (!parser1.parse()) { > Serial.println("parsing failed test2"); > return; > } > > String strValue; > if (!parser1.getOuterValueByKey("t1", strValue)) { > Serial.println("failed to get test2 t1"); > return; > } > > ``` Instead of `"t1"` you should be able use `"lat"` or `"lng"` ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 8, 2018, 8:36pm UTC](https://community.particle.io/t/long-json-strings/45059/10 "2018-11-08T20:36:40Z")
Thanks to all your help I am very close. The following code correctly extracts `arrive` and `id` and publishes both once using the static json string in setup. Yet using similar code in loop, I only get null values for both `arrive` and `id`. Importantly, the string published by “MBTA2” `"{ \"arr\": \"2018-11-08T15:10:49-05:00\", \"id\": \"38428527\" }"` is character for character IDENTICAL to the static json string in setup. Any suggestions? ```auto #include void getBusTimes() { Particle.publish("MBTA2"); } Timer timer(5000, getBusTimes); void setup() { timer.start(); Particle.subscribe("hook-response/MBTA2", condHandler, MY_DEVICES); StaticJsonBuffer<200> jsonBuffer; char json[] = "{ \"arr\": \"2018-11-08T15:10:49-05:00\", \"id\": \"38428527\" }"; JsonObject& root = jsonBuffer.parseObject(json); const char* arrive = root["arr"]; const char* id = root["id"]; Particle.publish("arrive", arrive); Particle.publish("time", id); } void condHandler(const char *event, const char *data) { StaticJsonBuffer<200> jsonBuffer; char copy[strlen(data)+1]; strcpy(copy, data); JsonObject& root = jsonBuffer.parseObject(copy); const char* arrive = root["arr"]; const char* id = root["id"]; Particle.publish("arrive", arrive); Particle.publish("time", id); } void loop() { } ``` I have included the response template and URL below. `{ \"arr\": \"{{{data.0.attributes.arrival_time}}}\", \"id\": \"{{{data.0.relationships.trip.data.id}}}\" }` `https://api-v3.mbta.com/predictions?filter[route]=134&filter[stop]=9147` ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 8, 2018, 9:01pm UTC](https://community.particle.io/t/long-json-strings/45059/11 "2018-11-08T21:01:01Z")
Can you `Serial.println(data)` and post here to see what you actually get as response? ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 9, 2018, 12:32pm UTC](https://community.particle.io/t/long-json-strings/45059/12 "2018-11-09T12:32:10Z")
Below is the result of `Serial.println(data)` and `Serial.println(root)` which is identical to the string produced by `Particle.publish("MBTA2")` as mentioned above and identical to the static json string in setup. `{ \"arr\": \"2018-11-09T07:25:49-05:00\", \"id\": \"38442668\" }` Kinda stumped. If `json` and `copy` are identical, how can the same code that acts on `json` and `copy` produce different results? Not sure what I am missing. ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 9, 2018, 7:10pm UTC](https://community.particle.io/t/long-json-strings/45059/13 "2018-11-09T19:10:57Z")
I would have tried your code, but for some reason I can’t import `ArduinoJson` library in Web IDE, but I’d usually rather go with the previously linked [`JsonParserGeneratorRK`](https://build.particle.io/libs/JsonParserGeneratorRK/0.0.5) anyway. ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 9, 2018, 8:20pm UTC](https://community.particle.io/t/long-json-strings/45059/14 "2018-11-09T20:20:37Z")
Still stuck. Just to simplify, I am only using serial print as suggested (see code below). Farther below, I wrote the same program using JsonPaserGeneratorRK with the same results. Below is the serial output. The static version correctly produces the parsed json arrival time. The dynamic version does not. Again the static and dynamic json strings are identical. ```auto dynamic { \"arr\": \"2018-11-09T15:10:49-05:00\", \"id\": \"38428527\" } static 2018-11-08T15:10:49-05:00 ``` ```auto #include void getBusTimes() { Particle.publish("MBTA2"); } Timer timer(5000, getBusTimes); void setup() { timer.start(); Particle.subscribe("hook-response/MBTA2", condHandler, MY_DEVICES); Serial.begin(9600); } void condHandler(const char *event, const char *data) { StaticJsonBuffer<200> jsonBuffer; char copy[strlen(data)+1]; strcpy(copy, data); StaticJsonBuffer<200> jsonBuffer1; char json1[] = "{ \"arr\": \"2018-11-08T15:10:49-05:00\", \"id\": \"38428527\" }"; JsonObject& root = jsonBuffer.parseObject(copy); const char* arrive = root["arr"]; const char* id = root["id"]; Serial.println("dynamic"); Serial.println(copy); Serial.println(arrive); JsonObject& root1 = jsonBuffer1.parseObject(json1); const char* arrive1 = root1["arr"]; const char* id1 = root1["id"]; Serial.println("static"); Serial.println(arrive1); } ``` In this version I am using JsonParserGeneratorRK. This produced the following serial output, and again did not print the parsed Json. ```auto { \"arr\": \"2018-11-09T16:00:49-05:00\", \"id\": \"38428529\" } { \"arr\": \"2018-11-09T16:00:49-05:00\", \"id\": \"38428529\" } ``` ```auto #include "Particle.h" #include "JsonParserGeneratorRK.h" JsonParserStatic<256, 20> jsonParser; void getBusTimes() { Particle.publish("MBTA2"); } Timer timer(5000, getBusTimes); void setup() { timer.start(); Particle.subscribe("hook-response/MBTA2", condHandler, MY_DEVICES); Serial.begin(9600); } void condHandler(const char *event, const char *data) { String strValue; int responseIndex = 0; const char *slashOffset = strchr(event, '/'); if (slashOffset) { responseIndex = atoi(slashOffset + 1); } if (responseIndex == 0) { jsonParser.clear(); } jsonParser.addString(data); Serial.println(data); if (jsonParser.parse()) { if (jsonParser.getOuterValueByKey("arr", strValue)) { Serial.printlnf("arrive", strValue.c_str()); } } } ``` ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 9, 2018, 8:53pm UTC](https://community.particle.io/t/long-json-strings/45059/15 "2018-11-09T20:53:39Z")
I now managed to test `ArduinoJson` by buiding via CLI and I can see the expected results. I just slightly changed the code to incorporate both fields into on publish and cut out the webhook itself. ```auto #include char txt[64]; void getBusTimes() { Particle.publish("hook-response/MBTA2", "{ \"arr\": \"2018-11-09T07:25:49-05:00\", \"id\": \"38442668\" }", PRIVATE); } Timer timer(5000, getBusTimes); void setup() { timer.start(); Particle.subscribe("hook-response/MBTA2", condHandler, MY_DEVICES); StaticJsonBuffer<200> jsonBuffer; char json[] = "{ \"arr\": \"2018-11-08T15:10:49-05:00\", \"id\": \"38428527\" }"; JsonObject& root = jsonBuffer.parseObject(json); const char* arrive = root["arr"]; const char* id = root["id"]; snprintf(txt, sizeof(txt), "arrive %s; time %s", arrive, id); Particle.publish("reply", txt, PRIVATE); } void condHandler(const char *event, const char *data) { StaticJsonBuffer<200> jsonBuffer; char copy[strlen(data)+1]; strcpy(copy, data); JsonObject& root = jsonBuffer.parseObject(copy); const char* arrive = root["arr"]; const char* id = root["id"]; snprintf(txt, sizeof(txt), "arrive %s; time %s", arrive, id); Particle.publish("reply", txt, PRIVATE); } void loop() { } ``` That’s the event I get every 5 seconds ```nohighlight arrive 2018-11-09T07:25:49-05:00; time 38442668 ``` With a slightly altered version using JsonParserGeneratorRK (see below) I get this ```nohighlight >{ "arr": "2018-11-09T07:25:49-05:00", "id": "38442668" }< arrive 2018-11-09T07:25:49-05:00 id 38442668 >{ "arr": "2018-11-09T07:25:49-05:00", "id": "38442668" }< arrive 2018-11-09T07:25:49-05:00 id 38442668 ``` ```auto #include JsonParserStatic<256, 20> jsonParser; void getBusTimes() { Particle.publish("hook-response/MBTA2", "{ \"arr\": \"2018-11-09T07:25:49-05:00\", \"id\": \"38442668\" }", PRIVATE); } Timer timer(5000, getBusTimes); void setup() { timer.start(); Particle.subscribe("hook-response/MBTA2", condHandler, MY_DEVICES); Serial.begin(9600); } void condHandler(const char *event, const char *data) { String strValue; int responseIndex = 0; jsonParser.clear(); jsonParser.addString(data); Serial.printlnf(">%s<", data); if (jsonParser.parse()) { if (jsonParser.getOuterValueByKey("arr", strValue)) { Serial.printlnf("arrive %s", (const char*)strValue); } if (jsonParser.getOuterValueByKey("id", strValue)) { Serial.printlnf("id %s", (const char*)strValue); } } } ``` So the parsing works either way. This would suggest the difference is in the data you try to parse. ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 10, 2018, 1:11pm UTC](https://community.particle.io/t/long-json-strings/45059/16 "2018-11-10T13:11:17Z")
Okay now I am really confused. I tried your example, but it only returns static results i.e. the results contained in the string you send as part of Particle.publish. I am trying to get and parse dynamic results from the MBTA API. I have had no problem parsing a static json string identical to the strings I receive from the MBTA API. I have had no problem getting a dynamic results string from the MBTA API and publishing this string from my response handler. The problem I have is parsing this dynamic string in my results handler, even though it is identical to the static string. ---
**Author:** ![ScruffR](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/scruffr/32/6952_2.png) [@ScruffR](https://community.particle.io/u/ScruffR)\ **Post date:** [November 10, 2018, 1:36pm UTC](https://community.particle.io/t/long-json-strings/45059/17 "2018-11-10T13:36:50Z")
> [@stk](#): > > I am really confused. I tried your example, but it only returns static results i.e. the results contained in the string you send as part of Particle.publish. That's because I have no webhook that would give me the dynamic result. But that shows it's _not_ the location _where_ the parsing happens (as you anticipated in one of your previous posts), but must be due to differences in the data you get from your webhook compared to the static version. > [@stk](#): > > The problem I have is parsing this dynamic string in my results handler, _ **even though it is identical to the static string.** _ It may _ **appear** _ to be the same, but since the result is different, it probably _isn't_. One suspicion is that you get the _escaped_ string back (aka "actually containing the ") which looks the same as the escaped way of _writing_ a string in C, but the backslash `\` actually never is part of the string. If you look at the `Serial.print()` of the static string, you won't see the backslashes, but in your post above they are there (but shouldn't). Your custom JSON template should _ **not** _ inject the backslashes. If you look at the sample template I linked earlier, you'll see they are _ **not** _ there either. [https://docs.particle.io/reference/device-cloud/webhooks/#responsetemplate](https://docs.particle.io/reference/device-cloud/webhooks/#responsetemplate) ---
**Author:** ![stk](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@stk](https://community.particle.io/u/stk)\ **Post date:** [November 10, 2018, 8:08pm UTC](https://community.particle.io/t/long-json-strings/45059/18 "2018-11-10T20:08:09Z")
Yes. I got it. Thanks so much! One of the problems I was having is that the response template in the webhook builder should be entered as a non-escaped string and the response template in the custom template should be entered as an escaped string. It works perfectly now.