Tutorial: Webhooks and Responses with Parsing (JSON, Mustache, Tokens)

@BulldogLowell Works like a charm. Not sure what I was doing wrong when I tried this earlier but it works now. Thanks for all the help with this, I appreciate it greatly.

@lmnichols1

Now you get:

  • Sunrise
  • Sunset
  • Current Time
  • Your UTC offset for Daylight Savings too! <<<<< Very useful, no?
1 Like

Thanks for the great tutorial. What you mentioned in this comment is exactly what Iā€™m trying to do - set the URL dynamically based on variables in my Photon code that define my location. These would get somehow passed into the webhook and be used to build the url when I call .publish(). Any leads on how to go about doing that?

@uzbhutta Look 3 posts up, in that example we pass city and state to the webhook.

Is that what you mean?

You could just as easily create a new ā€œpublishStringā€ dynamically if you wanted weather info for several cities, for example.

Yes, thatā€™s exactly what I meant. Sorry for the lack of vigilance before posting!

First of all thanks a lot for a well made tutorial, although it is brilliant I still have difficulties and spent numerous hours to get my ā€œWeather projectā€ to work. Can I yet again ask for help from this excellent comunity.

I get the hook response I want separated by ~ as below:

and try to pick out the first eight values and sum them to a +8h rain forecast with the pice of code below but I can not get it to work. Probably or as usual some missunderstanding from my side on how the commands really works (Iam learning but slow).

void mySMHIrain_Handler(const char *name, const char *data) 
{
    Particle.publish("Rain incoming data", (data));
   // String data1="0.1~0.2~0.0~0.4~0.7~0.9~0.0~0.0~"; //For testing purposes
    char rainBuffer[strlen(data)+1]; 
    strcpy(rainBuffer, data);
    float rain1 = atoi(strtok(rainBuffer, "\""));
    float rain2 = atoi(strtok(NULL, "~"));
    float rain3 = atof(strtok(NULL, "~"));
    float rain4 = atof(strtok(NULL, "~"));
    float rain5 = atof(strtok(NULL, "~"));
    float rain6 = atof(strtok(NULL, "~"));
    float rain7 = atof(strtok(NULL, "~"));
    float rain8 = atof(strtok(NULL, "~"));
    
    Particle.publish("rainBuffer", String(rainBuffer));
    delay(4000);
    
   float Precipitation=(rain1+rain2+rain4+rain5+rain6+rain7+rain8);
   //For testing purposes only
   Particle.publish("Precipitation", String(Precipitation));
    Particle.publish("Rain1", String(rain1));
    Particle.publish("Rain2", String(rain2));
    Particle.publish("Rain3", String(rain3));
    delay(4000);
    Particle.publish("Rain4", String(rain4));
    Particle.publish("Rain5", String(rain5));
    Particle.publish("Rain6", String(rain6));

        if (Precipitation !=0) 
        {
        Particle.publish("Precipitation in +8h",String(Precipitation));
        indicator_rain.attach(A5);
        indicator_rain.write(20);
        
        }
        if (Precipitation==0)
        {
        Particle.publish("No precipitation +8h",String(Precipitation));
        indicator_rain.attach(A5);
        indicator_rain.write(60);
        }
        
}

Any tip would be highly appriciated!

Your first two conversions use atoi() which would only give you integer results and ignore anything after the decimal point.
I'd also take these eight readings in a loop and and array.

Also your first strtok will give you the part from the beginning if the string to the first double quote in that string, but judging from your sample string that's not a number.
I think you want to drop the first token and only use the second and following.

Next, I'd rather round all your readings up in one Particle.publish() via snprintf(), that would also save you all those delay(4000) and would allow the subscribe handler to return in within a reasonable time (as it should).

And

rainBuffer is already a string, so no need to construct a new one :wink:
Just write Particle.publish("rainBuffer", rainBuffer);

And

The second if() { ... } would better be an else { ... } since Percipitation can be either non-zero or not, no other options exist.
Additionally indicator_rain.attach(A5) is common to both branches, so it can (should) be unconditional (outside of the if() else)

THANKS ScruffR!

I been tinkering around so much I didnā€™t see the atoi():s anymore, and many thanks for your suggestions on code enhancements I really need those (as you can see), I will look into how snprintf() works asap.

But first I still I have an issue with the extraction of data from the response, I get data alright, but every other extraction includes some readings from other parts of the dataset, not the first eight data points. Do I need to clear the data or is it the strtok that remembers and continues from last token reading when I do a new extraction?

strtok(NULL, ...) will always continue from the position where the last strtok() stopped.
If you want to set off at the begining (or any other position within the string), you need to use strtok((char*)desiredPos, ...)

Thanks again, Hmm I have to stop for the night, but it still finds data that just is not there, actually not at all in the dataset (just got an rain 1 = 8 but I have no such data in the response) I am doing something very wrong here, If you have a tip I would appreciate it.

Code is now looking like this

void mySMHIrain_Handler(const char *name, const char *data) 
{
    Particle.publish("Rain incoming data", (data));
    //String data1="1.1~0.2~2.0~0.4~0.7~0.9~0.0~0.0~"; //For testing purposes
    char rainBuffer[256]="";
    strcpy(rainBuffer, data);
    strtok(0,0);
    float rain1 = atof(strtok(rainBuffer, "~"));
    float rain2 = atof(strtok(NULL, "~"));
    float rain3 = atof(strtok(NULL, "~"));
    float rain4 = atof(strtok(NULL, "~"));
    float rain5 = atof(strtok(NULL, "~"));
    float rain6 = atof(strtok(NULL, "~"));
    float rain7 = atof(strtok(NULL, "~"));
    float rain8 = atof(strtok(NULL, "~"));
    
    
    float Precipitation=(rain1+rain2+rain3+rain4+rain5+rain6+rain7+rain8);

I would get rid of the publish at the beginning of the handler. It seems to me that you needed it once to verify that the string is what you expect. As @ScruffR suggested, parse the data and build a string back using snprintf() to validate your tokenizingā€¦ and then move on.

You want to get in and out of these webhook handlers like you would an ISR. Use flags and handle them in loop(). The indicator_rain and publish methods can likely come out, Iā€™d guess.

Like @ScruffR, Iā€™d also use an array to take the data, but I would not use floating point variables, considering the precision issues and trying compare that to zero. So, you can try multiplying the float by 10 and using an int array to store the data; precision to less than 1/10th of a millimeter is likely not necessary. If data is inches, ok take another degree of precision.

Work your way down the function validating each step and once you are certain that youā€™ve got that next piece working, get rid of the debug and focus on the next block/function/line.

You are almost there.

1 Like

Hi!
I have some questing, surfing in the web i found this Library, from bblachon:

Would you mind tell me the differences between using this library and your method.
Iā€™m new in this of parsing JSON, so I will really appreciate your help!
Thank you!

You could also use this version of SparkJSON library which is available on Particle Build already

The main difference between the two ways is convenience vs. ā€œvisibilitā€ and understandability.

Doing things ā€œby handā€ helps understanding whatā€™s going on and how things actually work.
Prebuilt libraries are convenient to use, but if you donā€™t know what they are doing and how they are working, you may get a hard time debugging if they donā€™t do what you expect.

I prefer doing things ā€œmanuallyā€ first to learn about the topic and then transition to using libraries (which I then can read and understand).

Oh, I see thank you so much!
I think that Iā€™m going to start using this method and then moving to the library, at least until I understand the process better.

1 Like

I have a hopefully simple question: why do we need to subscribe to the webhook, and publish it? It seems to me that one could just call the publish every so often in the applicaitonā€¦unless Iā€™m missing something?

The publishing event triggers the webhook while the subscribe catches the result.
Without a trigger the webhook would never fire and without the subscribe the returned result would just end up in Nirvana :wink:

Anything else would be like a telephone with only a microphone or a speaker - but it needs both to do its job properly.

3 Likes

By the way, I should point out there are a bunch of links around the forum pointing to trymustacheDOTcom which is now a shoe store. Evidently the domain owner decided selling shoes was more profitable than Mustache.
In my quest to learn the art of JSON parsing and response templates, I discovered tryhandlebarsjs.com which has helped me out a lot.

3 Likes

EDIT: I discovered from another post that Rikkas7 wrote an excellent tutorial on Mustache, and even an amazing tool for building templates. I thought I should mention that here too.

2 Likes

@LukeUSMC, thanks for the tutorial. Even three years after you published it, it is still valid. By just reading your original post I was able to create a web hook (though I used the WebIDE console to do it) and go to the TSA.gov website and get local Sunrise and Sunset info for my project. Wunderground no longer seems to offer a free API but the TSA does. Thanks again. A really big help.

1 Like
{
  "coord": {
    "lon": 10.75,
    "lat": 59.91
  },
  "weather": [
    {
      "id": 600,
      "main": "Snow",
      "description": "light snow",
      "icon": "13n"
    }
  ],
  "base": "stations",
  "main": {
    "temp": 273.71,
    "feels_like": 271.14,
    "temp_min": 273.71,
    "temp_max": 273.71,
    "pressure": 977,
    "humidity": 83
  },
  "visibility": 130,
  "wind": {
    "speed": 0.45,
    "deg": 90,
    "gust": 1.34
  },
  "snow": {
    "1h": 0.21
  },
  "clouds": {
    "all": 100
  },
  "dt": 1607040485,
  "sys": {
    "type": 3,
    "id": 2009047,
    "country": "NO",
    "sunrise": 1607068664,
    "sunset": 1607091438
  },
  "timezone": 3600,
  "id": 3143244,
  "name": "Oslo",
  "cod": 200
}

int weatherID = atoi(strtok(strBuffer, "\"~"));
    String weatherMain = strtok(NULL, "~");
    String weatherDesc = strtok(NULL, "~");
    float feelsLike = atof(strtok(NULL, "~"));
    float mintemp = atof(strtok(NULL, "~"));
    float maxtemp = atof(strtok(NULL, "~"));
    float pressure = atof(strtok(NULL, "~"));
    int mainHumidity = atoi(strtok(NULL, "~"));
    int maxwind = atoi(strtok(NULL, "~"));
    String locationCountry = strtok(NULL, "~");
    String locationCity = strtok(NULL, "~");
    int timeZone = atoi(strtok(NULL, "~"));
    String weatherName = strtok(NULL, "~");

How can i get only what i am specifying here? i struggle with mustache part. Cant figure out how to filter it.

Thank for help!!!