Cloud JSON Parsing

Continuing the discussion from Sending POST requests from the Core:

@peekay123

Are the instructions on how to do this available anywhere yet?

@BulldogLowell, we don't have instructions yet but here is how it works in an openweathermap example.

  • I created a webhook to get a 7-day JSON-formatted forecast for my city using openweathermap. The webhook uses a POST and query form to get the forecast. This is the equivalent http request (I suggest looking at the full response):

http://api.openweathermap.org/data/2.5/forecast/daily?q=Ottawa,ON&mode=json&units2metric&cnt=7

  • The JSON template for the multi-day forecast data is described HERE.

  • The large response (~ 2KB) meant about four 512-byte webhook responses were required and in order to properly parse the JSON payload, the entire message would need to be stored. This represents a LOT of RAM resources and at one webhook response per second, the weather would take a minimum of 4 seconds to receive! The only data I was looking for was:

  • Daily high temperature (temp.day)

  • Daily weather condition code (id)

  • Daily weather parameter, eg. cloudy (main)

  • To use the JSON parsing feature of the webhook, I used a "responseTemplate" entry in the webhook definition. A parsing template using "mustache" syntax is used. For the above data, the mustache template is:

    "responseTemplate": "{{#list}}{{temp.day}}{{#weather}}~{{id}}~{{main}}~{{/weather}}{{/list}}"

Note that I added a "~" (tilde) between data elements to make parsing simper in the code. An example of the resulting response from the webhook looks like this:

"7.09~800~Clear~14.95~800~Clear~18.64~500~Rain~13.25~501~Rain~13.78~500~Rain~15.92~500~Rain~12.55~501~Rain~"

The quotes are also included and hopefully @Dave can look into that.

  • The final webhook.json file I used to create the webhook looks like this:
{
"eventName": "some_event_name",
"url": "http://api.openweathermap.org/data/2.5/forecast/daily",
"requestType": "POST",
"headers": null,
"query": {
	"q": "Ottawa,ON",
	"mode": "json",
	"units": "metric",
	"cnt": 7
	},
"responseTemplate": "{{#list}}{{temp.day}}{{#weather}}~{{id}}~{{main}}~{{/weather}}{{/list}}",
"json": null,
"auth": null,
"mydevices": true
}

Using SparkCLI to create and trigger the webhook (via a publish command) coupled with using the new Spark Dashboard allowed me to quickly develop the correct mustache template. Hope that helps for now! :smiley:

4 Likes

Thanks @peekay123,

I appreciate the help, very nice.

I’ll try it this week!

2 Likes

This looks more complicated than I thought. I am trying to sift through a bunch of JSON or XML data and then pull out the nuggets, sending the pertinent values to the spark core for processing. Eventually, I’d like to make decisions in the cloud before sending commands to the core but for now the logic is fully on the core. The response looks like this (huge).

All I want is the qpf in U.S. units and the pop for each of the next three hours. Will this help me do that? I’ll buy cookies for anyone who can step me through this :cookie: :blush:

OMG - the response is so big that it won’t let me post all of the response here so I had to shorten it - quite simply, it goes way too many hours into the future to be really useful at this point.

{
  "response": {
  "version":"0.1",
  "termsofService":"http://www.wunderground.com/weather/api/d/terms.html",
  "features": {
  "hourly": 1
  }
              }
                             ,
              "hourly_forecast": [
                             {
                             "FCTTIME": {
                             "hour": "23","hour_padded": "23","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "22","mday_padded": "22","yday": "294","isdst": "1","epoch": "1382497200","pretty": "11:00 PM EDT on October 22, 2013","civil": "11:00 PM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Tuesday","weekday_name_night": "Tuesday Night","weekday_name_abbrev": "Tue","weekday_name_unlang": "Tuesday","weekday_name_night_unlang": "Tuesday Night","ampm": "PM","tz": "","age": ""
                             },
                             "temp": {"english": "48", "metric": "9"},
                             "dewpoint": {"english": "34", "metric": "1"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_chancerain.gif",
                             "fctcode": "12",
                             "sky": "86",
                             "wspd": {"english": "2", "metric": "3"},
                             "wdir": {"dir": "NE", "degrees": "45"},
                             "wx": "Chance of Light Rain",
                             "uvi": "0",
                             "humidity": "56",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "48", "metric": "9"},
                             "qpf": {"english": "0.01", "metric": "0.25"},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.86", "metric": "1011"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "0","hour_padded": "00","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382500800","pretty": "12:00 AM EDT on October 23, 2013","civil": "12:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "46", "metric": "8"},
                             "dewpoint": {"english": "35", "metric": "1"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_chancerain.gif",
                             "fctcode": "12",
                             "sky": "86",
                             "wspd": {"english": "2", "metric": "4"},
                             "wdir": {"dir": "NE", "degrees": "45"},
                             "wx": "Chance of Light Rain",
                             "uvi": "0",
                             "humidity": "64",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "46", "metric": "8"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.86", "metric": "1011"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "1","hour_padded": "01","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382504400","pretty": "1:00 AM EDT on October 23, 2013","civil": "1:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "45", "metric": "7"},
                             "dewpoint": {"english": "35", "metric": "2"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_chancerain.gif",
                             "fctcode": "12",
                             "sky": "86",
                             "wspd": {"english": "3", "metric": "4"},
                             "wdir": {"dir": "NE", "degrees": "45"},
                             "wx": "Chance of Light Rain",
                             "uvi": "0",
                             "humidity": "71",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "45", "metric": "7"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.86", "metric": "1011"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "2","hour_padded": "02","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382508000","pretty": "2:00 AM EDT on October 23, 2013","civil": "2:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "43", "metric": "6"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Rain",
                             "icon": "rain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_rain.gif",
                             "fctcode": "13",
                             "sky": "97",
                             "wspd": {"english": "3", "metric": "5"},
                             "wdir": {"dir": "East", "degrees": "99"},
                             "wx": "Definite Light Rain",
                             "uvi": "0",
                             "humidity": "79",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "43", "metric": "6"},
                             "qpf": {"english": "0.01", "metric": "0.25"},
                             "snow": {"english": "0.00", "metric": "0.00"},
                             "pop": "100",
                             "mslp": {"english": "29.82", "metric": "1009"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "3","hour_padded": "03","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382511600","pretty": "3:00 AM EDT on October 23, 2013","civil": "3:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "41", "metric": "5"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Rain",
                             "icon": "rain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_rain.gif",
                             "fctcode": "13",
                             "sky": "97",
                             "wspd": {"english": "4", "metric": "6"},
                             "wdir": {"dir": "East", "degrees": "99"},
                             "wx": "Definite Light Rain",
                             "uvi": "0",
                             "humidity": "83",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "41", "metric": "5"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.82", "metric": "1009"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "4","hour_padded": "04","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382515200","pretty": "4:00 AM EDT on October 23, 2013","civil": "4:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "39", "metric": "4"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Rain",
                             "icon": "rain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_rain.gif",
                             "fctcode": "13",
                             "sky": "97",
                             "wspd": {"english": "4", "metric": "7"},
                             "wdir": {"dir": "East", "degrees": "99"},
                             "wx": "Definite Light Rain",
                             "uvi": "0",
                             "humidity": "88",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "39", "metric": "4"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.82", "metric": "1009"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "5","hour_padded": "05","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382518800","pretty": "5:00 AM EDT on October 23, 2013","civil": "5:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "37", "metric": "3"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Rain",
                             "icon": "rain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_rain.gif",
                             "fctcode": "13",
                             "sky": "98",
                             "wspd": {"english": "5", "metric": "8"},
                             "wdir": {"dir": "NE", "degrees": "50"},
                             "wx": "Definite Light Rain",
                             "uvi": "0",
                             "humidity": "92",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "37", "metric": "3"},
                             "qpf": {"english": "0.07", "metric": "1.78"},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.80", "metric": "1009"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "6","hour_padded": "06","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382522400","pretty": "6:00 AM EDT on October 23, 2013","civil": "6:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "38", "metric": "3"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Rain",
                             "icon": "rain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_rain.gif",
                             "fctcode": "13",
                             "sky": "98",
                             "wspd": {"english": "6", "metric": "9"},
                             "wdir": {"dir": "NE", "degrees": "50"},
                             "wx": "Definite Light Rain",
                             "uvi": "0",
                             "humidity": "90",
                             "windchill": {"english": "-6654", "metric": "-6665"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "38", "metric": "3"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.80", "metric": "1009"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "7","hour_padded": "07","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382526000","pretty": "7:00 AM EDT on October 23, 2013","civil": "7:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "38", "metric": "4"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Rain",
                             "icon": "rain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/nt_rain.gif",
                             "fctcode": "13",
                             "sky": "98",
                             "wspd": {"english": "6", "metric": "10"},
                             "wdir": {"dir": "NE", "degrees": "50"},
                             "wx": "Definite Light Rain",
                             "uvi": "0",
                             "humidity": "89",
                             "windchill": {"english": "-3309", "metric": "-3331"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "38", "metric": "4"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.80", "metric": "1009"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "8","hour_padded": "08","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382529600","pretty": "8:00 AM EDT on October 23, 2013","civil": "8:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "39", "metric": "4"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/chancerain.gif",
                             "fctcode": "12",
                             "sky": "99",
                             "wspd": {"english": "7", "metric": "11"},
                             "wdir": {"dir": "NNW", "degrees": "339"},
                             "wx": "Chance of Light Rain",
                             "uvi": "0",
                             "humidity": "87",
                             "windchill": {"english": "34", "metric": "1"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "34", "metric": "1"},
                             "qpf": {"english": "0.07", "metric": "1.78"},
                             "snow": {"english": "0.00", "metric": "0.00"},
                             "pop": "100",
                             "mslp": {"english": "29.85", "metric": "1010"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "9","hour_padded": "09","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382533200","pretty": "9:00 AM EDT on October 23, 2013","civil": "9:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "40", "metric": "5"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/chancerain.gif",
                             "fctcode": "12",
                             "sky": "99",
                             "wspd": {"english": "8", "metric": "13"},
                             "wdir": {"dir": "NNW", "degrees": "339"},
                             "wx": "Chance of Light Rain",
                             "uvi": "0",
                             "humidity": "82",
                             "windchill": {"english": "-3309", "metric": "-3331"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "40", "metric": "5"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.85", "metric": "1010"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "10","hour_padded": "10","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382536800","pretty": "10:00 AM EDT on October 23, 2013","civil": "10:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "42", "metric": "5"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/chancerain.gif",
                             "fctcode": "12",
                             "sky": "99",
                             "wspd": {"english": "9", "metric": "14"},
                             "wdir": {"dir": "NNW", "degrees": "339"},
                             "wx": "Chance of Light Rain",
                             "uvi": "0",
                             "humidity": "77",
                             "windchill": {"english": "-6654", "metric": "-6665"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "42", "metric": "5"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "100",
                             "mslp": {"english": "29.85", "metric": "1010"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "11","hour_padded": "11","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382540400","pretty": "11:00 AM EDT on October 23, 2013","civil": "11:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "43", "metric": "6"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/chancerain.gif",
                             "fctcode": "12",
                             "sky": "81",
                             "wspd": {"english": "10", "metric": "16"},
                             "wdir": {"dir": "NW", "degrees": "319"},
                             "wx": "Slight Chance Light Rain",
                             "uvi": "1",
                             "humidity": "72",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "43", "metric": "6"},
                             "qpf": {"english": "0.03", "metric": "0.76"},
                             "snow": {"english": "", "metric": ""},
                             "pop": "60",
                             "mslp": {"english": "29.93", "metric": "1013"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "12","hour_padded": "12","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "23","mday_padded": "23","yday": "295","isdst": "1","epoch": "1382544000","pretty": "12:00 PM EDT on October 23, 2013","civil": "12:00 PM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Wednesday","weekday_name_night": "Wednesday Night","weekday_name_abbrev": "Wed","weekday_name_unlang": "Wednesday","weekday_name_night_unlang": "Wednesday Night","ampm": "PM","tz": "","age": ""
                             },
                             "temp": {"english": "45", "metric": "7"},
                             "dewpoint": {"english": "36", "metric": "2"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/chancerain.gif",
                             "fctcode": "12",
                             "sky": "81",
                             "wspd": {"english": "11", "metric": "18"},
                             "wdir": {"dir": "NW", "degrees": "319"},
                             "wx": "Slight Chance Light Rain",
                             "uvi": "1",
                             "humidity": "67",
                             "windchill": {"english": "-9998", "metric": "-9998"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "45", "metric": "7"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "60",
                             "mslp": {"english": "29.93", "metric": "1013"}
                             }
                             ,
                             {
                             "FCTTIME": {
                             "hour": "10","hour_padded": "10","min": "00","sec": "0","year": "2013","mon": "10","mon_padded": "10","mon_abbrev": "Oct","mday": "24","mday_padded": "24","yday": "296","isdst": "1","epoch": "1382623200","pretty": "10:00 AM EDT on October 24, 2013","civil": "10:00 AM","month_name": "October","month_name_abbrev": "Oct","weekday_name": "Thursday","weekday_name_night": "Thursday Night","weekday_name_abbrev": "Thu","weekday_name_unlang": "Thursday","weekday_name_night_unlang": "Thursday Night","ampm": "AM","tz": "","age": ""
                             },
                             "temp": {"english": "41", "metric": "5"},
                             "dewpoint": {"english": "30", "metric": "0"},
                             "condition": "Chance of Rain",
                             "icon": "chancerain",
                             "icon_url":"http://icons-ak.wxug.com/i/c/k/chancerain.gif",
                             "fctcode": "12",
                             "sky": "59",
                             "wspd": {"english": "9", "metric": "14"},
                             "wdir": {"dir": "West", "degrees": "259"},
                             "wx": "Slight Chance Light Snow Showers , Areas of Frost",
                             "uvi": "0",
                             "humidity": "65",
                             "windchill": {"english": "-6654", "metric": "-6665"},
                             "heatindex": {"english": "-9998", "metric": "-9998"},
                             "feelslike": {"english": "41", "metric": "5"},
                             "qpf": {"english": "", "metric": ""},
                             "snow": {"english": "", "metric": ""},
                             "pop": "20",
                             "mslp": {"english": "30.12", "metric": "1020"}
                             }
              ]
}

@ruben, if you give me the link to the site you are querying, I can get info on their JSON format. I can’t help if I don’t know the full JSON format.

Cheers peekay,

the info is here:
http://www.wunderground.com/weather/api/d/docs?d=data/hourly

@ruben, can you give your query minus your key? I will look ate the data and see if I can build a filter for the data you want.

Thanks PK ( @peekay123)

I’ve been using the following for JSON:
http://api.wunderground.com/api/tHiSiSmYkEy/hourly/q/OH/Cincinnati.json
I have not used this but it is also available for JSON
GET /api/tHiSiSmYkEy/hourly/q/OH/Cincinnati.json HTTP/1.1

XML:
http://api.wunderground.com/api/MyKEYisTHIs/hourly/q/OH/Cincinnati.xml

Also sending you a quick PM

@ruben, here is the webhook json file with the JSON filter you need to extract the data you specificed:

{
"event": "test1",
"url": "http://api.wunderground.com/api/yourkey/hourly/q/OH/Cincinnati.json",
"requestType": "POST",
"headers": null,
"query": null,
"responseTemplate": "{{#response}}{{#hourly_forecast}}{{FCTTIME.hour}}~{{qpf.english}}~{{pop}}~{{/hourly_forecast}}{{/response}}",
"json": null,
"auth": null,
"mydevices": true
}

The filer works this way:

  • the root object is response which leads to {{#response}} to “open” it
  • the object under the root where all your data is hourly_forecast which leads to {{#hourly_forecast}} to open it
  • the first item you want is hour under FCTTIME which leads to {{FCTTIME.hour}}
  • the ~ is included in the filtered string and acts as a separator
  • the second item you want is english under qpf which leads to {{qpf.english}}
  • another ~ separator
  • the final item you want is pop which leads to {{pop}}
  • a final ~ as separator
  • The {{/hourly_forecast}}{{/response}} are used to “close” the two open objects

You are left with 36 hours of data or one response “payload” sent by the cloud. Of this data, you keep only the first 28 bytes being the next 3 hours of data. You then parse that data with the ~ separating the hourly data sets. :smiley:

5 Likes

Outstanding explanation!

In order to keep only the first 28 bytes would you recommend looking at the packet number a la the example
http://docs.spark.io/webhooks/#handling-web-responses

or should I stream the response and drop off anything after the first three using something like strtok like described here?

or is there an easier way of just counting bytes as they pass through the buffer and copying the first 28 to a char array or string? Or am I misunderstanding.

To those who don’t know, you take this text including the curly braces and copy it into a blank xxx.json file in any directory on your computer. You open your terminal (or powershell in my case), nav to the folder w the file and issue the command spark webhook create xxx.json
An example is here http://docs.spark.io/webhooks/#lets-make-some-graphs-an-example-librato-webhook

@ruben, the response will be a String payload and I realized my comment on the first 28 bytes was too specific since the values in the string may change in size.

The best approach is to parse using the strtok() approach, looking for the first 3 sets of numbers. You can use a different delimiter for the “end” item in your webhook if that helps. Note that using a comma seemed to cause problems on the webhook parsing side so avoid those. Another thing to note is that the entire return payload will start and end with double quotes ("). :grinning:

I have a problem… not sure why this would happen. Just a minute ago, wunderground said that I made 11 API calls in less than 1 minute.
My device isn’t even plugged in.
Not sure how this is possible.
Can the webhook get called when my device isn’t online? Is it somehow public?
I only shared the key with @peekay123 privately so I’m not sure how this is happening (barring testing by peekay).

Any ideas?

I think I know the reason. I created a spark webhook with the title “get_weather” like the example documentation but I changed what the webhook did. Is it possible that as other people were trying the example they invoked this? I since then changed the title but that webhook has not been called :wink:
Still, other people shouldn’t have been able to call my webhook, even if it has the same name.

@Zachary Is this a possible bug? I killed the webhook so I’ll see if this flat-lines now.

Thanks for the report @ruben, possibly so. @Dave is the creator of the webhook functionality and should know for certain.

1 Like

Hi @ruben,

Good question! When you setup your webhook, make sure you set "mydevices": true. This will tell the webhook to only react to events from your private event stream, and not other people’s events. Webhooks is still somewhat new, so it’s possible there is a bug there, can you try creating your hook with that and see if its running as you expect?

Thanks,
David

Ah. mydevices:true. I don’t recall doing that. Is that in the docs example?

if you used the cli spark webhooks GET, then I’m guessing it didn’t put the parameter in there by default, but it’s in the docs here: http://docs.spark.io/webhooks/#webhook-options-mydevices

I agree this is a bit confusing, I think maybe I’ll switch it to be on by default instead of off by default.

Thanks!
David

1 Like

:+1:

1 Like

Hello folks,

I’m trying to parse the response I get from the weather server, so that I only get the temperature back.

@peekay123, I tried to follow the response structure you describe above but it isn’t working for me:

{
"eventName": "get_weather",
"url": "http://api.openweathermap.org/data/2.5/weather",
"requestType": "GET",	
"headers": null,
"query": {
	"q": "Chicago,US",
	"units": "imperial",
	"mode": "json",
},
"responseTemplate": "{{#main}}{{temp}}~{{/main}}",
"json": null,
"auth": null,
"mydevices": true
}

Using this webhook I still get the full response from the server:

{"name":"hook-response/get_weather/0","data":"\"{\\\"coord\\\":{\\\"lon\\\"
:-87.65,\\\"lat\\\":41.85},\\\"sys\\\":{\\\"message\\\":1.6062,\\\"country\\\":\\\"US\\\",\\\"sunris
e\\\":1433499371,\\\"sunset\\\":1433553730},\\\"weather\\\":[{\\\"id\\\":802,\\\"main\\\":\\\"Clouds
\\\",\\\"description\\\":\\\"scattered clouds\\\",\\\"icon\\\":\\\"03d\\\"}],\\\"base\\\":\\\"statio
ns\\\",\\\"main\\\":{\\\"temp\\\":51.62,\\\"temp_min\\\":51.62,\\\"temp_max\\\":51.62,\\\"pressure\\
\":1008.57,\\\"sea_level\\\":1030.85,\\\"grnd_level\\\":1008.57,\\\"humidity\\\":95},\\\"wind\\\":{\
\\"speed\\\":9.85,\\\"deg\\\":10},\\\"clouds\\\":{\\\"all\\\":36},\\\"dt\\\":1433508135,\\\"id\\\":4
887398,\\\"name\\\":\\\"Chicago\\\",\\\"cod\\\":200}\\n\"","ttl":"60","published_at":"2015-06-05T13:
05:44.517Z","coreid":"null"}

Could anybody tell me what I’m doing wrong?

Thanks!