Webhook substitution failing

[DaveH]

I’m trying to use an Argon as a gateway to a number of Xenons sending sensor data over BLE and I want to use a webhook to forward that on to a IoT server (ThingsBoard). When you post telemetry, you select what device you want from the API by encoding an API key in the URL. For now, I have hard coded the API key for each Xenon device in it’s firmware, and now the Argon can do a Particle.publish of the following data:

{"sensorName": "BLE-Xen1"; "apiKey"="xxxxxxxxxxxxx"; "randomNumber": 39810; "temperature": 23.80; "humidity": 64.70}

I have set up the webhook as follows using the example in the docs, for variable substitution:

{
    "event": "BLEgwData",
    "deviceID": "xxxxxxxxx",
    "url": "http://myserver.net:8082/api/v1/{{{apiKey}}}/telemetry",
    "requestType": "POST",
    "noDefaults": true,
    "rejectUnauthorized": false,
    "json": "{{{PARTICLE_EVENT_VALUE}}}"
}

However the webhook call fails.

Event:
{
  "name": "BLEgwData",
  "data": "{\"sensorName\": \"BLE-Xen1\"; \"apiKey\"=\"xxxxxxxxxxxxxx\"; \"randomNumber\": 40884; \"temperature\": 23.80; \"humidity\": 65.00}",
  "ttl": 60,
  "published_at": "2021-07-03T08:24:31.981Z",
  "coreid": "xxxxxxxx"
}

Request:
POST /api/v1//telemetry HTTP/1.1
Content-Type: application/json
Accept: application/json
User-Agent: ParticleBot/1.1 (https://docs.particle.io/webhooks)
host: myserver.net:8082
content-length: 123
Connection: keep-alive

It looks like the apiKey substitution is failing and returning an empty string. Any idea why? Is it the string escaping in the received JSON?

If I could get this working, rather than hardcode the API key in the device and have that sent in the JSON (which becomes published telemetry and has the potential to expose data if I’m not careful), it would be nice if I could put all this in the webhook. Is it possible to use variables within a webhook and do something like?:

apiKeys={
  "BLE-Xen1": "xxxxxxxxxx",
  "BLE-Xen6": "yyyyyyyyyy",
}
"url": "http://myserver.net:8082/api/v1/{{{apiKeys[{{{sensorName}}}]}}}/telemetry",

[rickkas7]

The reason passing the API key is failing is that you have an equal sign as the separator between the apiKey and its value rather than a colon as required by JSON.

Creating a separate set of variables within the webhook that way is not possible.

You could use the conditional inclusion mustache block, however. You’d need to include a separate variable key for each device

The URL field would look something like this:

http://myserver.net:8082/api/v1/{{#xen1}}xxxxxxxxxx/BLE-Xen1{{/xen1}}{{#xen6}}yyyyyy/BLE-Xen6{{/xen6}}/telemetry

Instead of passing up "sensorName":"BLE-Xen1" you’d pass up xen1:true.

You’ll probably want to use the mustache template tester because it’s a little complicated and error-prone to type in the templates but it should work.

https://docs.particle.io/tutorials/device-os/json/#body-with-conditional-blocks

[DaveH]

Thanks, I’m annoyed at that = sign, I should have spotted that. Also, I was using semicolons which although valid in the RFC, a comma is the widely accepted syntax and the only one that works with the webhooks.

That works, but I struggled to get the URL field you suggested working. With a number of sensors to connect, that is going to get messy. What I thought was just changing my JSON to have a name and API key with the payload nested. Sensors would only add the name while the gateway adds the API key and then the webhook only passes on the payload. That would mean I only need to maintain a list of API keys on the gateway(s).

That gives:

Source -
{
  "sensorName": "BLE-Xen1",
  "apiKey": "xxxxxxxxxxx",
  "payload": {
    "sensorName": "BLE-Xen1",
    "randomNumber": 31274,
    "temperature": 22.8,
    "humidity": 69.3
  }
}

Moustache template - 
{
    "event": "BLEgwData",
    "deviceID": "blahblahblah",
    "url": "http://myserver.net:8082/api/v1/{{{apiKey}}}/telemetry",
    "requestType": "POST",
    "noDefaults": true,
    "rejectUnauthorized": false,
    "json": "{{{payload}}}"
}

Using the tester, this just substitutes “[object Object]” for the json to send, which causes a 500 error from the IoT server. Is there a syntax I can use to strip out and send just the payload? If I use {{{PARTICLE_EVENT_VALUE}}} in the webhook then it subs in the whole JSON.

[DaveH]

Just to add to that:

"json": "{{{PARTICLE_EVENT_VALUE}}}"  -- fine
"json": "{{{payload.temperature}}}" -- fine, or at least gives the single value
"json": {{{payload}}}" -- not fine

[DaveH]

I found a solution. As I don’t need to parse the payload in any way, just making it a string solved the issue.

{
  "sensorName": "BLE-Xen6",
  "apiKey": "xxxxxxx",
  "payload": "{'randomNumber': 7621, 'temperature': 23.10, 'humidity': 65.10}"
}

This also helped with the gateway as I didn’t need to parse and reassemble the whole structure, I just worry about the 3 values (name, apikey and payload). It picks up sensor name from the Xenon, looks that up in an array, gets the API key and reassembles the JSON structure as above. The nice thing about that is that payload can be any JSON so I can use the same gateway code for different kinds of sensors and data.