Flexible webhooks for dynamic json data? [Solved]

EDIT: I solved my own issue using Custom Body webhook. Posted my approach below.

So I should start by saying that what I’m trying to achieve may not be possible or even advisable.

Basically I am utilising webhooks to write data to Firebase via it’s REST API.

My initial webhook’s json looked like this (and worked perfectly):

"json": {
        "tmp": "{{tmp}}",
        "rpm": "{{rpm}}",
        "volt": "{{volt}}",
        "sig": "{{sig}}",
        "ts": "{{PARTICLE_PUBLISHED_AT}}"
    },

With my publish data in this format:

sprintf(str, "{\"tmp\":%.2f,\"rpm\":%d,\"volt\":%.1f,\"sig\":%d}", 
        temp, 
        speed, 
        volt,
        signal);

This loads data into database as per the following:
image

What I would LIKE to achieve in future versions however, is a more dynamic push event where I can loop through a map of available data types with a key:value pair and publish all of these.

Something like the following pseudo-code:

for(available data) {
    addToPubStr(data.key : data.val);
}
publish(pubStr);

I played around with moustache functionality in the webhook but couldn’t get an output like I was desiring.

I’m imagining a publishString like {"key1": "value1", "key2": "value2", etc} being parsed in the same manner as the original webhook.

Any tips, advice or alternatives would be greatly appreciated!

Kind Regards,
Alex

You could use this JSON generator library for that

Thanks for the reply ScruffR. I realised I didn’t properly explain my issue or the solution I’m looking for!

I’m comfortable generating the JSON file at the firmware side, what I’m trying to do is POST (via the webhook) to firebase in a manner that it is able to parse.

For instance this is the current publishString as seen in the event monitor:
image

The webhook looks like this:

{
    "event": "pushdata",
    "url": "https://[REDACTED]/devices/{{PARTICLE_DEVICE_ID}}/pushdata.json",
    "requestType": "POST",
    "noDefaults": true,
    "rejectUnauthorized": false,
    "json": {
        "data": "{{{PARTICLE_EVENT_VALUE}}}",
        "ts": "{{{PARTICLE_PUBLISHED_AT}}}"
    },
    "query": {
        "auth": "[REDACTED]"
    }
}

Firebase interprets the publishString as just that however, a string.
image

If I POST the following JSON file manually however using curl I get the appropriate parsing into firebase.

_test.json_
{
    "data":{  
        "params":{  
            "190":606.12,
            "100":288.00,
            "158":3212.75,
            "92":11.00,
            "171":1735.00
        },
        "sig":3
    },
    "ts":"2019-01-14T01:21:16.897Z"
}
curl -X POST -H "Content-Type: application/json" -d @test.json "https://[REDACTED].firebaseio.com/push.json"

Gives me:

image

Could it be what’s going on here is that Particle’s webhook is automatically escaping my quotations?

SOLVED!

Basically I needed to ditch the json Request Format and use the Custom Body format instead.
This, coupled with some trial and error, allowed me to send the publishString without Particle escaping my quotations.

For anyone’s interest this is how the webhook now looks:

{
    "event": "pushdata",
    "responseTopic": "{{PARTICLE_DEVICE_ID}}/hook-response/{{PARTICLE_EVENT_NAME}}",
    "url": "https://[REDACTED].firebaseio.com/devices/{{PARTICLE_DEVICE_ID}}/pushdata.json",
    "requestType": "POST",
    "noDefaults": true,
    "rejectUnauthorized": false,
    "headers": {
        "Content-Type": "application/json"
    },
    "query": {
        "auth": "[REDACTED]"
    },
    "body": "{\"params\":{{{PARTICLE_EVENT_VALUE}}},\"ts\":\"{{{PARTICLE_PUBLISHED_AT}}}\"}"
}

Sending a publishString of the following:

{  
   "params":{  
      "190":599.88,
      "100":280.00,
      "158":3212.75,
      "92":11.00,
      "171":1735.00
   },
   "sig":3
}

Resulted in Firebase parsing it to:
image

Maybe this will help someone else in the future :slight_smile:

3 Likes

I'm looking to do something similar to this, just not using firebase.
Currently everything makes sense except exactly how you're publishing from the particle device.

I have a string that looks like this and changes dynamically in length

String raw_data_string; printed below is
{ "0X48": 71.94 , "0X49": 78.9 }

however when I publish it's being received as a form.....

{
"event": "pushData",
"responseTopic": "{{PARTICLE_DEVICE_ID}}/hook-response/{{PARTICLE_EVENT_NAME}}",
"url": "https://MYTESTURL.com",
"requestType": "POST",
"noDefaults": true,
"rejectUnauthorized": false,
"headers": {
"Content-Type": "application/json"
},
"body": "{"token": "SECRET_TEXT_AUTHENTICATION","variables":{{{PARTICLE_EVENT_VALUE}}} }"
}

In the .ino I am using:
Particle.publish("update_webhook", raw_data_string);

I’m not quite sure how these two things go together.
Your publishing event is update_webhook but your webhook expects pushData -
typically for a webhook to “respond” to an event the published event name and the "event" entry in the webhook definition must be compatible which update_webhook and pushData are not.

We seem to be missing some context here.

Also it’s no longer necessary to use “body” when you want to include a mustache template that is itself not valid JSON, but will be after variable replacement. You can just use “json” instead of “body” and then you don’t need the Content-Type header either.