Webhooks with JSON arrays or nested objects escaping quotes

Hey @Dave! Is there any reason why a webhook with the following format wouldn’t work?

"json": { "myarray": "{{somedata}}" },

Where somedata is published to the webhook as an array with the following format [{"id":"1234","status":"SOMESTATE"}].

When I look at the request coming in to my HTTP server, I see something like this:

{ "myarray": "[{\"id\":\"1234\",\"status\":\"SOMESTATE\"}]" },

I’d expect that the slashes shouldn’t be part of the request body. How else can I dynamically generate an array or even a nested JSON object without having the quotes escaped?

1 Like

There’s no way to do this right now.

This is a similar issue to {{PARTICLE_EVENT_VALUE}} gets HTML encoded: how to get values that are not strings into webhooks.

I’d favor the same syntax I recommended in that other post:

Allow the use of Javascript conversion methods like JSON.parse() in the template and skipping the quotes in the generated JSON if the result is an object or array.

"json": { "myarray": "{{JSON.parse(somedata)}}" },

Actually I just realized that JSON.parse can accomodate all these use cases:

JSON.parse("10") // 10
JSON.parse("true") // true
JSON.parse("1.5") // 1.5
JSON.parse("[{\"id\":\"1234\"}]") // [{"id":"1234"}]

So @Dave, how about adding a parse filter to the webhook format processor to be able to include numbers, booleans, arrays and object in the webhook?

"json": { "myarray": "{{parse(somedata)}}" },

1 Like

Hey all!

Ooh, interesting! I suspect substituting in complex json objects (instead of just properties) might be tricky to do at the moment. The current request templating is more oriented around simple property replacement.

Essentially I think this boils down to, having a way to specify type handling for properties in the json request object like @jvanier is suggesting would fix this. I’m still mulling a format for this that keeps things straightforward, but it’s definitely on my radar. I’ll let you know when I know more!

Thanks,
David

2 Likes

Could a new templating be used for those wanting to use the JSON.parse() approach. When the webhook is created include a new property like type: “standard” or type: “parsed”. Then “parsed” requests could be handled by a different templating handler. This keeps the simple straight forward way available for everything already in place but allows for advanced control using the JSON.parse. I’m sure I’m misusing terms but the main idea is to let the template declare that it’s sending parse data so you can process that knowing the way the data will arrive. Just a thought.

Okay, combining these together, how about new section tags, something like…

#current template
{{NORMAL_PARAM}}
#proposed for JSON parsing
{{_ParseJSON}}{{NORMAL_PARAM}}{{/_ParseJSON}}

#proposed for int parsing
{{_ParseInt}}{{NORMAL_PARAM}}{{/_ParseInt}}

#proposed for float parsing
{{_ParseFloat}}{{NORMAL_PARAM}}{{/_ParseFloat}}

#etc, etc.

I think something like this should be possible to implement, and wouldn’t be too confusing. Thoughts?

Thanks!
David

1 Like

Only parseJSON is necessary since JSON.parse(“1”) gives an integer (same for float and boolean). That’s why I suggested a simple “parse” to mean “feed it through JSON.parse and include the result into the webhook JSON without quotes”.

I think this syntax is simpler to understand

"json": {
  "gauges": "{{parse(GAUGES_ARRAY)}}"
}

than

"json": {
  "gauges": "{{_Parse}}{{GAUGES_ARRAY}}{{/_Parse}}"
}
1 Like

Hi @jvanier,

Sure, that makes sense. I was thinking some type-specific overrides in case someone wanted to enforce a particular type. Having the separate tags outside the template value lets me stay inside the template language spec, instead of needing to roll my own parser which I’d like to avoid. :slight_smile:

Thanks!
David

Don’t roll your own parser for sure :smile:

Mustache templates support using lambdas using # so what about this syntax:

"json": {
  "gauges": "{{#parse}}{{GAUGES_ARRAY}}{{/parse}}"
}
1 Like

Hi All,

Just wondering if this discussion actually came to a conclusion, has anything been implemented ?

Thanks,
Serge

1 Like

Hi,
I have similar problem to described by @bitwisekaizen.
I want to use webhook as template and I want to publish JSON data that is create in .ino sketch.
ex.

"json": {{my_custom_data}}

Then in code I want to set my_custom_data to \"{\"key\": \"value\"}\"

The problem is that I recived error from particle_internal: {"detail":"You do not have permission to perform this action."}

Is there any progress in parse nested data into "json" in Webhook?

Have you been able to implement a solution to produce result into the webhook JSON without quotes?
Thanks

@Dave any update on getting this implemented? I’m happy to help with a PR if necessary, just point me in the right direction!

Hi @neuhoffm,

Thanks for the ping! Yes, I implemented a solution to this, the “data_types” param which lets you post-process values to send raw primitives, etc. I don’t know if this has been exposed in the API / but I had intended to release it soon. Lets ping @bryce who can provide insight into where this lands priority wise in the coming weeks.

Thanks,
David

1 Like

hi guys,
has the double quotes issue been reosolved?
Thanks!

1 Like