Getting data from the Particle Cloud to RabbitMQ

Hello,
I am reading a set of sensors via a serial port, and using the Particle.Publish to send data to the Particle cloud. I run a RabbitMQ (AMQP) server, and need to ultimately get the data into my RabbitMQ server. Does anyone have any thoughts as to how to accomplish this?

I thought about sending the data from the Particle Cloud to the Google Pub/Sub, then writing a python script to read the data from the Google PubSub and re-publishing it to my local RabbitMQ server, but there has to be a better way…

Thanks

Where is your server? Can you make is accessible from the internet? If so, you could setup a webhook to push data to your server. If not, you can write a script which will listen for Particle events and insert them into your server.

2 Likes

The server is available on the public internet… I was interested in using a webhook, but I could not find any information on the forums for setting that up.

I found a Webhook plugin for RabbitMQ, but it only supports outbound webhooks. I couldnt find anything that would bring data into the RabbitMQ server.

Thoughts?

Can you find any plugins for receiving HTTP or HTTPS requests into your server? If so I can help you setup a Particle webhook

Yes, I did find a way to send HTTP requests to the RabbitMQ server, but they need to be formatted in a specific way. Is it possible to format the Particle.io webhooks as follows?

{“properties”:{},“routing_key”:“my key”,“payload”:“my body”,“payload_encoding”:“string”}

Here is the link: http://hg.rabbitmq.com/rabbitmq-management/raw-file/rabbitmq_v3_1_1/priv/www/api/index.html

The problem that I am seeing is that my “payload” from the Electron must be in the middle of the message (or be encapsulated by quotation marks).

Is this possible?

Yes, I imagine it’s very possible. Can you create a curl or postman request that achieves what you want and post it here? Once I have that I can build you a webhook configuration that achieves the same result.

Ok, I have the following command…

curl -i -u guest:guest -H "content-type:application/json" -POST -d'{"properties":{},"routing_key":"","payload":"PARTICLE PAYLOAD GOES HERE","payload_encoding":"string"}' URL

Mod Edit (@harrisonhjones): Removed your server URL for privacy reason, I don’t need it to help with the webhook

What would the actual payload look like (the PARTICLE PAYLOAD GOES HERE part)? Can I get dummy values?

When my Particle uploads data, it is uploading a set of sensor data. That payload looks like the following:

Sensor Base 13,1,2016-12-16T04:19:37.3776272Z,22:19:41,38.885101,0.000000,-12.271728,4.136775,12.032262,-11.670198,2.966006,0.000000,659.422241,16.759348,-3827.473633,0.000000,0.000000,3.893688,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.004073,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,673.229980,0.000000,1328.050049,0.000000,292.260010,0.000000,1620.310059,10.000000,0.000000,2468.239990,0.000000,0.130000,7.829093,0.000000,0.000000,0.000000,0.129811,4.454684,0.000000,0.023872,4.454684,0.023872,1.000000,0.000000,91.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,20.000000,0.000000,0.000000,0.000000,0.129811,8.348372,0.023872,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000001,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,2468.239990,-15.000000,127.198952,-25.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000,0.000000

Looks like that’s going to hit the 256 char publish limit. Can we break that up into multiple requests or ideally only send data that has changed?

Ok, lets use the following format:

0.000000,-12.271728,4.136775,12.032262,-11.670198,2.966006,0.000000,659.422241,16.759348,-3827.473633,0.000000

I will add the time stamp and additional information once I receive the data.

Just a question, what happens if I try to publish more than 256 characters? Will it automatically truncate or just discard the message?

I believe it automatically truncates the message unfortunately.

Hope you had a great new year! I just wanted to touch base and see if you had any progress or thoughts regarding the webhook?

Thanks again for your help!

Ok, looks like I am making some progress, but I am a little stumped…

I have created my webhook, and specified JSON under Send Custom Data. I then fill in the following:

{"properties":{},"routing_key":"","payload":"my body2","payload_encoding":"string"}

After I save my webhook, and it gets triggered, I noticed a failure reported by RabbitMQ. It says that I must include “properties” in my json string. If I scroll up in the webhook, under the JSON section, I see that my “properties” key is not listed. This is what it shows:

{
  "routing_key": "",
  "payload": "my body2",
  "payload_encoding": "string"
}

Playing around with Curl, I have discovered that I must have the properties key listed, or I can not sent data to the server. I also tried removing the {} after the properties key, and replacing with “” but that causes an error.

Thoughts?

Can you post your complete webhook definition/configuration JSON? I’m sorry for not responding till now; been swamped with family & holiday stuff.

No problem, I completely understand about the holidays…

I am not sure if I understand your request 100%… But, the JSON I am trying to pass, is the following:

{"properties":{},"routing_key":"","payload":"my body2","payload_encoding":"string"}

But, only the following is showing up after I create the webhook is:

{
  "routing_key": "",
  "payload": "my body2",
  "payload_encoding": "string"
}

For some reason, it keeps dropping my “properties” line… Ultimately, I will replace “my body2” with the actual payload from the Particle Electron…

Ok, I was able to resolve the issue of my “properties” key not being accepted. To fix this, I used the following JSON:

{"properties":[],"routing_key":"","payload":"my body2","payload_encoding":"string"}

I changes the {} to [], and it works. Also, in the headers I had to use the two following lines:

content-type:application/json
and
accept:text/html

Now, I am trying to replace my payload with the actual data coming from the Electron. Based on my understanding, it should look like:

{"properties":[],"routing_key":"","payload":"{{PARTICLE_EVENT_VALUE}}","payload_encoding":"string"}

But, this does not work. It literally sends the line {{PARTICLE_EVENT_VALUE}} to RabbitMQ.

Thoughts on how to properly include the payload?

What I was asking for was either a screenshot of your webhook when you created it (in the console) or a copy of the webhook.json file you used to create the webhook using the CLI

Ahhh, Ok… I created the webhook using the web interface. Attached are the screen shots of where I am at…

Harrisonhjones, any chance you could look at why I can not pass my payload onto the webhook?

1 Like