Webhook integration ETIMEDOUT with CGI script

Cloud
1

I am trying to use integrations to send data from the Particle Cloud to a python CGI script that I have running on a Digital Ocean server. The CGI script then populates a PostgreSQL database with the data from the Particle Boron. The overall system generally works as I have other devices that use the Iridium RockBlock modems and they are communicating with the server and sending data to the database.

Unfortunately, I can't get the Particle Boron to work. I get the following error message: ETIMEDOUT when events are published and sent to the webhook integration, which seems to be related to waiting 20 seconds and not getting a response from the server. The server is not that busy, so it must be something else.

The arduino code loaded on the Particle Boron includes the following event:

Particle.publish("waterTempC", String(waterTempC), PRIVATE);

The events do appear in the console in my browser, so the arduino code seems OK.

Then my webhook integration is set up as follows:

Name > TempToDB
Event Name > waterTempC
Full URL > http://myServerIPaddress/cgi_v4_boron2.py
Request Type > POST
Request Format > Web Form
Device > Temp1
Form >
{"event": "{{{PARTICLE_EVENT_NAME}}}",
"data": "{{{PARTICLE_EVENT_VALUE}}}",
"coreid": "{{{PARTICLE_DEVICE_ID}}}",
"published_at": "{{{PARTICLE_PUBLISHED_AT}}}"}
Headers
{"Content-Type": "application/x-www-form-urlencoded"}
Enforce SSL > No

My python CGI script looks like this (I do not include the database connection in this post as it is sort of irrelevant as I am not getting a response from the server at all):

import cgi
import cgitb
cgitb.enable()
form = cgi.FieldStorage()
data = form["data"].value
coreid = form["coreid "].value
published_at = form["published_at "].value
print("Content-Type: x-www-form-urlencoded\r\n\r\n")
print("OK")

And, I've added what I think is the IP address of the Particle Cloud to my firewall exemptions:

54.208.229.0/24

The CGI script DOES work when I force it to run with dummy data:

http://myserveripaddress/cgi_v4_boron2.py?data=21.187500&published_at=2023-09-11T19:04:48.115Z&coreid=examplexample

So, in conclusion, I think the problem is either:

1 - I have the wrong IP address / Subnet Mask for the Particle Cloud
54.208.229.0/24

2 - I am not properly sending the response in my CGI script
print("Content-Type: x-www-form-urlencoded\r\n\r\n")
print("OK")

3 - I am not understanding how the webhook is formated in the POST call
** http://myServerIPaddress/cgi_v4_boron2.py?data=21.187500&published_at=2023-09-11T19:04:48.115Z&coreid=examplexample**

Or maybe it is all three issues! Thanks in advance for any insights!

2

Hi and welcome to the community!

is your CGI script sending back a properly formatted 200 OK response?
Example:

HTTP/1.1 200 OK

Source:

3

Thanks for the suggestion @gusgonnet !

I've tried a variety of combinations, largely because I'm not very good at HTTP protocols.. sort of shooting darts in the dark. I just need this one line to behave, and the rest should be straight forward.

I've been attempting combinations of one of:

print("Content-Type: x-www-form-urlencoded\r\n\r\n")
print("Content-Type: application/x-www-form-urlencoded\r\n\r\n")
print("Content-Type: text/html\r\n\r\n")
print("Content-Type: text/json\r\n\r\n")

with one of

print("OK")
print("200 OK")
print("Status: 200 OK")
print("HTTP/1.1 200 OK")
But I don't quite understand how these protocols work.

With the rockblock modems that I mentionned, I use:

print("Content-Type: text/html\r\n\r\n")**
print("OK")

and it works great.

Again, my knowledge of these protocols is pretty basic, any ideas are welcome. I've been scouring the web for ideas to no avail :slight_smile:

4

It should be this one. Note the \r\n at the end of the line.

print("HTTP/1.1 200 OK\r\n")

This is OK, but make sure the two line terminators \r\n\r\n are only after the last header. Since you only have one it's fine, but if you add more headers terminate each line with \r\n and the last line has \r\n\r\n.

print("Content-Type: text/html\r\n\r\n")
1 Like
5

Thanks @rickkas7! So that worked! BUT I had to disable my firewall on the server using sudo ufw disable. This tells me that I have the wrong IP address or subnet mask whitelisted for the Particle Cloud.

Should I make a seperate issue to ask about whitelisting the IP address to allow the server to receive messages? Or can someone help here as it is part of my original question?

If interested, here is my modified python CGI script:

import cgi
import cgitb

cgitb.enable()
form = cgi.FieldStorage() 

print("Content-Type: text/html\r\n\r\n")
print("HTML/1.1 200 OK\r\n")
6

There's is no fixed IP address or CIDR block for the webhook requests so you cannot filter by IP address.

The easiest way to secure a webhook connection is:

  • Use TLS/SSL (https)
  • Have the webhook include an Authorization: Bearer header with a random token
  • Validate for the presence of this token/pre-shared key on your server. Reject any requests without valid authorization.

Also those two print statements should be in the opposite order.

7

Thanks again :slight_smile:

I will try to get one of these options working and report back (maybe with a tutorial):

As for:

That is what I understood from your last post, but for some reason it only works in the order I showed with Content Type first.. otherwise I get: error status 500 from XXX.XXX.XXX.XXX

8

Sorry, it should be HTTP not HTML:

print("HTTP/1.1 200 OK \r\n")
9

Ah! Typo, oops. Thanks!