Web hooks timeout problem w. Heroku

Working with web hooks, I’ve recently come across a problem causing Heroku to timeout only when publishing (POST) directly through a build on the Spark Core.

The web hook includes a dynamic value (the Spark Core ID), which is sent to a Heroku server, which then returns server time-out (over 30 sec.).

While if I do a command line particle publish my-webhook {\"object\":\"value\"} everything is working and the event is received in a split second and using a service like hurl.it returns same success in a matter of seconds, thus indicating that it isn’t due to any problems server-side.

Thus I’m wondering whether the fact that the web hook is fired on the device vs. through the terminal is the reason for this, and whether there are any obvious reasons for the core timing out when POST’ing to Heroku.

The code executing the web hook on the Spark Core looks like this:

 void detect(){
  detect=true;
  RGB.color(255,255,255); //display red color
  digitalWrite(led, HIGH);
  
  if(Spark.connected()) {
    lastmotiontime = millis();
    
    String myIDStr=Spark.deviceID(); // Get Core's UDID to use for webhooks
    String one = "{ \"my_value\": \""+myIDStr;
    String two = "\"}";
    String my_value = one+two;
    
    if((lastmotiontime-previousMotion) > 30000) {
        previousMotion = millis();
        RGB.color(148,0,211); //display gray color

        Spark.publish("my_heroku_webhook", my_value);
        delay(5000);
    } else {
        RGB.color(0,0,255); // display blue color if latest activity is within 30 sec.
    }
  } else {
      Spark.connect(); // if device for some reason is not connected, connect
  }
}

This sound be the reason —> delay(5000);

Thanks for the quick reply.
Removed delay(5000), and still returns timeout from the Spark Core (looking at the server connection logs).

How do you consider a time-out? Is the core breathing cyan and publishing periodically?

i realized this is a function void detect(). Where are you running this and what if your loop() doing?

Timeout is based on Heroku’s logs and error codes. Digging deeper into the error code H12 these are thrown when the request takes longer than 30 secs (removed uuid & host).

at=error code=H12 desc="Request timeout" method=POST path="/api/detect/activity?my_uuid=xxxxxxxxxxxxxxxxxxxx" host=my-host.herokuapp.com request_id=d343e932-41b1-4026-984f-867a7b46f3b5 fwd="52.5.21.62" dyno=web.1 connect=0ms service=30038ms status=503 bytes=0

loop() code is here:

void loop(){
  digitalWrite(led, LOW);
  unsigned long currentMillis = millis(); //catch when millis function overflows
 
  if (currentMillis < lastmotiontime){ 
    lastmotiontime = 0; 
    motion = false;
  }
  
  //resetting motion variable if nothing has been detected for a while
  //Reset WiFi after 5 minutes of no motion detected to ensure Core reliability over longer periods of time
  if (currentMillis - lastmotiontime > interval){
    detect = false;
    
    Spark.disconnect();
    RGB.color(255,0,0); //display red color
    delay(5000);
    RGB.color(255,255,255); //display white color
    Spark.connect();
    delay(5000); //set delay to allow for connecting
  }
  
   //Call void detect(), which has been attached to CHANGE
    interrupts();
    RGB.color(0,255,255); //display blue color
    
    currentMotion = detect;  //store the current state in a non volatile variable - for access from spark.variable 

    // Otherwise re-loop
    delay(5000);
    Serial.println(F("Reached end of loop. Starting over."));
}

Commented on bits that might need some explaining.

There’s a series of long delay() that is blocking the processing of :cloud: connection so you might want to rewrite them as well.

Two of them are only called if there’s a timeout (no motion detected for 5 minutes). I limited the delay(5000) at end of loop to delay(1000) but still getting time-out error. Would delays in general affect web hooks? As far as I understand web hooks.

Also considering whether or not it might due to Particle’s limit of sending no more than 120 web hook requests to same host per minute: http://docs.particle.io/core/webhooks/#limits-limits-by-host

Are you even close to the limit?

The delay() will not affect the webhook but causes a time-out with the :cloud: and any :cloud: related functions will cease to function.

I would test this:

unsigned long old_time = millis();

void setup(){
  pinMode(D7,OUTPUT);
}

void loop{}{
  if(millis() - old_time >= 5000){
    digitalWrite(D7, !digitalRead(D7));
    Spark.publish("my_heroku_webhook", "test");
    old_time = millis();
  }
}

I’m not close to the limit, but the documents state that it is a Particle-wide host limit, so all users and their web hooks are accounted for on a first-come first-serve basis. And since Heroku is a popular service I would not be surprised if more than a 120 web hooks were already registered.

Just tested with the provided code (thanks a bunch for your help, btw) and it seems that POST to Heroku from the Spark Core does not work and returns a time out (503, H12), while a GET request works returning 200 OK. POST does work however when using services like hurl.it.

Then that would be a question of what .json settings did you use to setup that webhooks. You wlil need to setup up a POST webhook if you want to perform a POST.

If you post the webhooks .json or tell us how create the webhook, we will be able to probe further.

This is the .json file used to create the web hook with my_uuid being the dynamic value passed to the server. I created several web hooks to test (one w. requestType GET another w. requestType POST).

{
      "eventName": "my_heroku_webhook",
      "url": "https://my-server.herokuapp.com/api/detect/activity?my_uuid={{my_uuid}}",
      "coreID": "",
      "access_token": "xxxxxxxxxxxxxxxxxxxxxxxxx",
    "requestType": "POST",
    "headers": {
        "Content-Type": "application/json",
        "Accept": "application/json"
  },
  "query": null,
  "json": null,
  "auth": null,
  "mydevices": true
}

I just tested your provided code inserting the Spark.deviceID() dynamically vs. hardcoded as a JSON value, and this causes the server to return a time out for all requestTypes. This is the modified code:

void loop(){
    String myIDStr=Spark.deviceID(); // Get Core's UDID to use for webhooks and identification of user
    String one = "{ \"my_uuid\": \""+myIDStr;
    String two = "\"}";
    String uuid = one+two;
        
  if(millis() - old_time >= 5000){
    Spark.publish("my_heroku_webhook", uuid);
    old_time = millis();
  }
}

I'm not sure if this works as a null json field might not be creating a proper header.

Can you try to use a http://requestb.in/ and look at the request sent by the Particle :cloud:?

Also for better coding:

String uuid = "";

void setup(){
    String myIDStr=Spark.deviceID(); // Get Core's UDID to use for webhooks and identification of user
    String one = "{ \"my_uuid\": \""+myIDStr;
    String two = "\"}";
    String = one+two;
}
void loop(){
  if(millis() - old_time >= 5000){
    Spark.publish("my_heroku_webhook", uuid);
    old_time = millis();
  }
}

Tried using

"headers": {
"content-type": "application/x-www-form-urlencoded"
}

returning same error.

Adding Spark.deviceID() code to setup() returns null for data sent vs. having it inside the loop() (double checked in Particle dashboard). Just added a delay to setup, which ensures that the Spark.deviceID() is passed to the String, and then available for loop().

RequestBin doesn’t reveal much (http://requestb.in/qj6ccjqj?inspect)

So it looks like POST is working using http://requestb.in/qj6ccjqj?inspect

Have you tested your Heroku server with a POST from other means like curl?

Tested POST using [Hurl.it][1] which works like a charm and returns success.

So far I can boil the problem down to:

1) When hardcoding the my_uuid value into the web hook, POST web hook returns success.
I.e. hardcoded like this: Spark.publish("my_heroku_webhook", {\"my_uuid\":\"xxxxxxxxxxxxxxxxxx\"});

2) As soon as my_uuid is added dynamically to the web hook using Spark.deviceID(); POST returns time out server-side (Heroku 503 error, H12).