Core becomes unresponsive after Http request [Solved]

[bijay]

I have a motion sensor connected to the A0 pin of the core.And a RISING interrupt is assigned to this pin. So upon detection of motion it calls an ISR and this ISR does a POST request to a PHP page on my server. This PHP file is sending a push notification to my phone via Google cloud messaging.

Everything works fine for the first time. I do receive the push notification on my phone but after that the core freezes indefinitely and any line below the http.post() doesnot get executed.

Below are the codes

#include "HttpClient/HttpClient.h"

int interval = 30000;  //30 seconds

volatile boolean motion = false; 
volatile unsigned long lastmotiontime = 0; 

HttpClient http;
#define HOSTNAME "mydonain.in"
#define PATHNAME "/push2.php"


http_header_t headers[] = {
    { "Accept" , "*/*"},
    { "Referer" , HOSTNAME},
    { "Content-Type", "application/x-www-form-urlencoded" },
    { NULL, NULL } // NOTE: Always terminate headers will NULL
};


http_request_t request; 
http_response_t response;

boolean MOTION = false; 
    
void setup(){
    Spark.variable("motion", &MOTION , BOOLEAN);
    delay(20000); 
    
    Serial.begin(9600); 
    while(!Serial.available())  // Wait here until the user presses ENTER 
        SPARK_WLAN_Loop();  
        
    Serial.print("Setup done\r\n");

    attachInterrupt(A0, motionInterupt, RISING);
    
}

void loop(){
  noInterrupts();
  unsigned long currentMillis = millis();

   //fallback if  millis() function overflows.  
  if (currentMillis < lastmotiontime){ 
    lastmotiontime = 0 ; 
    motion = false; 
    
  } 

if (motion & (currentMillis - lastmotiontime   > interval)){
    motion = false; 
    
  }
  interrupts();


  MOTION = motion; 

  delay(1000); 
}

void motionInterupt(){
    Serial.print("Motion detected.\r\n");
   motion=true;  
   lastmotiontime = millis(); 
   pushNotify("Alert","Motion detected..");
}


void pushNotify(const char* title,const char* msg){
    Serial.print("inside pushNotify methode..\r\n");
    char queryString[100]="title=";
    strcat(queryString,title);
    strcat(queryString,"&message=");
    strcat(queryString,msg);
    
    request.hostname = HOSTNAME;
    request.port = 80;
    request.path = PATHNAME;

    // The library also supports sending a body with your request:
    request.body = queryString;
    Serial.print("request created\r\n");
    
    // POST request
    http.post(request, response, headers);
    //Below this point nothing is executed and the core freezes
    Serial.print("POSTED\r\n");
    
    
}

The code gets executed till

http.post(request, response, headers);

and then the core freezes.It doesn’t even print “POSTED” to serial out.Below is a screenshot of the output

After that the core becomes unresponsive.

Any help as to why I am facing this issue??

[bijay]

If it helps below is the php code

<?php
    //Generic php function to send GCM push notification
   function sendMessageThroughGCM($registatoin_ids, $message) {
        //Google cloud messaging GCM-API url
        $url = 'https://android.googleapis.com/gcm/send';
        $fields = array(
            'registration_ids' => $registatoin_ids,
            'data' => $message,
        );
        // Update your Google Cloud Messaging API Key
        //define("GOOGLE_API_KEY", "API_SERVER_KEY");         
        $headers = array(
            'Authorization: key=' . 'My_google_API_Console_key',
            'Content-Type: application/json'
        );
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_POST, true);
        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);    
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
        curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($fields));
        $result = curl_exec($ch);                
        if ($result === FALSE) {
            die('Curl failed: ' . curl_error($ch));
        }
        curl_close($ch);
        return $result;
    }
?>
<?php

//Device registration ID for Google cloud messaging 
        $gcmRegID  = 'APA91bES3eDgDOLfhJVLmRvGtwnnYhVxq6__flv7EU9MxJup5dVHGs98MH_0YxM1XlKNXPO7-hpb2_mUQLuDkcXkYioljnMYrsGlEIV5ANQSu0pvpQ_6y4CyEVWWI6oPfrV2KYIQ8W0CaKdtQXELDbruDVmqiROdZg';
        
        
        $pushMessage = $_POST["message"];    
        if (isset($gcmRegID) && isset($pushMessage)) {        
            $gcmRegIds = array($gcmRegID);
            //print_r($gcmRegIds);
            $message = array("payload" => $pushMessage);    
            $pushStatus = sendMessageThroughGCM($gcmRegIds, $message);
            echo "done";

            
        }        

?>

[bijay]

Guys can please anybody help me. I am stuck at this point and really need some help.Plz

[Maddimax]

Try:

a) Do not do the http from the interrupt, but only set a variable, and when that changes call the HTTP request from inside your normal loop().
b) Make sure you read the response from the HTTP request, I don’t really know what that HTTP library you use does, maybe it reads it already.
c) If a and b don’t help, try disconnecting from the Spark Cloud, it helped in my project where I used TCPClient, but of course you cannot flash via the WEB UI anymore then. ( see http://docs.spark.io/firmware/#advanced-system-modes-automatic-mode )

[bijay]

Thanks for the reply @Maddimax. I was also afraid that the http call from the interrupt may the causing the problem. I am using HttpClient for making the request. I also tried to use basic tcp client code and the core becomes unresponsive while executing client.stop(). I guess there is some issue with stop() methode of TCPClient when called from interrupt.

Anyways thanks for your help.

[Hootie81]

Try using these functions instead of the HTTP client. this is cut and paste from my doorbell where i POST to a PHP script on my server. its pretty well tested (its getting used several times a day)
I use a bit of PHP as a security check to make sure the coreID, spark user ID and event name is correct (as these are also the standard things in a webhook)
it wouldn’t be hard at all to add loads of extra data too if you need it, just add it to the line commented //this is the POST data… make sure you also update the content length.

before setup()

char phpServer[] = "server address";
int phpPort = 80;
char phpScript[] = "/folder/script.php";
char sparkUID[] = "12345abcde";
char eventName[] = "Doorbell";

String myID = Spark.deviceID(); //store the device ID - its used in the PHP script 
char coreID[25];//as 1 char space for null is also required

in setup()

    myID.toCharArray(coreID, 25); //convert the string to a char array
    coreID[24] = '\0';

the call

postDoorbell(phpServer, phpPort, phpScript, sparkUID, coreID, eventName);

the functions

/*----------------------------------------------------------------------*/
/* out - outputs supplied string to TCPclient */
void out(const char *s) {

client.write( (const uint8_t*)s, strlen(s) );

if (DEBUG)Serial.write( (const uint8_t*)s, strlen(s) );

}
/*----------------------------------------------------------------------*/

/*----------------------------------------------------------------------*/
/* in - reads from TCPclient, with timeout */
void in(char *ptr, uint8_t timeout) {
        int pos = 0;
        unsigned long lastTime = millis();
        
        while( client.available()==0 && millis()-lastTime<timeout) { //timeout
        }  //do nothing
        lastTime = millis();
        unsigned long lastdata = millis();
        
        while ( client.available() || (millis()-lastdata < 500)) {  //500 millisecond timeout
            if (client.available()) {
                char c = client.read();
                if(DEBUG)Serial.print(c);
                lastdata = millis();
                ptr[pos] = c;
                pos++;
            }
            if (pos >= 512 - 1)
            break;
        }
        ptr[pos] = '\0'; //end the char array
        while (client.available()) client.read(); // makeshift client.flush()
        client.flush();  //for safety
        delay(400);
        client.stop();
        if(DEBUG){
            Serial.println();
            Serial.print("Done, Total bytes: ");
            Serial.println(pos);
        }
        
}
/*-----------------------------------------------------------------------*/

void postDoorbell(char *hostname, int port, char *script, char *uid, char *cid, char *event) {

    char line[255];
    client.stop();
    //digitalWrite(LED, HIGH); // sets the LED on
    
    if(DEBUG){Serial.print("connecting... ");}
    if (client.connect(hostname, port)) {
        if(DEBUG){
            Serial.print("connected to ");
            Serial.println(hostname);
        }
        delay(500);
        digitalWrite(LED, LOW);
        sprintf(line, "POST %s HTTP/1.1\r\n", script);
        out(line);
        out("Connection: close\r\n");
        sprintf(line, "Host: %s:%d\r\n", hostname, port);
        out(line);
        sprintf(line, "Content-Length: %d\r\n", strlen(uid) + strlen(cid) + strlen(event) + 15);
        out(line);
        out("Content-Type: application/x-www-form-urlencoded\r\n");
        out("\r\n"); //end of headers
        sprintf(line, "event=%s/%s&coreid=%s\r\n", uid, event, cid); //this is the POST data
        out(line);
        in(reply, 20000);
     }
    else {
        while (client.available()) client.read(); // makeshift client.flush()
        client.flush();
        client.stop();
        if(DEBUG)Serial.println("Could not connect");
    }
        
}

PHP security check

// Spark core stuff
$spark_id = '55ffxxxxxxxxxxxxxxxxxxxxxxx'; // coreID of the doorbell
$eventname = 'Doorbell';                // This is the name of the event triggered by the webhook

// Security check for correct core and Event Name
if (isset($_POST['event']) && isset($_POST['coreid'])) { 

    //split the event into spark user id and event name
    list($sparkuid, $event) = explode( '/', ($_POST['event']), 2); 
    if (!ctype_xdigit($sparkuid)) { 
        $file = 'connect.txt';
        file_put_contents($file, 'sparkuid not hex ' , FILE_APPEND);
        die(); } //check spark user id is hex characters
    if ($event != $eventname) { 
        $file = 'connect.txt';
        file_put_contents($file, 'event name mismatch ' , FILE_APPEND);
        die(); } //check spark user id is hex characters
    $coreid = $_POST['coreid'];
    if (!ctype_xdigit($coreid)) { 
        $file = 'connect.txt';
        file_put_contents($file, 'coreid not hex' , FILE_APPEND);
        die(); } //check spark user id is hex characters//check spark core id is hex characters
    if ($coreid != $spark_id) { $file = 'connect.txt';
        file_put_contents($file, 'coreid mismatch' , FILE_APPEND);
        die(); } //check spark user id is hex characters //must be the right core
    
} else { die(); } // Hint: comment out just before the else to disable the checks for testing 

[bijay]

Thanks @Hootie81 for your help. I will definitely try your code and post my feedback. As you suggested to drop the HTTP cliend I did that earlier and here is the code for the http request.

**Note:**POST didn’t work so switched to GET

void pushNotify(const char* title,const char* msg){
    Serial.print("inside pushNotify methode..\r\n");
    char queryString[100]="";
    strcat(queryString,"message=");
    strcat(queryString,msg);
    strcat(queryString,"&title=");
    strcat(queryString,title);
    
    Serial.println("connecting .test..");
    client.connect(HOSTNAME, 80);
    client.println("POST /push2.php HTTP/1.0");
    client.println("Host: " HOSTNAME);
    client.print("Content-Length: ");
    client.println(strlen(queryString));
    client.println();
    client.print(queryString);
    client.stop();
    

    Serial.print("POSTED\r\n");
    
    //Serial.println(response.body);

    //return response.status; // 200 if successful.
    
}

With this also I faced the same issue of core freezing after first http request. Then I removed client.stop() from the code and things worked fine. So I am guessing it might be an issue of TCPCLIENT’s stop() function when called from an interrupt.

Plz let me know your thoughts.

[Hootie81]

Yeah, just set the flag is best practice anyway. And you must must must read the reply from the client… even if you don’t need it otherwise it will hard fault

[Hootie81]

Oh and 400ms delay is required before client.stop otherwise it will die

[bijay]

I am sorry @Hootie81 I could not understand what you meant by 400ms delay, Could you please explain when and where that delay is required.
hanks
T

[Hootie81]

in your code you have:

client.print(queryString);
client.stop();

you will need to have:

client.print(queryString);
delay(500); //this will depend on the time it takes your server to reply
while (client.available()) client.read(); // makeshift client.flush() - expect a hard fault if you leave this line out
client.flush();  //wont do anything because of the line above but i leave it in anyways
delay(400);  //prevent a race condition when calling client.stop()
client.stop();

so there are 2 fixes in here - the 400ms delay between flush and stop, i remember reading that there may be some race condition when calling flush and stop, and this was the fix - although ive searched high and low and cant find the thread anymore.
the second and the one i find the worst - is the hard fault many people see (red SOS followed by 1 flash) - One of the elites may have to correct this as im not 100% on it…
if there is any data in the cc3k buffer when you stop the client, i think it will stay there with no way to clear it out. then next time you do a request there will be more and the next time it will be full… and then it will hard fault. the client.flush just clears the local buffer so calling client.flush wont do much, but the client.available() copies the data to a local buffer and the read clears it. i think its possible to do client.available(); followed by client.flush(); to get the same result but i haven’t tested that as the while loop works perfectly.

having said all that, the biggest difference you will notice is changing to client.write. the print method is very very slow, it sends each byte as a packet and its not super reliable. since changing to client.write i have had a massive improvement in connection speed and reliability.

[bko]

Hi @Hootie81 and @bijay

I don’t recommend the delays and flush methods any more. It is much more reliable to have a first while loop that waits for client.available() to be non-zero with a timeout, followed by a second while loop that reads the data until no more is available, like what you have above but with a (say) 5 second timeout.

The problem with client.flush is that it only flushes the TCPClient, not the packet buffers in the TI WiFi chip, so those packets have to timeout and bad things happen.

The problem with client.stop is similar plus you have a bit of a race condition.

[Hootie81]

I think that’s what i was trying to say thanks for making it clear and yes as you say the first delay(500) in the example above is not the best way to go about it.

The in() function I posted above (post 6) has these timeouts and reads the buffer into a char array. I pass the timeout time to it too as some of my PHP scripts take 15 seconds to complete before returning the data so i need a 20sec timeout and other scripts are near instant so i pass a 1 second timeout. the other timeout in the function is because sometimes there is little gaps where there is no data available but then a split second later there is. this is most noticeable when receiving large amounts of data.

[bijay]

Thanks @bko and @Hootie81 for your help

@Hootie81

you will need to have:

client.print(queryString);
delay(500); //this will depend on the time it takes your server to reply
while (client.available()) client.read(); // makeshift client.flush() - expect a hard fault if you leave this line out
client.flush(); //wont do anything because of the line above but i leave it in anyways
delay(400); //prevent a race condition when calling client.stop()
client.stop();

So if I understood correctly after making a http request I will have to wait for a while till there is some response then read everything from the TI WiFi chip buffer using and flushing it by

while (client.available()) client.read(); 
client.flush();

Then again wait for a while till the request is complete and then call client.stop().

So if my understanding is correct please let me know if my below code implementation is correct

void pushNotify(const char* title,const char* msg){
    Serial.print("inside pushNotify methode..\r\n");
    char line[255];
    char response[512];
    int pos=0;
    
    Serial.print("connected to ");
    Serial.println(HOSTNAME);
    
    client.connect(HOSTNAME,80);
    sprintf(line, "POST %s HTTP/1.1\r\n", PATHNAME);
    client.write( (const uint8_t*)line, strlen(line) );
    client.write( (const uint8_t*)"Connection: close\r\n", strlen("Connection: close\r\n") );
    
    sprintf(line, "Host: %s:%d\r\n", HOSTNAME, 80);
    client.write( (const uint8_t*)line, strlen(line) );
    
    sprintf(line, "Content-Length: %d\r\n", strlen(title) + strlen(msg)  + 12);
    client.write( (const uint8_t*)line, strlen(line) );
    
    sprintf(line, "Content-Type: application/x-www-form-urlencoded\r\n");
    client.write( (const uint8_t*)line, strlen(line) );
    
    sprintf(line, "title=%s&msg=%s\r\n", title, msg); //this is the POST data
    client.write( (const uint8_t*)line, strlen(line) );
    
    //Wait till there is any data and then read it
    unsigned long lastTime = millis();
        
    while( client.available()==0 && millis()-lastTime<20000) { //a 20 second timeout
        char c=client.read();
        response[pos] = c;
        pos++;
    }
    response[pos]='\0';
    
    Serial.println(response);
    //Again check if there is still anything left in the buffer
    while (client.available()) client.read(); 
    client.flush();
    
    delay(500);//wait till the request is completed to avoid race condition
    client.stop();
    
    }

Again how can I be sure as to how long I will have to wait before performing a read() and stop(). Because you can never be sure as how long your script is going to take.

Also this function is called from a interrupt routine and if I am correct I cant use delay() inside an ISR.

Please let me know your opinion on this.

[UPDATE] : Getting multiple red flash after calling the function and the core restarts.The response I get from the php page is "Motion detected @ Jan 16th 03:44:49 PM". This is just to let you know that the response does not use much memory.

Can anybody please please help me for a working implementation of the above function.

Thanks

[bijay]

@bko

The problem with client.flush is that it only flushes the TCPClient, not the packet buffers in the TI WiFi chip, so those packets have to timeout and bad things happen.

Hi @bko,
Shouldn't the data packets in the buffer should be auto flushed or over written with successive http requests or do we need to handle the buffer data management?

[bko]

@Hootie81

unsigned long lastTime = millis();
        while( client.available()==0 && millis()-lastTime < timeout) { //timeout
        }  //do nothing
        lastTime = millis();
        unsigned long lastdata = millis();
        while ( client.available() || (millis()-lastdata < 500)) {  //500 millisecond timeout
            if (client.available()) {
                char c = client.read();
                if(DEBUG)Serial.print(c);
                lastdata = millis();
                ptr[pos] = c;
                pos++;
            }
            if (pos <= 512 - 1)
            break;
        }

You code will work better if it is more like the @Hootie81 code above that I have quoted. This is what he and I have been saying--don't depend on flush since it only flushes some of the buffers. Yes the packet buffers in the TI WiFi chip do get reused but there is a 6 (I think) second timeout involved, so other traffic including your next try will suffer.

[Hootie81]

whats the red flash you get? is it SOS followed by a single flash? or SOS followed by 8 flashes?

[Hootie81]

hey @bko in your cut and paste above there is a little websafe thingy the < gets changed to &gt… i guess for database storage, it might make for some extra debugging.

[bko]

Thanks – my bad.

I believe I have fixed it now.

[mtnscott]

@bko

unsigned long lastTime = millis();
while( client.available()==0 && millis()-lastTime < timeout) { //timeout

Nothing wrong with this code, but I just thought I would suggest that doing the math once and then the test multiple times is probably a more efficient way to write the timeouts

unsigned long lastTime = millis() + timeout;
        while( client.available()==0 && millis() < lastTime) { //timeout