HttpClient to get data from website - and SOS blinking

Parador · July 10, 2016, 11:23am

Dear all,

thanks for reading my post - I’m a bit in trouble with my code…

First of all I have a website that gives me data out of a database - and I’m trying to get this data into a variable.
The second step will be that I alter that variable and save it back to the database.

The collecting of the data worked (not completly fine, but it worked) till today. Without changing the code my photon startet to send out an sos - hard fault after powering it… and now sometimes it works, sometimes it gives me an sos…
Your hints are really welcome…

Another question I have and you may have a hint for me…as you can see in my code below I ask the website for the data and normally get it, but sometimes that doesn’t work, how can I control that an ask again, to get the data…

    void setup() {
      
      //http Client  
      request.hostname = "www.___mywebsite__.xyz";
      request.port = 80;
      request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
      request.path = String::format(request.path);
      http.get(request, response, headers);
      
            String web    = response.body;
            startup = atof( response.body );  
      Particle.publish("photon", "Response.body: " + String(response.body) + " startup: " + startup + " Response.status " + String(response.status) + " ID: " + String(deviceID) )
}

ScruffR · July 10, 2016, 11:35am

How long is your response string?
Why are you using String::format() althought you don’t format anything?
But where it would make sense you don’t use it

      Particle.publish("photon", "Response.body: " + String(response.body) + " startup: " + startup + " Response.status " + String(response.status) + " ID: " + String(deviceID) )

Before using response.body you should check if the request succeeded.

Parador · July 10, 2016, 11:50am

Dear ScruffR,
thanks for your fast reply…
a) The respose string is a number “12345.789” 9-11 digits
b) I used the emample I found at the community… so I will delete the line wit String::format… But where should I use it?

ScruffR · July 10, 2016, 11:57am

The code line I quoted is a good candidate for String::format()

Particle.publish("photon", String::format("Response.body: %s startup: %.3f Response.status: %d ID: %s", (const char*)response.body, startup, response.status, (const char*)deviceID), PRIVATE);

Parador · July 10, 2016, 5:40pm

Sorry didn’t see that…Thanks

void setup() {
  
  //http Client  
  request.hostname = "www.___mywebsite__.xyz";
  request.port = 80;
  request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
  http.get(request, response, headers);
   
   if(response.status == 200){
 startup = atof( response.body );
   }

  Particle.publish("photon", String::format("Response.body: %s startup: %.3f Response.status: %d ID: %s", (const char*)response.body, startup, response.status, (const char*)deviceID));
  }

I also added now a check if the status is ok to only make use of it, if there is something to use…
But how can I use that to ask the website again (and again, and again…) till I get something …?

ScruffR · July 10, 2016, 5:44pm

I'd say by use of something like a while(response.status != 200) loop that performs the GET request repeatedly - but use some delay between attempts!

Parador · July 10, 2016, 5:58pm

What about something like that:

 void setup() {
      
      //http Client  
      request.hostname = "www.___mywebsite__.xyz";
      request.port = 80;
      request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
      
      http.get(request, response, headers);
      while(response.status != 200) {
        http.get(request, response, headers);       
        startup = atof( response.body );
        Particle.publish("photon", String::format("Connecting failed - Response.status: %s Response.body: %s", response.status, (const char*)response.body));
        delay(5000);
       }

      Particle.publish("photon", String::format("Response.body: %s startup: %.3f Response.status: %d ID: %s", (const char*)response.body, startup, response.status, (const char*)deviceID));
      }

ScruffR · July 10, 2016, 6:17pm

That looks OKish, but you are again using the response.body without checking the status first. That if() you had is still required.
And since response.body is a String object, you might rather want startup = response.body.toFloat();

Parador · July 10, 2016, 6:53pm

Thanks alot for your help!

void setup() {
  
  //http Client  
  request.hostname = "www.___mywebsite__.xyz";
  request.port = 80;
  request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
  
  http.get(request, response, headers);
  if(response.status != ""){
    while(response.status != 200) {
      http.get(request, response, headers);       
      startup = response.body.toFloat();
      Particle.publish("photon", String::format("Connecting failed - Response.status: %s Response.body: %s", response.status, (const char*)response.body));
      delay(5000);
     }
  }
  Particle.publish("photon", String::format("Response.body: %s startup: %.3f Response.status: %d ID: %s", (const char*)response.body, startup, response.status, (const char*)deviceID));
  }

Parador · July 10, 2016, 6:55pm

Have seen another thing…:

void setup() {
  
  //http Client  
  request.hostname = "www.___mywebsite__.xyz";
  request.port = 80;
  request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
  
  http.get(request, response, headers);
  if(response.status != ""){
    while(response.status != 200) {
      http.get(request, response, headers);   
      Particle.publish("photon", String::format("Connecting failed - Response.status: %s Response.body: %s", response.status, (const char*)response.body));
      delay(5000);
    }
  }
  if(response.status == 200){
     startup = response.body.toFloat();
  }
  Particle.publish("photon", String::format("Response.body: %s startup: %.3f Response.status: %d ID: %s", (const char*)response.body, startup, response.status, (const char*)deviceID));
  }

ScruffR · July 10, 2016, 7:29pm

You are getting there, but response.status is an int so you wouldn’t compare for == "" but since you already got a check in your while() that’ll do anyway


void setup() {
  uint32_t msTimeout;
  
  //http Client  
  request.hostname = "www.___mywebsite__.xyz";
  request.port = 80;
  request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 

  msTimeout = millis(); // for timeout check
  do { // enter here anyway and try till success but max 60sec
    http.get(request, response, headers);   
    Particle.publish("photon", String::format("Response.status: %s Response.body: %s", response.status, (const char*)response.body));
    delay(5000);
  } while(response.status != 200 && millis() - msTimeout < 60000);

  if(response.status == 200){
     startup = response.body.toFloat();
  }
}

Parador · July 10, 2016, 7:44pm

looks much better again: thank you…
enough for today…

Parador · July 10, 2016, 8:14pm

Only problem from my last test …(cant take away my fingers )
I get a sos blink and restart after the first response:
from dashboard:
{“data”:“Response.status: \u0005/ Response.body: 2379.170”

ScruffR · July 10, 2016, 8:46pm

I missed that you changed the %d for response.status in my original String::format() for an %s - that’s wrong.
int variables (like response.status need a format placeholder %d not %s - that’s for strings).

If you add some Serial.print() statements in your code, you can norrow down which line of code causes the SOS.

Parador · July 11, 2016, 4:58pm

Hi ScruffR,

thanks for the hint with the serial communication, but I’m using the WebUI / WebCli to change to code on the photon (at the moment) so serial won’t help… I’ll try to comment out if something doesn’t work…

After changing to %d the response.status is correct

So at the moment I have the next problem using “startup” - again I get a sos

    //Variablen
        int AnzahlPulseProKwh = 2000;
        unsigned long PulseTime=0;
        byte minPulseWidth = 25; 
        unsigned int PulseCount=0;   
        float kwhCount=0;                       
        float kwhCount_startup;                  
        float kwhCount_overall;  
    
    //Funktion blink 
    void blink() {
      if ( (millis() - PulseTime) > minPulseWidth) {
        PulseCount++;                                   
        kwhCount = PulseCount / AnzahlPulseProKwh ;     
        kwhCount_overall = kwhCount + kwhCount_startup; 
      }
    }
    
    void setup() {
      pinMode(D3, INPUT);                           
      attachInterrupt(D3, blink, RISING);           
      
      //http Client 
      uint32_t msTimeout;
    
      request.hostname = "www.___mywebsite__.xyz";
      request.port = 80;
      request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
      
      request.path = String::format(request.path);
      msTimeout = millis(); // for timeout check
    
      do { // enter here anyway and try till success but max 60sec
        http.get(request, response, headers);   
        Particle.publish("pmr-photon-1", String::format("Response.status: %d Response.body: %s", response.status, (const char*)response.body));
        delay(5000);
      } while(response.status != 200 && millis() - msTimeout < 60000);
    
      if(response.status == 200){
         kwhCount_startup = response.body.toFloat();
         kwhCount_overall = kwhCount + kwhCount_startup;  //This line seems to be the problem
      } 
      
    }

peekay123 · July 11, 2016, 6:52pm

@Parador, in blink() you use PulseTime but never set it to any value except 0 when you define it. As such, the if() will fire anytime after millis() exceeds the value of minPulseWidth or 25. You may want to revise.

Parador · July 11, 2016, 7:04pm

Hi peekay123,

that happens within the loop,… (variables are defined…)

    void loop() {
      Timestamp = Time.now();                            
      DifPulseCount = PulseCount - OldPulseCount;        
      PulsePower = DifPulseCount * 30 / ReportIntervall; 
      PulseTime = millis();                              
      OldPulseCount = PulseCount;
      
      if (DifPulseCount >= aktiveNutzung) {Particle.publish("pmr-photon-1", "NUTZUNG (Aktiv)entdeckt"); ActiveUsageMessage=1;}
      else {ActiveUsageMessage=0;}
      
      if ((DifPulseCount >= passiveNutzung) && (DifPulseCount <= (aktiveNutzung-1) )) {PassiveUsageCounter++;}
      else {PassiveUsageCounter=0; PassiveUsageMessage=0;}
      
      if (PassiveUsageCounter>4) {Particle.publish("pmr-photon-1", "NUTZUNG (passiv) entdeckt: "+PassiveUsageCounter ); PassiveUsageMessage=2;}
      
      UsageMessage = ActiveUsageMessage+PassiveUsageMessage;
        
      delay(15 * 60 * 1000);
    }

ScruffR · July 11, 2016, 8:07pm

You also need to declare all variables changed inside your ISR as volatile.

Also this doesn't qualify as an argument against Serial

I'm also updating my devices via Build IDE and still can connect to the device via USB.
If your device was inaccessible, that would be a reason tho'

You still haven't told us how many blinks you see after the SOS pattern - that is crucial info in order to undestand the reason.

What's the value of ReportIntervall?
I can't see the declaration or the initialization. If you want us to spot a problem, you need to provide all info at once.
"Lass Dir nicht alles aus der Nase ziehen!"

This won't work either

Particle.publish("pmr-photon-1", "NUTZUNG (passiv) entdeckt: "+PassiveUsageCounter )

String literals and (another "undeclared" variable) PassiveUsageCounter which supposedly is of numeric data type can't be added like this.
Try applying the lessons from above (Stichwort String::format).

Parador · July 13, 2016, 7:19pm

Hi ScruffR,

thanks for your reply and hints… I thought not showing the “unneeded parts” helps focus on the problems…
Here is the complete code…
I have one photon up and running without the "new"part within setup to get the startup value from the website. It does its job without problems for months now. After adding the setup-get-the-value-part I allways get and SOS-Hard fault error on this “test”-Photon.

I started to install the CLI on my system to get Serial working. But to problem is, that system is not accessible from outside my home… that is the reason why Build IDE is a really nice option…

I did find the problem in the marked line of code… if delete this line everything is working “fine”

And yes I know I have a serious problem with the different variable typs, there format and usage… but I’m working on it That is the reason I need an explaination of “declare all variables changed inside your ISR as volatile”

//HTTP Client
HttpClient http;
    
// Headers currently need to be set at init, useful for API keys etc.
http_header_t headers[] = {
//  { "Content-Type", "application/json" },
//  { "Accept" , "application/json" },
    { "Accept" , "*/*"},
    { NULL, NULL } // NOTE: Always terminate headers will NULL
};
        
http_request_t request;
http_response_t response;
    
byte TryNr = 128;                         
int ReportIntervall = 15;                 
byte minPulseWidth = 25;                  
int AnzahlPulseProKwh = 2000;             
int aktiveNutzung = 975;                  
int passiveNutzung = 800;                 

char resultstr[128];                      
unsigned long PulseTime=0;                
int PulseCount=0;       
int DifPulseCount=0;    
int OldPulseCount=0;    
unsigned long PulsePower=0;      
float kwhCount=0;                        
float kwhCount_startup=0;                  
float kwhCount_overall=0;                 
int Timestamp=0;                          
String deviceID=Particle.deviceID();;     
    
int day_now = 0;
int day_old = 0;
int hour_now = 0;
int hour_old =0; 
int PassiveUsageCounter = 0;
int ActiveUsageMessage;
int PassiveUsageMessage;
int UsageMessage;
        
    
//Funktion blink 
void blink() {
  if ( (millis() - PulseTime) > minPulseWidth) {
    PulseCount++;                                   
    kwhCount = PulseCount / AnzahlPulseProKwh ;     
    kwhCount_overall = kwhCount + kwhCount_startup; 
  }
}
    
//Funktion Setup
void setup() {
  pinMode(D3, INPUT);                           
  attachInterrupt(D3, blink, RISING);                      
      
  Particle.publish("pmr-photon-1", "gestarted - Version: "+String(TryNr)); 
  Particle.variable("result", resultstr, STRING); 
      
  //http Client  
  uint32_t msTimeout;
     
  request.hostname = "www.___mywebsite__.xyz";
  request.port = 80;
  request.path = "/Testumgebung/Photon/index2.php?deviceID=" + deviceID + "&requestID=startupkwhlookup"; 
     
  request.path = String::format(request.path);
  msTimeout = millis(); // for timeout check
    
  do { // enter here anyway and try till success but max 60sec
    http.get(request, response, headers);   
    Particle.publish("pmr-photon-1", String::format("Response.status: %d Response.body: %s", response.status, (const char*)response.body));
    delay(5000);
  } while(response.status != 200 && millis() - msTimeout < 60000);
    
  if(response.status == 200) {
    kwhCount_startup = response.body.toFloat();
    kwhCount_overall = kwhCount + kwhCount_startup;   // Here seems to be the problem... if deleted everything is ok
  }
}
    
//Loop
void loop() {
  Timestamp = Time.now();                            
  DifPulseCount = PulseCount - OldPulseCount;        
  PulsePower = DifPulseCount * 30 / ReportIntervall; 
  PulseTime = millis();                             
  OldPulseCount = PulseCount;                    
      
  day_now = Time.day();      //  if (day_now  != day_old )  { /*reset all*/ }
  hour_now = Time.hour();    //  if (hour_now != hour_old ) { /*reset all*/ }
      
  if (DifPulseCount >= aktiveNutzung) {
    Particle.publish("pmr-photon-1", "NUTZUNG (Aktiv)entdeckt"); 
    ActiveUsageMessage=1;
  }
  else {
    ActiveUsageMessage=0;
  }
      
  if ((DifPulseCount >= passiveNutzung) && (DifPulseCount <= (aktiveNutzung-1) )) { 
    PassiveUsageCounter++;
  }
  else {
    PassiveUsageCounter=0; 
    PassiveUsageMessage=0;
  }
      
  if (PassiveUsageCounter>4) {
    Particle.publish("pmr-photon-1", "NUTZUNG (passiv) entdeckt" );
    PassiveUsageMessage=2;
  }
      
  UsageMessage = ActiveUsageMessage+PassiveUsageMessage;
      
  //http Client
  request.hostname = "www.___mywebsite__.xyz";
  request.port = 80;
  request.path = "/Testumgebung/Photon/";
  request.path = String::format("/Testumgebung/Photon/index2.php?deviceID=%s&Timestamp=%d&PulseCount=%d&DifPulseCount=%d&kwhCount=%d&kwhCount_overall=%d&PulsePower=%d&UsageMessage=%s"
                                 , (const char*)deviceID
                                 , Timestamp 
                                 , PulseCount
                                 , DifPulseCount
                                 , kwhCount
                                 , kwhCount_overall
                                 , PulsePower
                                 , (const char*)UsageMessage
                                 );                             
                                 
  Particle.publish("pmr-photon-1", "Versuch: "+String(TryNr)+" - kwh: " + String(kwhCount) + " - kwh_overall: " + String(kwhCount_overall) + " - Difpulses: " + String(DifPulseCount));
  sprintf(resultstr, "{\"data1\":%d,\"data2\":%d,\"data3\":%d,\"data4\":%d,\"data5\":%d,\"data6\":%d,\"data7\":%d}", Timestamp, PulseCount, DifPulseCount, ReportIntervall, kwhCount, PulsePower, UsageMessage);
     
  delay(ReportIntervall * 60 * 1000); //Publish every (10Sec = 10000, 60sec = 60000)
      
  //if (day_now != day_old) { kwhCount_overall=kwhCount_overall+kwhCount; Particle.publish("power-meter-reporter", "Wert wird irgendwann gespeichert");}
  //day_old = day_now;
}

ScruffR · July 14, 2016, 7:18am

Let's start with the easy part

Since interrupts access variables out of the normal code flow, variables used in the ISR can change at any time "unpredictably". On the other hand the C++ optimizer does its job by analyzing the normal code flow and might decide to save some clock cycles by just holding a variable value in a register for later use if it doesn't see any inidcation that that value would change between now and its next use. And that is where volatile comes in. This modifyer informs the optimizer that this particular variable might change at any time, so it can't be optimized the normal way but has to be retrieved from RAM each time it's accessed.
So your variable declaration would need to look like this

/// since you are alterning these variables inside your ISR
    volatile int   PulseCount=0;       
    volatile float kwhCount=0;                        
    volatile float kwhCount_overall=0;

As for your error I might think you've got a problem here

  request.path = String::format("/Testumgebung/Photon/index2.php?deviceID=%s&Timestamp=%d&PulseCount=%d&DifPulseCount=%d&kwhCount=%d&kwhCount_overall=%d&PulsePower=%d&UsageMessage=%s"
                                 , (const char*)deviceID
                                 , Timestamp 
                                 , PulseCount
                                 , DifPulseCount
                                 , kwhCount
                                 , kwhCount_overall
                                 , PulsePower
                                 , (const char*)UsageMessage
                                 );                  
// and here
  sprintf(resultstr, "{\"data1\":%d,\"data2\":%d,\"data3\":%d,\"data4\":%d,\"data5\":%d,\"data6\":%d,\"data7\":%d}", Timestamp, PulseCount, DifPulseCount, ReportIntervall, kwhCount, PulsePower, UsageMessage);

You need to get the format placeholders (%...) right for your variables.
You are using %d for float where it should be %f (or better %.2ffor two decimal places) or you need to cast the variables to int (e.g. (int)kwhCount).

And you've got this line in again

 request.path = String::format(request.path);

BTW, I reformatted your code a bit (don't write sausage lines ) and found that you had an extra double quote in

 request.hostname = "www.___mywebsite__.xyz"";

I removed it to get the code hilighting corrected.

I'd also try to replace all usages of String with C strings (char[]) to avoid heap issues.

How does your code behave if you add a kwhCount_overall = 0; as first line in loop() while leaving the "suspicious" code line in?

Topic		Replies	Views
Photon sends sos after flashing my firmware - help needed Firmware	7	1362	November 29, 2015
HTTP POST (Get works fine) Troubleshooting	7	2929	February 7, 2018
HTTP client inestability problem Troubleshooting	2	906	October 13, 2015
Photon Problem - http.get(request, response, headers) Causing reset Troubleshooting	2	2192	October 12, 2015
Having trouble checking data from subscribe - myhandler() Troubleshooting	2	1536	June 14, 2016

HttpClient to get data from website - and SOS blinking

Related topics