Electron in the Field Disconnecting Every 10-15 min or so

[GasGen]

Howdy.
I have my first unit in the real world and I am getting some odd disconnect-reconnect. It seems the unit is going offline at random intervals. 10 min here, 15, min, 7 mins. The device vitals show 97% battery and good cell signal. I am about 200 miles away from the unit so I cant visually inspect it but the client did send an install pic. See below. It is installed in an industrial park in Los Angeles.

Could some type of cell interference cause this? I am worried that I am going to burn through data with all these reconnects. That and I am also worried that I am going to miss some readings. I do have a Micro SD Card in the unit that is hopefully logging the data for backup. So that is less of a concern.

What are your thoughts?

Cheers,
Tom

Device OS: 0.8.0-rc.4

[GasGen]

It looks like it is going offline after posting 5 times, then waits 10 to 12 mins and comes back online. Not sure what is going on. So odd.

[ScruffR]

One shot in the blue, the 5-time interval may hint some trouble with running out of free sockets on the radio module.
Are you using TCPClient or UDP in your code?

For other possible reasons, it would be good to see some code.

[GasGen]

Howdy @ScruffR. Here is the code that I am using. I tried to remove all the notes. Let me know what you think. It ran fine here at the office for a month with no problems. I am sure there is something in my rookie coding that is cousing havok.
I appreciate you taking the time to look at it.
Cheers,
Tom

#include <Particle.h>
#include <SdFat.h>
#include <application.h>
#include "SdCardLogHandlerRK.h"
#include <Adafruit_SSD1306.h>
extern Adafruit_SSD1306 display;
#include <AssetTracker.h>
#include "ads1115.h"

ADS1115 inputBoard;
SYSTEM_THREAD(ENABLED);
        
        const int SD_CHIP_SELECT = A1;
        SdFat sd;
        

        SdCardLogHandler sdLogHandler(sd, SD_CHIP_SELECT, SPI_FULL_SPEED, LOG_LEVEL_ERROR, {
        	{ "app.sd", LOG_LEVEL_INFO }
        });

        STARTUP(sdLogHandler.withNoSerialLogging());
        
        Logger LogToSD("app.sd");
        
        SerialLogHandler serialLogHandler;

size_t counter = 0;
long lastpub = 0;
long lastlog = 0;
long lastdisplay =0;
int transmittingData = 1;
long lastPublish = 0;
int delayMinutes = 720; //changed from 360 for every 6 hours to 720 for every 12 hours. 
AssetTracker t = AssetTracker();
FuelGauge fuel;

#define OLED_DC     D3
            #define OLED_CS     D4
            #define OLED_RESET  D5
            Adafruit_SSD1306 display(OLED_DC, OLED_RESET, OLED_CS);
            
            
            #define NUMFLAKES 10
            #define XPOS 0
            #define YPOS 1
            #define DELTAY 2                        
            #define LOGO16_GLCD_HEIGHT 16 
            #define LOGO16_GLCD_WIDTH  16                       
            #if (SSD1306_LCDHEIGHT != 64)
            #error("Height incorrect, please fix Adafruit_SSD1306.h!");
            #endif

float ma = 0.0;
float flow = 0.0; 
float adc =0.0;

char publishStateString[256];

void setup() {
    inputBoard.setAddress(0);
    
//OLED SETUP STUFF

              display.begin(SSD1306_SWITCHCAPVCC);
              display.clearDisplay();
              delay(500);
//ASSET TRACKER SETUP STUFF
    t.begin();
    t.gpsOn();
    Serial.begin(9600);
    Particle.function("tmode", transmitMode);
    Particle.function("batt", batteryStatus);
    Particle.function("gps", gpsPublish);

}

void loop() {


int16_t input_1 = inputBoard.readInput(1);
ma=(input_1); 
adc=(input_1); 
float flow = map(adc,2139.0, 10614.0, 0.0, 200.0);
String F = String::format("%.1f", flow);
if (Particle.connected() && millis()-lastpub > 60000) {
		lastpub = millis();

Particle.publish("EF", 
                "{\"cf\":\"" + F + 
               "\"}", 60, PRIVATE, NO_ACK);  
}

//TIMER FOR LOGGER      
    if(millis()-lastlog > 60000){
    lastlog = millis();

Serial.print(Time.timeStr());
Serial.print(",");
Serial.printf("Flow: %.1f", flow);
Serial.printf("%.1f", flow); 

LogToSD.info(",%s, %.1f", Time.timeStr().c_str(), flow); 
}
if(millis()-lastdisplay > 2500){ 
lastdisplay = millis();
               display.setTextSize(1);
                  display.setTextColor(WHITE);
                  display.setCursor(0,0);
                  Time.zone(-7);
                  display.println(Time.format(Time.now(), "%a %b,%e %l:%M%p"));
                  display.println("------------------");
                  display.print("FLOW RATE: ");
                  display.printf("%.1f", flow);
                  display.println (" CFM");                
                  display.print("BATT VOLTAGE: ");
                  display.print( fuel.getVCell() );
                  display.println("V");
                  display.print("SOC: ");
                  display.print( fuel.getSoC() );
                  display.println("%");
                  CellularSignal sig = Cellular.RSSI();
                  display.print("RSSI: ");
                  display.print(sig.rssi);
                  display.println(" -dB");
                  display.print("ADC: ");
                  display.println(input_1);
                  display.display();
                  display.clearDisplay();
}
//ASSET TRACKER LOOP STUFF
    t.updateGPS();
    if (millis()-lastPublish > delayMinutes*60*1000) { 
     lastPublish = millis();
        Serial.println(t.preNMEA());
        if (t.gpsFix()) {
            if (transmittingData) {
                Particle.publish("G", t.readLatLon(), 60, PRIVATE, NO_ACK); 
            }
            Serial.println(t.readLatLon());
        }
    }
}
int transmitMode(String command) {
    transmittingData = atoi(command);
    return 1;
}
int gpsPublish(String command) {
    if (t.gpsFix()) {
        Particle.publish("G", t.readLatLon(), 60, PRIVATE)
        return 1;
    } else {
      return 0;
    }
}
int batteryStatus(String command){
    Particle.publish("B",
          "v:" + String::format("%.2f",fuel.getVCell()) +
          ",c:" + String::format("%.2f",fuel.getSoC()),
          60, PRIVATE
    );
    if (fuel.getSoC()>10){ return 1;}
    else { return 0;}
}

[GasGen]

@ScruffR

Are you using TCPClient or UDP in your code?

I forgot to answer no. I am not using either.

[ScruffR]

One advice I keep giving is: Avoid using String wherever possible and rather use char arrays and for formatted strings use snprintf() - String tends to cause heap fragmentation that will eventually lead to unexpected behaviour of your code.

Some other hints that won’t pose a problem but may improve the code

• Only use Particle.h and not application.h since the latter was superceded by the former
• I’d suggest AssetTrackerRK instead of AssetTracker library
• Instead of AssetTracker t = AssetTracker(); you can use AssetTracker t; which avoids creating a temporary object.
• All the variables that you use in connection with millis() should be unsigned long instead of long
• It might be a copy/paste issue, but proper code indentation helps to follow the flow better.
• Time.format(Time.now(), "%a %b,%e %l:%M%p") can be written as Time.format("%a %b,%e %l:%M%p") since Time.format() “assumes” current time when no timestamp is provided.
• transmitMode() could return transmittingData; to provide some useful feedback about the actual effect of the call.

[Rftop]

Did you change anything in the code between testing at your office and shipping to LA?

As @ScruffR mentioned, this String on every pass is a bad idea:

String F = String::format("%.1f", flow);

But any problems should have shown up in your month of local testing.

Do you know of any significant RF sources near the Electron, since it's in an Industrial Environment ?
The cyclic nature with 10 minutes before reconnect could be caused nearby equipment as you mentioned in the first post.

[Edit]: Can your customer Operate the Electron via Li-Po only for an hour or so after a disconnect event? That may help identify Power Supply Problems: crazy things like harmonics,etc, from the Industrial Environment.

[GasGen]

Thanks Guys. I appreciate the tips and help. Ok @ScruffR, I made all the changes you suggested except for the transmitMode( ). Did not understand that one.

I have some old code that you helped me with that I have copied below. but I am stuck on where to put the webhook variable “cf”. So my webhook event name is called “EF” and my variable is “cf” with the data now being “flow” instead of “F”.

Old code.

Particle.publish("EF", 
                "{\"cf\":\"" + F + 
               "\"}", 60, PRIVATE, NO_ACK);  

New code from one of your previous tips to me.

char publishStateString[256];

void loop() {

float flow = map(adc,2139.0, 10614.0, 0.0, 200.0);

if (Particle.connected() && millis()-lastpub > 60000) {
		lastpub = millis();

snprintf(publishStateString, sizeof(publishStateString), "%.2f:",flow );
Particle.publish("EF", publishStateString, 60, PRIVATE, NO_ACK);
}

Where do I put the variable in the publish to trigger the webhook? or do I need to change my webhook? Note the data goes to a google sheet. Here is how my webhook is formatted.

{
  "event": "{{{PARTICLE_EVENT_NAME}}}",
  "data": "{{{PARTICLE_EVENT_VALUE}}}",
  "coreid": "{{{PARTICLE_DEVICE_ID}}}",
  "published_at": "{{{PARTICLE_PUBLISHED_AT}}}",
  "Flow": "{{{cf}}}",
}
        

Hi @Rftop, I don’t think I changed anything so that’s what’s really strange. It is possible but not likely as I have been using this same code for some time.

It’s possible that there is local interference as it is an electronics manufacturing facility. There could be all kinds of crazy stuff going on there. If I can get this new code to publish on my other electron here at the office then I will do an over the air flash. I am already eating through data with all the disconnects and reconnects. I am at 0.3 mb in just 24 hours.

Thanks for taking a look at this and offering suggestions. You and @ScruffR have really helped me out.

[joost]

The unit is battery powered only? No recharging circuit of any kind? Btw, the reset cycle (5 messages + restart) is exactly 920 seconds, thats awfully unrandom - perhaps you can link this to an internal/external event? (e.g., the security guy smacking the device on his round through the building

[ScruffR]

@GasGen

except for the transmitMode( ). Did not understand that one.

Instead of always returning 1 it's usually more useful if your return value reports what actually happened inside the function, so I'd write the function like this

int transmitMode(String command) {
    return (transmittingData = atoi(command));
}

This way you'll get back the actual value transmittingData was set to.

@GasGen

I am stuck on where to put the webhook variable “cf”. So my webhook event name is called “EF” and my variable is “cf” with the data now being “flow” instead of “F”.

If I understand your intent correctly it should work like this

//                                intended result string  { "cf":"123.45" }
snprintf(publishStateString, sizeof(publishStateString), "{ \"cf\":\"%.2f\" }", flow);

@GasGen

Where do I put the variable in the publish to trigger the webhook?

By default the data payload you send via the publish will be held in {{{PARTICLE_EVENT_VALUE}}}

[GasGen]

lol. Anything is possible. The unit has a 24volt power supply that is brought down to 12 volts and fed into the electron. This keeps the battery charged. It only a 1amp power supply so I need the battery for the extra boost needed.

The unit is will now run for 5 posts go off for ten min 5 more then off again. Sometimes it will do 12 posts then go off for 30 mins come on and do 5 more then off again. Other times it will do 1 post then go off again.

[RWB]

Do you have this device setup under a product account where you can flash the latest device OS and see if that has any effect on performance?

Also, are you logging battery voltage over time? Is that looking steady over time?

[GasGen]

Hey Ryan,
I was not logging the voltage but I could, and probably should have… ugh.

I can do an over the air flash with the changes @ScruffR had me make. Are you suggesting that I go to 0.8.0-rc.10? Right now I am at 0.8.0-rc.4

[GasGen]

@ScruffR

snprintf(publishStateString, sizeof(publishStateString), "{ "cf":"%.2f" }", flow);

GasGen:

Yup that should have been obvious. Thank you and sorry for the dumb question.

[RWB]

Yes, I’m thinking that Maybe the new OS could give you different results.

[GasGen]

ok I will flash it now to my other electron here, run it for an hour or so then do an over the air flash to the remote one. If I just select the new OS and hit flash will it automatically install the new OS with the new code? sorry I have only done OS updates via the CLI so I am not sure how that works.

[RWB]

I think you can only flash the new OS if you have the Electron under a Particle product account.

[ScruffR]

You can OTA update the Device OS OTA via CLI by sending the binaries to the device.

And I’ve also just checked, even Safe Mode Healer is now updating Electrons when you flash a user firmware that requires a Device OS update.

IIRC, that was introduced when Particle increased the default data allowance with the new pricing scheme a few months ago.

[GasGen]

I tried flashing the unit here with the new code and the version 10 of the OS. It is stuck in safe mode. I need to figure out a way to do this over the air so the unit in the field does not get stuck. Note that I had already flashed the device here with the new code and 0.8.0-rc4 and no problem. When I went up to rc10 and the same code it got stuck in safe mode.

[ScruffR]

Did you let the device go through the 4/5 restart cycles required to have the application, 3-part Device OS and bootloader without interference?

However, if you have access to the device, just download these binaries and flash them via USB

in DFU Mode via particle flash --usb <fileName>
DevOS part1
DevOS part2
DevOS part3
Tinker

in Listening Mode via particle flash --serial bootloader-0.8.0-rc.10-electron.bin
bootloader