Functions and Variables Disappearing

I have 5 Xenons with Ethernet, not using mesh, running on a customer network. At times the Xenons will lose their functions and variables. They can be running for hours without losing access to the functions and variables. I can still ping the Xenon, see events, and get the vitals (via the console) but the console says there are no functions and variables. If I wait long enough (5 minutes to a couple of hours) they will reappear. The same behavior exists on both OS 1.4.4 and 1.5.0-rc.1.

If I do a GET request to https://api.particle.io/v1/devices/{deviceId} all of the info comes back correct except for “variables”:{},“functions”:{} Eventually they will return the correct values without power cycling or any other event.

What would be causing this to happen?

I have the same code running on dozens of other networks and 50+ Xenons and have yet to see this issue.

Make sure you declare then early in the setup, preferably first thing.

@Moors7 I am, the only thing before declaring them are 4 pinMode’s and 3 digitalWrite. Also, I see them when I power the Xenon on, but after some time usually 30-90 minutes they disappear and reappear as described above.

I would suggest running SerialLogHandler() to a PC where you can store the log output with time and also monitor the console events. It would be interesting to know what is happening when you lose the cloud connection. Clearly the recovery is not happening. In 1.4.4 this was a problem and the workaround was to reset if there was no cloud connection for a period (60 seconds say). With 1.5.0-rc.1 I haven’t noticed this problem occur although the devices go offline more often than expected for an ethernet connection.

The issue pops up when the describe message from the device does not make it through to our cloud.

Our engineering team is fully aligned behind the issue and the next two engineering sprints are devoted to solving it from the ground up.

The expected release date for the device side fixes is with DeviceOS 1.6 that is tentatively scheduled for the end of March. Cloud side fixes will be going out as they are tested and validated.

A few incidences can be fixed by running your code in SYSTEM_MODE(SEMI_AUTOMATIC); and calling Particle.connect() at the end of Setup(), after all the Particle Variables/Functions have been declared.

There is a temporary fix that should work in some cases:
Redeclaring a dummy with a different type variable (double instead of int for example) Particle.variable and then toggling Particle.connect() will cause the device service to request a new describe message. This has been tested on DeviceOS 1.4.4 and results are promising.

Thanks for the two fix instances. The first

helped my firmware on a Xenon in a mesh network with Argon gateway. The Xenon setup() originally used SYSTEM_MODE(AUTOMATIC) while not using SYSTEM_THREAD(ENABLED).

Was running ok with DeviceOS 1.5.0-rc.1 but after installing 1.5.0-rc.2 had issues with Particle.function(s) and Particle.variable(s) ghosting and plenty of disconnects from cloud.

Worked fine with no disconnects or disappearing cloud functions/variables once I changed setup() to use SYSTEM_MODE(SEMI_AUTOMATIC) and added the following statements to the end of setup():

Particle.connect();
waitFor(Particle.connected, 1000);

The above lines, in some fashion, were already in the loop() so reconnection could take place in the case of disconnection.

@no1089 I joined the Spectra keynote late and cannot find a recording, but as I was joining I thought I heard that there won’t be a DeviceOS 1.6, if that is true can you confirm what DeviceOS this fix will be in?

I’ll need to confirm, but a fix was made to DeviceOS 1.5.1-rc.1 that should work.
There has been a lot of work on the cloud-side to fix the missing functions and variables.

Yes, there won’t be a 1.6.

The recordings have been uploaded! spectra.particle.io/recordings

Awesome, thanks for the info.

@no1089 I deployed two Xenons with Ethernet on 2 different networks running DeviceOS 1.5.2 and both worked for some time (I can only confirm for a couple of hours for sure), now the functions and variables are missing.

Any guidance would be much appreciated.

I will PM you the 2 device IDs in case that helps.

I’ll take a detailed look tomorrow.
I have seen it once or twice, but it’s usually the result of the user code not allowing the device enough time to send the messages, or bad connectivity.
What mode are you in? Automatic, Semi-Automatic or Manual?
What does your setup() function look like?

Thank you.

Mode: Automatic

void setup() {
    pinMode(photoresistor,INPUT); 
    pinMode(power,OUTPUT);
    pinMode(relay1,OUTPUT);
    pinMode(relay2,OUTPUT);
    digitalWrite(power,HIGH);
    digitalWrite(relay1,LOW);
    digitalWrite(relay2,LOW);

    localIP = Ethernet.localIP().toString();

    Particle.variable("beamIntact", beamBroken);
    Particle.variable("isLockOrOpen", isLockOrOpen);
    Particle.function("cabinet",cabinetToggle);
    Particle.variable("localIP", localIP);
    Particle.keepAlive(20);
}

Try to move your Particle.functions to the beginning?

Amelien, are you running with SYSTEM_THREAD(ENABLED)?

@no1089 OK, I will try that.
@peekay123 Nope.

New setup and all code before the setup

int photoresistor = A0;
int power = A5;
int relay1 = D7; 
int relay2 = D6; 
bool beamBroken = false; 
bool isLockOrOpen = false;
int analogvalue = 0;
String localIP;

void setup() {
  Particle.variable("beamIntact", beamBroken);
  Particle.variable("isLockOrOpen", isLockOrOpen);
  Particle.function("cabinet",cabinetToggle);
  Particle.variable("localIP", localIP);
  pinMode(photoresistor,INPUT);
  pinMode(power,OUTPUT);
  pinMode(relay1,OUTPUT);
  pinMode(relay2,OUTPUT);
  digitalWrite(power,HIGH);
  digitalWrite(relay1,LOW);
  digitalWrite(relay2,LOW);

  localIP = Ethernet.localIP().toString();
  
  Particle.keepAlive(20);
}
1 Like

@amillen, this used to happen to me all the time in older OS. It used to be related to function name length.
I don’t think it’s true anymore, but I would try renaming cabinetToggle to less than 12 characters.