3rd Party SIM - Best Approach to Minimize Data Usage

We have been using 3rd party SIMs, reluctantly, in Rwanda and have noticed improved coverage and more reliable connections. However, we do miss the Particle SIMs 23 minute keep alive time. This is an amazing benefit which I had not thought of until it was gone.

I have written some code that allows us to update the keep alive time using a Particle function which let us try different values until we found one that worked. It turns out that MTN/Rwandacell will only allow for a keep alive of 120 seconds or so. Here is the code that I used to figure this out in case others want to try it:

// In header
Timer keepAliveTimer(1000, keepAliveMessage);

// In Setup()
Particle.variable("Keep Alive Sec",sysStatus.keepAlive);
Particle.variable("3rd Party Sim", sysStatus.thirdPartySim);

Particle.function("Keep Alive",setKeepAlive);
Particle.function("3rd Party Sim", setThirdPartySim);

if (sysStatus.thirdPartySim) {
  waitFor(Particle.connected,300 * 1000);                                               // keep alive does nothing if we are not connected
  Particle.keepAlive(sysStatus.keepAlive);                                              // Set the keep alive value
  keepAliveTimer.changePeriod(sysStatus.keepAlive*1000);                                  // Will start the repeating timer
}

// After main loop - function
int setThirdPartySim(String command) // Function to force sending data in current hour
{
  if (command == "1")
  {
    sysStatus.thirdPartySim = true;
    Particle.keepAlive(sysStatus.keepAlive);                                                // Set the keep alive value
    keepAliveTimer.changePeriod(sysStatus.keepAlive*1000);                                  // Will start the repeating timer
    if (Particle.connected()) publishQueue.publish("Mode","Set to 3rd Party Sim", PRIVATE);
    sysStatusWriteNeeded = true;
    return 1;
  }
  else if (command == "0")
  {
    sysStatus.thirdPartySim = false;
    if (Particle.connected()) publishQueue.publish("Mode","Set to Particle Sim", PRIVATE);
    sysStatusWriteNeeded = true;
    return 1;
  }
  else return 0;
}


int setKeepAlive(String command)
{
  char * pEND;
  char data[256];
  int tempTime = strtol(command,&pEND,10);                                                  // Looks for the first integer and interprets it
  if ((tempTime < 0) || (tempTime > 1200)) return 0;                                        // Make sure it falls in a valid range or send a "fail" result
  sysStatus.keepAlive = tempTime;
  Particle.keepAlive(sysStatus.keepAlive);                                                // Set the keep alive value
  keepAliveTimer.changePeriod(sysStatus.keepAlive*1000);                                  // Will start the repeating timer
  snprintf(data, sizeof(data), "Keep Alive set to %i sec",sysStatus.keepAlive);
  publishQueue.publish("Keep Alive",data, PRIVATE);
  sysStatusWriteNeeded = true;                                                           // Need to store to FRAM back in the main loop
  return 1;
}

void keepAliveMessage() {                       // Function called by the keep alive timer
  Particle.publish("*", PRIVATE,NO_ACK);
}

This has made a big difference in these counters which used to look like this:

to this:

We are looking into the data usage implications of this approach but our assumptions are:

  • Given we are reporting every 20 minutes - staying connected is likely the best approach. It also helps to ensure that the 20 minute reporting is almost exactly on the mark.
  • setting a 120 second keep alive will burn data but it should be less than the data burned in constantly going offline and coming back online as we were doing

A few questions if anyone has experience with this:

  • Is there a value of keep alive below which it makes more sense to just disconnect and reconnect every 20 minutes?
  • In testing with US based devices, it seems that the Cellular.keepAlive() command is simply ignored. Is this really the case? Should we avoid using this command with a device using a Particle SIM?
  • Has anyone had any luck approaching carriers for longer keep alive periods? Is there another MVNO option which we could look at? We have looked at Hologram, Ping and others but none had the carrier we need in Rwanda. ;-(

Any advice appreciated and I hope what we have learned can be helpful to others.

Chip

@mstanley - is your KeepAliveTester also potentially relevant to share here?

Hey @marekparticle , thanks for the ping!

And thanks chip for sharing your insight and application there! I wrote a similar application which may be found on our Github at GitHub - particle-iot/keep-alive-tester: An app to help test and approximate necessary keep alive for a Particle cellular connected device. It uses a combination device + cloud side application through our event system to start at a short interval bi-directional communication and slowly increments its way up until the duplex communication is broken.

There is no expectation on our end that Cellular.keepAlive is region specific. We would expect it to to be compatible in all regions. It may be possible that if you are using a third party SIM, the NAT timeout for the US may differ from Rwanda.

A NAT timeout of 120 seconds is not atypical. I have observed operators with NAT timeouts anywhere in the range of 30 seconds to around 7 minutes at the most outside of Particle. 30s to 2 min is more common from my observations.

Assuming your data flow is unidirectional (device-out), letting the NAT timeout lapse shouldn’t be as big of an issue, as regardless of NAT state, the unit should always be able to talk out. The issue is simply acting down on the device while this NAT timeout lapses.

If you are comfortable with your operations down to the device always immediately following a data report in, you could design your infrastructure in such a way that it enqueues commands or messages to be sent down to a device following an incoming message. You could reasonably assume that a device is able to be communicated with for approximately 2 minutes following an incoming message.

1 Like

@mstanley,

Very cool application. I will certainly give it a go both here in the US and in Rwanda (3rd Party SIM) and New Zealand (Particle SIM). I have been very impressed that Particle is willing to invest significant time and support for customers who use 3rd Party SIMS.

Great point on the device-out mode of operation. If I understood correctly, if the device would publish data every twenty minutes and the connection would timeout 120 seconds later. The data payload would be delivered (device-out) and I would simply need to do any device-in interactions in that 120 seconds windows.

I assume that there would be some data overhead with re-establishing the connection each 20 minute period but this would be less than the 10x one-character publish with NO_ACK events I would avoid.

Thanks,

Chip

1 Like

Your assumption is correct here, yes.

There is an expectation of consumption per negotiation with the tower. I don’t know exactly what those numbers would be, but that might be worth doing some data analysis on in a few bench tests to compare pros and cons on!

1 Like