Electron Data Usage Overhead

General
1

I am developing a thin gateway for mesh networks with cellular capability. Maximizing data usage will be an important factor to consider due to a considerable amount of data created in the network.

I am testing Electron as the possible modem for each network. With that in mind, I ran some tests to measure resources consumption (Data usage and Battery). My code follows (firmware version 0.6.0-rc.1):

//#include "cellular_hal.h"
#include "Particle.h"

SYSTEM_MODE(SEMI_AUTOMATIC);
//STARTUP(cellular_credentials_set("apn.konekt.io", "", "", NULL));

void sendStream();
void readBattery();


// called once on startup
void setup() {
    
    Cellular.on();
    Cellular.connect();
    Particle.connect();
    waitUntil(Particle.connected);
}

void loop() {
        
    sendStream();
    readBattery();
    
    // sleeps for 5 min
    System.sleep(SLEEP_MODE_DEEP, 300, SLEEP_NETWORK_STANDBY); 
    
}

void sendStream() {
    String data = "{\"key\":\"bb16471e088b989350aa79a\",\"body\":\"40c06b72&102&M0:4.096:4.3&M1:0:0:0:358:359&M2:34.89:0.00&M3:63.0:75&M4:0.347:0.347&M5:7.8:20.7\"}";
    
    Particle.publish("streams", data, PRIVATE, NO_ACK);
}

void readBattery() {
    FuelGauge fuel;    
    String value = String(fuel.getVCell());    
    Particle.publish("librato", value, PRIVATE, NO_ACK);
}

Basically the code publishes a stream and the battery level to webhooks every 5 minutes. The tests were run with a third party SIM card and then with a Particle one (because at first I though it was a issue using a third party one).

Each call is publishing 156 Bytes of data (stream string, event names and float value).

These are my results (battery results are not include - they were pretty satisfactory):

Third party SIM card:

Total time: 7 hr 26 min
Total data consumption: 112,575 Bytes
89 calls * 156 Bytes each = 13,884 Bytes of application data

Particle SIM card

Total time: 7 hr 32 min
Total data consumption: 0.14 MB
90 calls * 156 Bytes each = 14,040 Bytes of application data

We see there is a pattern here. We have an overhead in an order of 10. This is too much to consider using Electron in my application.

I am using the Deep Sleep with SLEEP_NETWORK_STANDBY, which is supposed to give me lower data usage for Particle cloud re-connection and NO_ACK for my publish events.

Am I missing something here? Is there any other snippet of code I could incorporate in mine?

2

I’m running the same test with an Electron right now also.

I’m not sleeping in my test, though.

I am publishing every five mins 200 bytes of data.

I have calculated that the data used to post every five mins for 24 hours ends up being 0.07 MB per day.

If you’re using the 0.6.0_rc1 firmware then every time you wake up withing a 23 min timeframe the handshake consumes approx 136 bytes. So Every time you wake up you’re consuming 136 bytes x 20 times per hour = 2720 bytes per hour. That’s equals 65280 bytes per 24 hours just for waking up every five mins.

The problem with the 3rd party sims is that they require a ping with the Particle Servers every 30 seconds and adds up big time. If you Ping at 136 bytes x 120 times an hour that equals 16320 bytes. 16320 bytes x 24 hours equals 391680 bytes per day. You would eat up 1Mb every three days doing this assuming your not sleeping which you are.

3

Here is the test code I used to see how much data was actually received and sent using the Electron for each Webhook event.

Load the code, change the publish event below for the P switch case event to whatever your using.

Then hook the Electron up to your computer and open the serial monitor window and you will have really accurate numbers on what the difference in data consumption is between different setups. The numbers worked out to be accurate to me and what was shown in the Particle data usage dashboard.

// EXAMPLE



void setup()
{
  Serial.begin(9600);
}

void loop()
{
    
    
    if (Serial.available() > 0)
    {
        char c = Serial.read();
        if (c == '1') {
            Serial.println("Read counters of sent or received PSD data!");
            CellularData data;
            if (!Cellular.getDataUsage(data)) {
                Serial.print("Error! Not able to get data.");
            }
            else {
                Serial.printlnf("CID: %d SESSION TX: %d RX: %d TOTAL TX: %d RX: %d",
                    data.cid,
                    data.tx_session, data.rx_session,
                    data.tx_total, data.rx_total);
                Serial.println(data); // Printable
            }
        }
        else if (c == '2') {
            Serial.println("Set all sent/received PSD data counters to 1000!");
            CellularData data;
            data.tx_session = 1000;
            data.rx_session = 1000;
            data.tx_total = 1000;
            data.rx_total = 1000;
            if (!Cellular.setDataUsage(data)) {
                Serial.print("Error! Not able to set data.");
            }
            else {
                Serial.printlnf("CID: %d SESSION TX: %d RX: %d TOTAL TX: %d RX: %d",
                    data.cid,
                    data.tx_session, data.rx_session,
                    data.tx_total, data.rx_total);
                Serial.println(data); // Printable
            }
        }
        else if (c == '3') {
            Serial.println("Reset counter of sent/received PSD data!");
            if (!Cellular.resetDataUsage()) {
                Serial.print("Error! Not able to reset data.");
            }
        }
        else if (c == 'p') {
            Serial.println("Publish some data!");
              float f = 100.5;
             // Temperature is measured every time RH is requested.
             // It is faster, therefore, to read it from previous RH
             // measurement with getTemp() instead with readTemp()
             char payload[255]; 

             snprintf(payload, sizeof(payload), "{ \"s\":\"wthr\", \"u\":\"F\",\"l\":\"%s\",\"m\":\"Temperature\",\"o\":\"%s\",\"v\": %f,\"d\":\"%s\" }", f);
             Spark.publish("AzureElectron", payload, NO_ACK );   //The NO_ACK is currently only working with RC 0.6.0 rc1 firmware. This is just a test. 
        }
        while (Serial.available()) Serial.read(); // Flush the input buffer
    }

}
4

Have you tried something like System.sleep(BTN, FALLING, 300, SLEEP_NETWORK_STANDBY);?
You might need to pull that waitUntil(Particle.connected); into loop() tho’.
This might have a marginal impact on the power consumption with only 5min sleep time, but could help pushing data consumption down a bit.

You can also drop Cellular.connect() since Particle.connect() does that implicitly.

Pulling your two publishes together into one will also save data.

But, do you always see all you publishes being successfully delivered?
There was the need to add some 5sec wait before going to sleep after a publish and I haven’t heard anything about that having changed recently.

5

@ScruffR What does the “BTN” in the System.sleep code stand for?

6

That’s the “pin name” for the SETUP/MODE button, which i like to use to wake devices from sleep without the need to add my own button.

1 Like
7

@RWB have you taken a look at your battery levels during the test? Keeping the device up 24/7 seems to be very battery consuming. I had the same problem with the Particle SIM too. I was expecting to have an overload of twice as much as sent data.

@ScruffR do you know if there is a maximum amount of data that should be sent in the same publish event? Does System.sleep(BTN, FALLING, 300, SLEEP_NETWORK_STANDBY); wake the device up either by the SETUP/MODE button or when the time goes off?

8

I am only leaving the Electron powered up all the time because I'm just testing data rates for sending the data over to Azure to be stored longterm. In some of my applications I will have to put the device into deep sleep just like you.

Is your data consumption number making more sense now that your calculating the 136 byte pings 20 when you wake up 20 times per hour?

I have some Konekt sim cards also but have not tried them yet. Is all that you needed to do to get it working was to add the lines of code that are commented out below?

//#include "cellular_hal.h"
#include "Particle.h"

SYSTEM_MODE(SEMI_AUTOMATIC);
//STARTUP(cellular_credentials_set("apn.konekt.io", "", "", NULL));

Do you know what the required Particle Ping rate is when using the Konekt cellular service? I hear some cellular services require 30 seconds pings to keep the connection alive but I'm not sure what the minimum ping rate is for the Konekt network. Do you know?

9

That's currently documented as
https://docs.particle.io/reference/firmware/photon/#particle-publish-

Exactly! Just as the docs say
https://docs.particle.io/reference/firmware/photon/#sleep-sleep-

10

Would it be 12 times per hour? If the device wakes up every 5 minutes, in 60 minutes it will wake up 12 times. It makes a lot of sense. Although it explains 1/9 approx. of the overload. Using the ping calculation of every 30 seconds totally adds up though. I would expect that the Particle SIM would not have that.

Exactly, simple as that. To be honest with you, observing the LEDs from the board, Konekt would connect faster to the cell tower than the SIM one. Maybe it is due the location I am in and the carrier they work with.

I am not sure of that. I guessed that it is close to 30 since using it adds up.

11

@ScruffR Do you know if including the following line of code to use 3rd part sim cards automatically sets the Particle ping to 30 seconds are does the ping have to be called in the main loop to run every 30 seconds?

#include "cellular_hal.h"

Yes your right, I got my numbers wrong :wink:

Yea I don't think there is that much overhead. My calculations come out right and I'm shurly not seeing 9 x overhead like you are.

12

For instance, the Konekt SIM used 112,575 bytes, from which 13,884 bytes were application data. If we had only a ping when the device wakes up it would be 136127.5 = 12,240 bytes. But if the device works pinging every 30 seconds even in deep sleep it totally fills in the rest of the data.

13

@BDub When using a 3rd party sim & this code

System.sleep(SLEEP_MODE_DEEP, 300, SLEEP_NETWORK_STANDBY);

The 136 byte pings do not keep happening while in this sleep mode, would they?

14

Not currently, and not required at all for 300 second sleep delays :smile: Using SLEEP_NETWORK_STANDBY keeps the modem on and connected to the tower in the background.

15

No, this line doesn't really do anything other than making functions (in this case cellular_credentials_set()) available/usable in your own code.
The command for the cloud ping would be Particle.keepAlive() which is not declared in that header.
And that command also rids you of the need to care about pings because it sets the periode with which the system will do the pinging in the background if your code doesn't talk to the cloud for "too long".

16

Hey @RWB I perform another test similar to the first one. It is not using the SLEEP functionality though. This time I just created a timer to count 5 minutes and publish the data.

Started at 8:00am EDT
Battery: 3.97V

Finished at 15:30 EDT
Battery: 3.83V

Total time: 7.5 hours
Data consumed: 0.04 MB (41,943 bytes).

Using Electron data usage doc I calculated the data used in this period.

90 times sending x 156 bytes of app data = 14,040 bytes
318 bytes avg per hour for pings x 7.5 hours = 2,385 bytes
67 bytes of overhead per publish event x 90 times sending x 2 events = 12,060 bytes

Total: 28,485 bytes

Unknown: 13,458 bytes - most likely used for performing the connection for the first time

This is a better scenario than the one I had using SLEEP. The SLEEP code consumed 0.14 MB in 7.5 hours in contrast to 0.04 MB in the same time frame.

It seems the problem lies with the wake up and context re-connection.

17

Maybe you are already doing this but could you use the Cellular.getDataUsage() function before sleep and immediately after sleep to get an exact amount used in the process?