@bpr @sbright33 Can you guys take a look at the latest Cellular Data Counter API code and see if you can work some code up that will allow us to see the actual data used for each transmission. I’m have no free time at the moment.
It looks like it should be easy to do with the latest firmware: https://docs.particle.io/reference/firmware/electron/#getdatausage-
I'd say it's getting contact to the cell tower, then making contact to the relay servers and acquiring an IP address.
Which will (in parts) also happen when you see it directly go to breathing cyan, since it'll be near instant (unseen) since all the small talk neccessary when they first met had already happened earlier.
Simply using the example in the docs I get:
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 8419 RX: 5867 TOTAL TX: 8419 RX: 5867
31,8419,5867,8419,5867
Set all sent/received PSD data counters to 1000!
CID: 31 SESSION TX: 1000 RX: 1000 TOTAL TX: 1000 RX: 1000
31,1000,1000,1000,1000
Reset counter of sent/received PSD data!
Publish some data!
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 134 RX: 122 TOTAL TX: 134 RX: 122
31,134,122,134,122@bpr So does this data look right to you?
Can you compare them to deep sleep and see what the data differences are?
They look WAY underestimated to me, but what do I know!? As a matter of fact just starting to test with sleep.
@Bdub @Mdma Does the above data returned from the Electron Data API look correct to you?
Were trying to use this to see how much data is actually being used for different sleep modes and sending data via Particle.publish vs Ubidots transmit code.
Here’s the serial1 output (with annotation{}) I get using stop mode sleep. If I’m going about this wrong someone please tell me:
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 12789 RX: 8355 TOTAL TX: 12789 RX: 8355
31,12789,8355,12789,8355
{before any reset}
Publish some data!
{"e1", "0MSSA"}
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 12983 RX: 8538 TOTAL TX: 12983 RX: 8538
31,12983,8538,12983,8538
{yields 194, 183}
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 13044 RX: 8599 TOTAL TX: 13044 RX: 8599
31,13044,8599,13044,8599
{just wakeup and get new data usage - 61, 61}
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 13105 RX: 8660 TOTAL TX: 13105 RX: 8660
31,13105,8660,13105,8660
{just wakeup and get new data usage - 61, 61}
Reset counter of sent/received PSD data!
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 61 RX: 61 TOTAL TX: 61 RX: 61
31,61,61,61,61
{usage reported with wakeup only, no publish, after reset}
{CODE used}
#include "Particle.h"
SYSTEM_MODE(SEMI_AUTOMATIC);
char publishStr[10];
int sleepInterval = 0;
bool pubsuccess;
uint32_t start;
void readCounters() {
Serial1.println("Read counters of sent or received PSD data!");
CellularData data;
if (!Cellular.getDataUsage(data)) {
Serial1.print("Error! Not able to get data.");
}
else {
Serial1.printlnf("CID: %d SESSION TX: %d RX: %d TOTAL TX: %d RX: %d",
data.cid,
data.tx_session, data.rx_session,
data.tx_total, data.rx_total);
Serial1.println(data);
}
}//void resentCounters()
void setCountersto1000() {
Serial1.println("Set all sent/received PSD data counters to 1000!");
CellularData data;
data.tx_session = 1000;
data.rx_session = 1000;
data.tx_total = 1000;
data.rx_total = 1000;
if (!Cellular.setDataUsage(data)) {
Serial1.print("Error! Not able to set data.");
}
else {
Serial1.printlnf("CID: %d SESSION TX: %d RX: %d TOTAL TX: %d RX: %d",
data.cid,
data.tx_session, data.rx_session,
data.tx_total, data.rx_total);
Serial1.println(data);
}
}//void setCountersto1000()
void resetCounters() {
Serial1.println("Reset counter of sent/received PSD data!");
if (!Cellular.resetDataUsage()) {
Serial1.print("Error! Not able to reset data.");
}
}//void resetCounters()
void publishData() {
Serial1.println("Publish some data!");
sprintf(publishStr, "%iMSSA",sleepInterval);
Particle.connect(); //do we need this??? apparently DO if
//sleep duration > 25 mins
waitUntil(Particle.connected);
//do we need this??? apparently DO if
//sleep duration > 25 mins
start = millis();
while (millis() - start < 6000UL) {Particle.process();}
pubsuccess = Particle.publish("e1", publishStr, 60, PRIVATE);
//do we need this??? apparently DO if
//sleep duration > 25 mins
start = millis();
while (millis() - start < 6000UL) {Particle.process();}
}//void publishData()
void setup() {
pinMode(D1, INPUT);
pinMode(WKP, INPUT);
Serial1.begin(9600);
Particle.connect();
waitUntil(Particle.connected);
start = millis();
while (millis() - start < 2000UL) {Particle.process();}
}//setup()
void loop() {
System.sleep(D1, RISING, SLEEP_NETWORK_STANDBY);// wake on pin D1
start = millis();
while (millis() - start < 6000UL) {Particle.process();}
while (!Serial1.available()) {}
char c = Serial1.read();
if (c == '1') {
readCounters();
}
else if (c == '2') {
setCountersto1000();
}
else if (c == '3') {
resetCounters();
}
else if (c == 'p') {
publishData();
}
while (Serial1.available()) Serial1.read(); // Flush the input buffer
}//loopUsing almost same code in AUTOMATIC mode (without Particle.connect() or waitUntil…) I get similar data usage:
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 14282 RX: 9225 TOTAL TX: 14282 RX: 9225
31,14282,9225,14282,9225
Publish some data!
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 14476 RX: 9408 TOTAL TX: 14476 RX: 9408
31,14476,9408,14476,9408
{Usage 194, 183}
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 14537 RX: 9469 TOTAL TX: 14537 RX: 9469
31,14537,9469,14537,9469
{just upon wakeup usage 61, 61}
Reset counter of sent/received PSD data!
Publish some data!
Read counters of sent or received PSD data!
CID: 31 SESSION TX: 266 RX: 244 TOTAL TX: 266 RX: 244
31,266,244,266,244
{Usage 266, 244}When I run the following loop under AUTOMATIC mode I seem to be able to retrieve the data usage involved in negotiating the connection and usage due to data publish altogether. I’m assuming this kind of usage (depending of course on the size of the data) will occur on startup as well as after about 25 minutes of sleep (?) as renegotiation seems to occur then:
#include "Particle.h"
//SYSTEM_MODE(SEMI_AUTOMATIC);
SYSTEM_MODE(AUTOMATIC);
char publishStr[10];
int sleepInterval = 0;
bool pubsuccess;
uint32_t start;
CellularData data;
void readCounters() {
Serial1.println("Read counters of sent or received PSD data!");
// CellularData data;
if (!Cellular.getDataUsage(data)) {
Serial1.print("Error! Not able to get data.");
}
else {
Serial1.printlnf("CID: %d SESSION TX: %d RX: %d TOTAL TX: %d RX: %d",
data.cid,
data.tx_session, data.rx_session,
data.tx_total, data.rx_total);
Serial1.println(data);
}
}//void resentCounters()
void setCountersto1000() {
Serial1.println("Set all sent/received PSD data counters to 1000!");
// CellularData data;
data.tx_session = 1000;
data.rx_session = 1000;
data.tx_total = 1000;
data.rx_total = 1000;
if (!Cellular.setDataUsage(data)) {
Serial1.print("Error! Not able to set data.");
}
else {
Serial1.printlnf("CID: %d SESSION TX: %d RX: %d TOTAL TX: %d RX: %d",
data.cid,
data.tx_session, data.rx_session,
data.tx_total, data.rx_total);
Serial1.println(data);
}
}//void setCountersto1000()
void resetCounters() {
Serial1.println("Reset counter of sent/received PSD data!");
if (!Cellular.resetDataUsage()) {
Serial1.print("Error! Not able to reset data.");
}
}//void resetCounters()
void publishData() {
Serial1.println("Publish some data!");
sprintf(publishStr, "%i,%i",data.tx_total, data.rx_total);
// Particle.connect(); //do we need this??? apparently DO if
//sleep duration > 25 mins
// waitUntil(Particle.connected);
//do we need this??? apparently DO if
//sleep duration > 25 mins
start = millis();
while (millis() - start < 6000UL) {Particle.process();}
pubsuccess = Particle.publish("e1", publishStr, 60, PRIVATE);
//do we need this??? apparently DO if
//sleep duration > 25 mins
start = millis();
while (millis() - start < 6000UL) {Particle.process();}
}//void publishData()
void setup() {
pinMode(D1, INPUT);
pinMode(WKP, INPUT);
Serial1.begin(9600);
// Particle.connect();
// waitUntil(Particle.connected);
start = millis();
while (millis() - start < 2000UL) {Particle.process();}
}//setup()
void loop() {
System.sleep(D1, RISING, SLEEP_NETWORK_STANDBY);// wake on pin D1
start = millis();
while (millis() - start < 6000UL) {Particle.process();}
// resetCounters();
readCounters();
publishData();
}//loop
the Data reflects tx and rx total upon startup and after 1st publish of that data so just connecting uses 3589, 2476 and subsequent publish only 137, 122 bytes. Does this seem correct? If so, the hefty part is the cellular handshaking
The good news is that it looks like the data counters are working.
Keep it up, I don’t have time to play or fully understand the results your getting just yet but I will soon.
If you can make a chart showing different data points using system.sleep vs deep sleep vs system sleep for the longer sleep periods that would be interesting to see. I’ll do that if you don’t have time for it before I can get to it.
Overall how is it looking? Better or worse than expected?
Looking good data usage wise according to the data counters - we’ll see tomorrow when the the Billing & Usage display updates 
I did find that sending two small floats (Volts and SoC) directly to Ubidots appeared to consume 5-7 times what a publish to the particle cloud consumes, so I simply have the electron publish to the particle cloud and have a photon subscribing to that publish event and the photon then sends the data to Ubidots.
Thanks for running that test of Particle.publish (Volts and SoC) vs Ubidots publish (Volts and SoC).
5-700% more data consumption per transmission really adds up if your sending frequently. I’m currently publishing (Volts and SoC) data every 60 mins on one device and every half hour on another Electron just for play. They both wake up instantly to berating blue LED and go right back to sleep exactly as expected.
It’s so nice that the Electrons maintain solid reliable data network connections. Since I use my cell phone for my main WiFi hot spot my Wifi connection is always going in and out and leaves with me every time I go some where. The Electons just work all the time now and that’s great to see now with the System.sleep function working correctly.
@aguspg Based on the testing that @bpr has done with tracking data used vs Particle Publish vs Ubidots publish is there any way we can get the data into Ubidots using the lower data rate Particle.Publish method?
Can Ubidots listen for Particle.publish events and store them in the database like normal some how?
Thanks for your input @bpr! we’re improving the Particle library now that the electron is out and data usage quickly became a priority. To make sure you have the latest, click on the “i” for more info next to the library:
Then click on the latest one:
@RWB yes we had done a Particle publish integration but there’s a limit on how much Particle users can send data to external hosts, so it quickly became crowded. We’re talking to Particle to extend this limit.
Next step: provide an option to choose another protocol different than TCP.
@bpr Hey how has the system sleep mode been working for you with the new firmware?
I have been reliably sending data to Ubidots from 2 Electrons with wake periods of 30 mins on one and 1 hour on the other. Zero problems so far.
I’ve been running the Electron off the fully charged 2000mAh battery for a week straight with 30 min wake up periods.
@RWB Yes works well though mine pooped out at Volts= 3.79 and SoC= 72.09 after just under 5 days using NO_ACK (pretty darn reliable) using the code below. Could you show your code?
Edit: forgot to mention I just pulished to cloud and then had a photon subscribe to the event which then published to Ubidots and maybe used only 0.16MB? or so
#include "Particle.h"
SYSTEM_MODE(AUTOMATIC);
char publishStr[20];
int sleepInterval = 30;
uint32_t start;
void publishData() {
FuelGauge fuel;
sprintf(publishStr, "%02.2f %03.2f", fuel.getVCell(), fuel.getSoC());
Particle.publish("e2", publishStr, PRIVATE, NO_ACK);
start = millis();
while (millis() - start < 1000UL) { Particle.process();}
}//void publishData()
void setup() {
pinMode(D1, INPUT);
pinMode(WKP, INPUT);
}//setup()
void loop() {
publishData();
System.sleep(D1, RISING, 60 * sleepInterval);
}//loop@BPR I like how your keeping data rates down by using particle publish to a Photon.
Have you tried this yet?
It looks as if it would accomplish the same thing without needing the extra Photon in the mix. Basically Ubidots will receive the Particle.publish event directly without needing the Photon to redirect it. Let me know.
And here is the code I'm using to send data directly to Ubidots. If your willing run a Data rate test vs the Particle.publish directly to Ubidots and see what the difference is.
// This #include statement was automatically added by the Particle IDE.
#include "Ubidots/Ubidots.h"
#define TOKEN "TGiZfnCx9eWGlLC" // Put here your Ubidots TOKEN
Ubidots ubidots(TOKEN); // A data source with particle name will be created in your Ubidots account
int button = D0;
int ledPin = D7; // LED connected to D1
int sleepInterval = 60;
void setup(){
pinMode(button, INPUT_PULLDOWN); // sets pin as input
pinMode(ledPin, OUTPUT); // sets pin as output
//Serial.begin(115200);
ubidots.setDatasourceName("PinWakeTestCode"); // Uncomment this line to change the data source Name.
}
void loop(){
FuelGauge fuel;
float value1 = fuel.getVCell();
float value2 = fuel.getSoC();
ubidots.add("Volts", value1); // Change for your variable name
ubidots.add("SOC", value2);
ubidots.sendAll();
digitalWrite(ledPin, HIGH); // sets the LED on
delay(500); // waits for a second
digitalWrite(ledPin, LOW); // sets the LED off
delay(500); // waits for a second
digitalWrite(ledPin, HIGH); // sets the LED on
delay(500); // waits for a second
digitalWrite(ledPin, LOW); // sets the LED off
delay(500); // waits for a second
digitalWrite(ledPin, HIGH); // sets the LED on
delay(500); // waits for a second
digitalWrite(ledPin, LOW); // sets the LED off
delay(500); // waits for a second
System.sleep(D0, RISING, sleepInterval * 60, SLEEP_NETWORK_STANDBY);
}
@RWB the webhook method is interesting. Haven’t tried it yet. I am finding that publishing when using the SLEEP_NETWORK_STANDBY parameter fails except for the first initial bootup publish unless the sleep period is rather short (only tested 3 minutes vs 30 minutes). Wonder why SLEEP_NETWORK_STANDBY is not working for me? Could it be that NO_ACK and SLEEP_NETWORK_STANDBY don’t play well together at longer intervals ? Something else the matter with my code?
@BPR The SLEEP_NETWORK_STANDBY just allows a quick re-connection to the 3G network when waking right? I think without the SLEEP_NETWORK_STANDBY you get the flashing green light for approx 30 seconds while waiting on the Electron to reconnect to the 3G network after waking up. I think I have that right.
I have not tried the NO_ACK yet so maybe that is causing your problems. Or it could be just a Particle Publish issue since I’m having no problems sending data to Ubidots directly.
When you tested the Ubidots data rates were you using their latest code? I wonder if the new code is still using 600% more data than the Particle.publish?
I’m just happy the System.sleep is working and I can wake from a Pin change state now but as your trying now getting the data transfer rates as low as possible is the next step for optimization.
Keep me updated, I’m very short on time or I would be doing all this with you.
Hi, All. I’m trying to post data to Ubidots using my Electron (Firmware 0.4.8). I can post approx 5 secs worth of data to my Ubidots dashboard, but then my Electron hangs. That is, after 5 secs of running it does not post data to Ubidots dashboard. In addition, I can not re-flash my Electron remotely (over 3G), and therefore, I have to set manually the system into SAFE MODE. Note: I’m not using Particle SIM, but a Telstra SIM card (Australian). Also, I’m using Particle’s Solar Sheild. Can someone please help me or point me in the right direction?
Long term, I’m keen to use System.sleep(SLEEP_MODE_DEEP,60) to increase my Electron solar / battery life.
Thanks
// This example is to save multiple variables to the Ubidots API
// Define Telstra APN
#include "cellular_hal.h"
STARTUP(cellular_credentials_set("telstra.iph", "", "", NULL));
// This #include statement was automatically added by the Particle IDE.
#include "Ubidots/Ubidots.h"
#define TOKEN "BYVc##############syZ" // Put here your Ubidots TOKEN
Ubidots ubidots(TOKEN); // A data source with particle name will be created in your Ubidots account
void setup(){
pinMode(A0,INPUT); // Our photoresistor pin is input (reading the photoresistor)
pinMode(D7,OUTPUT); // Make D7 an output pin
}
void loop(){
// Define fuel
FuelGauge fuel;
float value1 = fuel.getVCell();
float value2 = fuel.getSoC();
float value3 = analogRead(A0);
ubidots.add("BatVolts", value1); // Change for your variable name
ubidots.add("SoC", value2); // Change for your variable name
ubidots.add("Depth", value3); // Change for your variable name
ubidots.sendAll();
digitalWrite(D7,HIGH);
delay(100);
digitalWrite(D7,LOW);
delay(1000);
// Loop again...
}@samb375, you may want to try with the latest 0.5.0 firmware which has a lot of fixes since 0.4.8.