Hi everybody. It is a few days that I am doing transmission tests on the Electron and the photon: the purpose is to send from the electron some data that has been recorded previously to my server (a standard MySQL database). An example could be the gps data read from the sensor at 5Hz, stored inside an array, sent every 5 seconds. I have no problem storing data, but I cannot find a good way to send it: the first problem using the particle cloud, you can publish events every one second, but the maximum length of the data is at least 250 byte. That is not enough. Maybe it could be, but doing lots of tests, I see that the Particle.publish function is not so reliable. Sometimes data is not sent/received, especially if I try to publish every second.
Another way is to use a TCP connection: I created a php webservice on my host that communicates data to the MySQL database. So, creating from a photon a TCP client with POST/GET request, it seems very working good: posting 2KB of data every 5 seconds, everything is great (with 3KB there are some problems, but 2KB is enough now). The problem comes with the electron: with the same code and the same amount of data, connection is about 25-45 seconds. If I try to send 10 bytes, it holds for 20 seconds. And it is not so reliable.
Actually, the Particle cloud is great because the communication is very fast, but it is not reliable, and the amount of data is limited. I have seen the discussions on TCP/UDP here, but none of them is satisfying my question: is there a reliable way to send about 2 KB of data every 4/5 seconds to a remote server/cloud with electron? Is there a way to create a continuos streaming to a server/web service (remote server/hosting/MySQL)? Do you have any advice? Thanks in advance.
I was able to send 2048 bytes every 4 seconds from an Electron 3G with the code below. The server side validated the data and it was sent correctly. I didn’t let it run that long, because I don’t have an unlimited data SIM, but the transmit only takes one second so there’s time for it to catch up if it falls behind. The main thing is that you probably want to use the same TCP connection (don’t close and reopen) and write 512 bytes at a time for best results (don’t write a byte at a time).
This code also works on the Photon. In a different test I was able to transmit about 800 Kbytes/sec continuously from the Photon for over 24 hours without difficulty.
// Test app for sending large amounts of data over a TCP connection
#include "Particle.h"
// This test can be run with system thread enabled or disabled.
SYSTEM_THREAD(ENABLED);
// Finite state machine states
enum State { STATE_CONNECT, STATE_FILL_BUFFER, STATE_SEND, STATE_SEND_WAIT, STATE_RETRY_WAIT };
// Various constants
// bufSize is the size of the data buffer we want to send
const size_t bufSize = 2048;
// maxWriteSize is the maximum number of bytes to write in a single call
// 1024 works well for the Photon.
// Setting it larger than 1024 doesn't really do anything for the Electron
// because it breaks large writes into smaller writes, anyway.
const size_t maxWriteSize = 512;
// Various timeout values.
const unsigned long retryWaitTimeMs = 2000;
const unsigned long sendTimeoutMs = 60000;
const unsigned long sendPeriodMs = 4000;
// Set to the IP address of the server to connect to
IPAddress serverAddr(10,10,10,10);
const int serverPort = 5123;
// Global variables
State state = STATE_CONNECT;
TCPClient client;
unsigned long stateTime = 0;
unsigned char buf[bufSize];
size_t sentInBuf;
unsigned long totalSent = 0;
unsigned char bufStartChar = 0;
unsigned long sendStart = 0;
void setup() {
Serial.begin(9600);
}
void loop() {
switch(state) {
case STATE_CONNECT:
Serial.printlnf("** starting millis=%lu size=%u", millis(), bufSize);
if (!client.connect(serverAddr, serverPort)) {
// Connection failed
stateTime = millis();
state = STATE_RETRY_WAIT;
break;
}
totalSent = 0;
state = STATE_FILL_BUFFER;
// Fall through
case STATE_FILL_BUFFER:
// Each buffer has different bytes so we can detect corruption on the server
for(size_t ii = 0; ii < bufSize; ii++) {
buf[ii] = bufStartChar++;
}
sentInBuf = 0;
state = STATE_SEND;
stateTime = millis();
sendStart = millis();
// Fall through
case STATE_SEND:
if (client.connected()) {
size_t reqCount = bufSize - sentInBuf;
if (reqCount > maxWriteSize) {
reqCount = maxWriteSize;
}
int count = client.write(&buf[sentInBuf], reqCount);
if (count == -16) {
// Special case: Internal buffer is full, just retry at the same offset next time
// I'm pretty sure the result code for this is different on the Core
// This condition does not appear to occur on the Electron, which tends to block instead.
}
else
if (count > 0) {
// Normal case, sent some bytes. count may be less than the buffer size, which
// means we need to send the rest later.
stateTime = millis();
totalSent += count;
sentInBuf += count;
if (sentInBuf >= bufSize) {
Serial.printlnf("sent in %lu", millis() - sendStart);
// Sent whole buffer, refill after waiting sendPeriodMs from last send start
state = STATE_SEND_WAIT;
}
}
else {
// Error
Serial.printlnf("** error sending error=%d sentInBuf=%u totalSent=%lu millis=%lu", count, sentInBuf, totalSent, millis());
client.stop();
stateTime = millis();
state = STATE_RETRY_WAIT;
}
}
else {
Serial.printlnf("** connection closed totalSent=%lu millis=%lu", totalSent, millis());
client.stop();
stateTime = millis();
state = STATE_RETRY_WAIT;
}
break;
case STATE_SEND_WAIT:
if (millis() - sendStart > sendPeriodMs) {
// Wait before sending the next buffer
state = STATE_FILL_BUFFER;
break;
}
break;
case STATE_RETRY_WAIT:
if (millis() - stateTime > retryWaitTimeMs) {
// Wait a few seconds before retrying
state = STATE_CONNECT;
break;
}
break;
}
}
2 Likes
Hi rickkas7,
first of all, thanks a lot for the code. Thanks for the help.
I am testing your code (without changing it, I changed the server with my website “www.skyavio.com” and the port is 80) on my photon and electron, and I see that it seems to be working on the photon, also if it is blocking after 8/9 connections: the serial logs are the following (I only added serial print STATE FILL BUFFER in the case):
On the Electron, it is not connecting at all:
As you can see, I set also 1024 byte (instead of 2048).
Actually, on the server there is not a TCP Server, but if you try with online tools that verify TCP connection, it is working fine. I am just wondering why on the photon is working and on the electron is not. (The electron is a 3G, Model U270, I tested both particle sim and 3rd party sims).
Thank in advance!
Marco
I got error -1 when sending on the Electron when I used maxWriteSize = 1024. I’d try making it even smaller, maybe 256 or even 128. The IP stack on the Electron is completely different than the Photon, so it’s not surprising that they behave differently.
1 Like
