Trouble capturing a serial stream on Serial1

Troubleshooting
1

I started out trying to capture a serial stream from a device @115200, but ran into troubles. So I created a bare bones example code to test, and i still cannot capture all the data properly. Am I doing something wrong, or what ?
It seems its filling the 64byte buffer faster than it can pull it out. Sadly acquireSerialBuffer() is not available for serial1 ;(

What I am doing here is I have a terminal window opened on the Photons USB port, and another terminal window with a USB converter on Serial1 tx/rx pins. On app start I send a text file which contains the following(see below), then when it's sent I press "!" on the USB terminal to see what we captured. The output shows different chars captured every time.

Here is the output of the terminal after I have sent the text file, and pressed "!"
Then send it again and "!" again. So you can see its getting different char counts.

System Boot - sv: 0.6.0 - fv 1.0.0

Got: 371
This is the serial test for trying to see if it hiccups #1
Thiis the serial test for trying to see if it hiccups #2
This is is the serial test for trying to see if it hiccups #4
This is ts the serial test for trying to see if it hiccups #6
This is ts the serial test for trying to see if it hiccups #8
This is ts the serial test for trying to see if it hiccups #10
Got: 365
This is the serial test for trying to see if it hiccups #1
Thi serial test for trying to see if it hiccups #2
This is the se serial test for trying to see if it hiccups #4
This is the serserial test for trying to see if it hiccups #6
This is the serserial test for trying to see if it hiccups #8
This is the serserial test for trying to see if it hiccups #10

Here is the text file I am sending to mimic a device sending data.

This is the serial test for trying to see if it hiccups #1
This is the serial test for trying to see if it hiccups #2
This is the serial test for trying to see if it hiccups #3
This is the serial test for trying to see if it hiccups #4
This is the serial test for trying to see if it hiccups #5
This is the serial test for trying to see if it hiccups #6
This is the serial test for trying to see if it hiccups #7
This is the serial test for trying to see if it hiccups #8
This is the serial test for trying to see if it hiccups #9
This is the serial test for trying to see if it hiccups #10

And here is my code:

//https://docs.particle.io/reference/firmware/photon/#system-thread
SYSTEM_THREAD(ENABLED);

//**************************************************************************
//
//
//**************************************************************************
void setup()
{
Serial.begin(115200);
Serial1.begin(9600);

delay(4000);

Serial.printf("\r\n************************************\r\n");
Serial.printf("System Boot - ");
Serial.printf("sv: %s", System.version().c_str());
Serial.printf(" - fv 1.0.0");
Serial.printf("\r\n************************************\r\n");

}

char temp[1024];
uint16_t loc=0;
uint16_t t;
//**************************************************************************
//**************************************************************************
void serialEvent1()
{
// Grab all the data from Serial1 before its 64 byte buffer fills
while(Serial1.available() > 0)
{
// read the incoming byte:
temp[loc] = Serial1.read();
if(++loc == 1024)
{
loc=0;
}
}
}
//**************************************************************************
//**************************************************************************
void loop()
{

    // Check USB serial to see if we want to print the result yet
    if(Serial.available())
      {
          // If ! is pressed
          if(Serial.read() == 0x21)
          {
            //Show how many chars we captured
            Serial.printf("Got: %u\r\n", loc);

            for(t=0;t<loc;t++)
            {
                Serial.print(temp[t]);
            }

            // Now reset it for the next go arround
            loc=0;
          }
    }

  delay(125);

}

2

One major issue here is your delay(125) and the problem that void serialEvent1() is a misleading name for a synchronous functiont that’s only called between iterations of loop() so when you keep waiting for 125ms in loop() you will not read out the RX buffer which will lead to your overflow.

I’ve argued against that behaviour of serialEvent() but always got the “well, that’s how Arduino does it” response, so I gave up.

1 Like
3

Ah, ok thank you for the clarification!

I put the delay in there to mimic me doing other things in the loop, like updating the LCD and reading sensors. Having no control when I would receive serial data from the external device, I thought that serialEvent1() could be used to pull out the buffered data in the background.

Even though i read this about it, “The serialEvent functions are called by the system as part of the application loop.” I somehow took it as doing it in the background , go figure…

Well, this would be a perfect example should you talk to them again about this of why you should not do it the Arduino way. Popularity of something does not mean they do it right, and more importantly serial data should be considered a critical event, you cannot be expected to sit in a loop somewhere and wait for data to arrive when you have other things to do.

Now i really need them to make acquireSerialBuffer() work for Serial1, not sure why they didn’t add that in there while they would adding the others.

1 Like
4

You might be able to use Software Timers tho’
They can fire every 1ms async to your main app code - and to keep emptying the RX buffer with constant 115200 streaming in you’d need to do that at least once per 2.7ms

1 Like
5

LOL, that’s exactly what I am working on trying now. Thanks for the tip though.

6

Just an update FWIW.
using a timer does work and allow me to grab everything @115200.
Timer is set for 4ms, and I am able to pull out the data ~20 bytes before she hits the Serial1 size limit of 64.

Now, I am off to add the rest of the code in to talk to the other devices, and I am going to bet that with me getting yanked away every 4ms is going to cause problems with the SPI & I2c.