Serial1 buffer acting up?

Respected folks,

I can’t get rid of ‘gibberish characters’ when monitoring “Serial1” and printing to “Serial” (Serial1 buffer to USB Serial monitor, no wires hooked to Tx, Rx pins of Serial1) … I’ve read nearly all the guides (“serial tutorial”, troubleshooting posts like this, general C++ syntax, and even the Arduino forum!).

I’m simply trying to get 2 Photons talk to each other before more ambitious plans. Any pointers would be much appreciated, especially for a confidence boost!

MY CODE:
(On transmitting Photon, could be receiver as well, device firmware = 0.7.0)

#include "Particle.h"

// Constants
const size_t READ_BUF_SIZE = 64;
const unsigned long SEND_INTERVAL_MS = 2000;

// Forward declarations
// Put function definitions here

// Global variables
char readBuf[READ_BUF_SIZE];
size_t readBufOffset = 0;
unsigned long lastSend = 0;

int counter = 0;                            // Can pre or post-increment
char c = '\0';                              // This is 1 char, denoted by ''
String s[] = "Test";                        // char s[] is a char array, String is an object; both have ""
int stat = 0;
int stat2 = 0;
unsigned long start_time = 0;
unsigned long before_flush = 0;
unsigned long after_flush = 0;


void setup() {
    Serial.begin(9600);                     // Starts USB serial
    Serial1.begin(9600, SERIAL_7E1);        // Starts TTL pins TX RX
}


void serialEvent1() {                       // Should run separately, but holds until loop
        while (Serial1.available() > 0) {
            c = Serial1.read();
            readBuf[readBufOffset] = c;
            readBufOffset++;
            
            if (c == '\0') {                // If character is nulll
                if (readBuf == '\0') {      // Compares to pointer, the 1st char in array
                    Serial.printlnf("Buf blank");
                }
                else {
                    Serial.printlnf("Buffer: %s", readBuf);  // Presuming something in buffer, prints it
                    readBufOffset = 0;      // Resets buffer counter
                }
                break;
            }
        }
}


void loop() {
        stat = Serial1.available();

        start_time = millis();
    
        // Serial.print(counter++);
        Serial.printf("Test#: %d", counter++);
        // Serial1.print(counter);
        Serial1.printf("Test#: %d", counter);
        
        Serial.printf("  Serial1 stat: %d", stat);
        Serial.printf("  Start time: %d", start_time);

        stat2 = Serial.available();        
        before_flush = millis();
        Serial.flush();
        after_flush = millis();
        
        Serial.printf("  Serial1 stat2: %d", stat2);
        Serial.printf("  Before flush: %d", before_flush);
        Serial.printlnf("  After flush: %d", after_flush);
    
    delay(4000);
    // Particle.process();
}

EXAMPLE OUTPUT:
Buffer:
Test#: 1 Serial1 stat: 6 Start time: 8772 Serial1 stat2: 0 Before flush: 8773 After flush: 8773
Buffer:
Test#: 2 Serial1 stat: 8 Start time: 12775 Serial1 stat2: 0 Before flush: 12775 After flush: 12776
Buffer:
Test#: 3 Serial1 stat: 10 Start time: 16777 Serial1 stat2: 0 Before flush: 16777 After flush: 16778
Buffer:
Test#: 4 Serial1 stat: 12 Start time: 20779 Serial1 stat2: 0 Before flush: 20779 After flush: 20779
Buffer:
Test#: 5 Serial1 stat: 14 Start time: 24780 Serial1 stat2: 0 Before flush: 24780 After flush: 24781
Buffer:
Test#: 6 Serial1 stat: 16 Start time: 28782 Serial1 stat2: 0 Before flush: 28782 After flush: 28782
Buffer:
Test#: 7 Serial1 stat: 19 Start time: 32783 Serial1 stat2: 0 Before flush: 32783 After flush: 32783
Buffer:
Test#: 8 Serial1 stat: 22 Start time: 36784 Serial1 stat2: 0 Before flush: 36784 After flush: 36784
Buffer: 
Test#: 9 Serial1 stat: 22 Start time: 40785 Serial1 stat2: 0 Before flush: 40785 After flush: 40785
Buffer: 8
Test#: 10 Serial1 stat: 21 Start time: 44786 Serial1 stat2: 0 Before flush: 44786 After flush: 44786
Buffer: 4
Test#: 11 Serial1 stat: 20 Start time: 48787 Serial1 stat2: 0 Before flush: 48787 After flush: 48787
Buffer: tc
Test#: 12 Serial1 stat: 19 Start time: 52789 Serial1 stat2: 0 Before flush: 52789 After flush: 52789
Buffer: t~
Test#: 13 Serial1 stat: 18 Start time: 56791 Serial1 stat2: 0 Before flush: 56791 After flush: 56791
Buffer: VZ{@e
Test#: 14 Serial1 stat: 14 Start time: 60793 Serial1 stat2: 0 Before flush: 60793 After flush: 60794
Buffer: t:
Test#: 15 Serial1 stat: 14 Start time: 64795 Serial1 stat2: 0 Before flush: 64795 After flush: 64796
Buffer:
Test#: 16 Serial1 stat: 15 Start time: 68797 Serial1 stat2: 0 Before flush: 68797 After flush: 68797
Buffer: t
Test#: 17 Serial1 stat: 18 Start time: 72798 Serial1 stat2: 0 Before flush: 72798 After flush: 72798
Buffer: V
Test#: 18 Serial1 stat: 21 Start time: 76799 Serial1 stat2: 0 Before flush: 76799 After flush: 76799

Notes:

  • I borrowed program structure from the “serial tutorial”
  • I specifically put “Serial.flush()” and “delay(4000)” in “void loop()” to slow everything down, and block the next iteration until the current finishes
  • I’m monitoring “Serial1.available()” along the way (I understand it returns # chars in buffer)

Are you sure you need SERIAL_7E1 here?

Thanks for your quick reply above!

Yea so the hardware I plan to integrate will most likely need this, but for giggles stand by and I’ll run without it and report back…

EDIT: I still see occasional errors, but appears Serial1 buffer count is now zero (“Serial1 stat: 0”). And not sure I can get around the “7E1” requirement.

OUTPUT:
Test#: 85 Serial1 stat: 0 Start time: 344860 Serial1 stat2: 0 Before flush: 344860 After flush: 344860
Test#: 86 Serial1 stat: 0 Start time: 348862 Serial1 stat2: 0 Before flush: 348862 After flush: 348862
Buffer:
Test#: 87 Serial1 stat: 0 Start time: 352864 Serial1 stat2: 0 Before flush: 352864 After flush: 352865
Test#: 88 Serial1 stat: 0 Start time: 356866 Serial1 stat2: 0 Before flush: 356866 After flush: 356867
Test#: 89 Serial1 stat: 0 Start time: 360868 Serial1 stat2: 0 Before flush: 360868 After flush: 360869
Test#: 90 Serial1 stat: 0 Start time: 364870 Serial1 stat2: 0 Before flush: 364870 After flush: 364870
Test#: 91 Serial1 stat: 0 Start time: 368871 Serial1 stat2: 0 Before flush: 368871 After flush: 368871
Test#: 92 Serial1 stat: 0 Start time: 372872 Serial1 stat2: 0 Before flush: 372872 After flush: 372872
Test#: 93 Serial1 stat: 0 Start time: 376873 Serial1 stat2: 0 Before flush: 376873 After flush: 376873
Test#: 94 Serial1 stat: 0 Start time: 380874 Serial1 stat2: 0 Before flush: 380874 After flush: 380874
Buffer: X.��
Test#: 95 Serial1 stat: 0 Start time: 384875 Serial1 stat2: 0 Before flush: 384875 After flush: 384875
Test#: 96 Serial1 stat: 0 Start time: 388876 Serial1 stat2: 0 Before flush: 388876 After flush: 388876
Buffer: ʲ�
Test#: 97 Serial1 stat: 0 Start time: 392878 Serial1 stat2: 0 Before flush: 392878 After flush: 392878
Test#: 98 Serial1 stat: 0 Start time: 396880 Serial1 stat2: 0 Before flush: 396880 After flush: 396881
Test#: 99 Serial1 stat: 0 Start time: 400882 Serial1 stat2: 0 Before flush: 400882 After flush: 400883
Buffer:
Test#: 100 Serial1 stat: 0 Start time: 404884 Serial1 stat2: 0 Before flush: 404884 After flush: 404885

Not addressing your actual issue, but if you happen to not receive a \0 character within the 64 byte since last, you’ll overflow your buffer and cause memory corruption.

Roger, thank you. Suppose the most robust way is if(c == ‘\0’ || c == ‘\r’ || c == ‘\n’) after the Serial1.read()?

So overflow the buffer means it’ll keep adding bogus chars? I thought the while (Serial1.available() > 0) counts characters and returns 0 if none, thus skipping the loop (Serial1.read()).

And guessing memory corruption is solved by reflashing the app or worst case “particle device doctor” via CLI (and corruption happens when you read or write past the memory space allocated for an array).

Actually the only really robust way would be to actually constrain the index readBufOffset to the allowed range of 0 through to 63.

Not quite that severe. The mentioned memory corruption only occures in RAM and hence no reflash would be required. A simple reset should cure it - or often enough the corruption alone might cause an SOS panic crash with follwoing reset.

That's true, but as you are encountering "stray" characters - maybe due to EMI - Serial1.available() might actually find some characters in the RX buffer which might be read on subsequent visits to SerialEvent1() resulting in a "buildup" in your buffer wich eventually might overflow it.

1 Like

You know I wondered about EMI - I thought about jumpering Tx, Rx and rerunning.

Let me at least try this and report back.

P.S. - This forum is amazing, and the markup, quote, etc. features are great.

2 Likes

Call me crazy but I think it’s self-induced EMI (I’m getting stray characters, usually those being sent) when Tx, Rx pins are free floating. Specifically the Serial1.printf() command (in void loop()) is still running as the “while” in serialEvent1() runs a number of times (ie, Photon runs much faster than serial transmission!).

NEW CODE:
(I tried to better status program steps)

#include "Particle.h"

// Constants
const size_t READ_BUF_SIZE = 64;
const unsigned long SEND_INTERVAL_MS = 2000;

// Forward declarations
// Put function definitions here

// Global variables
char readBuf[READ_BUF_SIZE];
size_t readBufOffset = 0;
unsigned long lastSend = 0;

int counter = 0;                            // Can pre or post-increment
char c = '\0';                              // This is 1 char, denoted by ''
String s[] = "Test";                        // char s[] is a char array, String is an object; both have ""
int statr = 0;
int statp = 0;
unsigned long read_time = 0;
unsigned long start_time = 0;
unsigned long before_flush = 0;
unsigned long after_flush = 0;


void setup() {
    Serial.begin(9600);                     // Starts USB serial
    // Serial1.begin(9600, SERIAL_7E1);        // Starts TTL pins TX RX
    Serial1.begin(9600);
}


void serialEvent1() {                       // Should run separately, but holds until loop
        statr = Serial1.available();
        read_time = millis();
    
        Serial.printf("StatR: %d", statr);
        Serial.printf("  Read: %d", read_time);
    
    while (Serial1.available() > 0) {       // Should run until buffer empty
        // Prevent buffer overrun if readBufOffset already 63
        if (readBufOffset < READ_BUF_SIZE) {                        // (buffer 0 to 63) < 64
                c = Serial1.read();
                readBuf[readBufOffset] = c;
                readBufOffset++;

                if (c == '\r' || c == '\n') {                       // Checks if end of transmission
                    Serial.printlnf("  Final Msg: %s", readBuf);    // Prints buffer
                    readBufOffset = 0;                              // Resets buffer counter
                    break;
                }
                else if (Serial1.available() < 1) {
                    Serial.printlnf("  No end marker, Msg: %s", readBuf);
                    readBufOffset = 0;
                    break;
                }    
                else {
                    Serial.printf("  Dunno, interim Msg: %s", readBuf);
                    Serial.printlnf("  readBufOffset: %s", readBufOffset);
                    readBufOffset = 0;
                }
        }
        else {
            Serial.print("  readBuf overflow; printing, resetting");
            Serial.printlnf("  Msg: %s", readBuf);
            readBufOffset = 0;
            break;
        }
    }
}


void loop() {
    // if (Serial1.available() == -1) {
    //     Serial.print("Serial 1 = -1");
    // }
    // else if (Serial1.available() == 0) {
    //     Serial.print("Serial 1 = 0");
    // }
        start_time = millis();
        Serial.printf("Start: %d", start_time);
    
        // Serial.print(counter++);
        Serial.printf("  Test#: %d", counter++);
        // Serial1.print(counter);
        Serial1.printf("Test#: %d", counter);
        
        statp = Serial1.available();        
        before_flush = millis();
        Serial.flush();
        after_flush = millis();
        
        Serial.printf("  StatP: %d", statp);
        Serial.printf("  Before flush: %d", before_flush);
        Serial.printlnf("  After flush: %d", after_flush);
    
    // else {
    //     Serial.print("Dunno");
    // }    

    delay(4000);
    // Particle.process();
}

OUTPUT (new code, without “grounded” Tx, Rx pins):
StatR: 10 Read: 148551 Dunno, interim Msg: T readBufOffset:
Dunno, interim Msg: e readBufOffset:
Dunno, interim Msg: s readBufOffset:
Dunno, interim Msg: t readBufOffset:
Dunno, interim Msg: # readBufOffset:
Dunno, interim Msg: : readBufOffset:
Dunno, interim Msg: readBufOffset:
Dunno, interim Msg: 3 readBufOffset:
Dunno, interim Msg: � readBufOffset:
No end marker, Msg:
Start: 148554 Test#: 36 StatP: 0 Before flush: 148554 After flush: 148554
StatR: 6 Read: 152554 Dunno, interim Msg: readBufOffset:
Dunno, interim Msg: . readBufOffset:
Dunno, interim Msg: � readBufOffset:
Dunno, interim Msg: � readBufOffset:
Dunno, interim Msg: readBufOffset:
No end marker, Msg: �
Start: 152557 Test#: 37 StatP: 0 Before flush: 152557 After flush: 152557
StatR: 9 Read: 156558 Dunno, interim Msg: T readBufOffset:
Dunno, interim Msg: e readBufOffset:
Dunno, interim Msg: s readBufOffset:
Dunno, interim Msg: t readBufOffset:
Dunno, interim Msg: # readBufOffset:
Dunno, interim Msg: : readBufOffset:
Dunno, interim Msg: readBufOffset:
Dunno, interim Msg: 3 readBufOffset:
No end marker, Msg: 8
Start: 156560 Test#: 38 StatP: 0 Before flush: 156560 After flush: 156560

OUTPUT (new code, with “grounded” Tx, Rx pins):
Start: 12084837 Test#: 3019 StatP: 0 Before flush: 12084837 After flush: 12084837
Start: 12088838 Test#: 3020 StatP: 0 Before flush: 12088838 After flush: 12088838
Start: 12092839 Test#: 3021 StatP: 0 Before flush: 12092839 After flush: 12092839
Start: 12096840 Test#: 3022 StatP: 0 Before flush: 12096840 After flush: 12096840
Start: 12100841 Test#: 3023 StatP: 0 Before flush: 12100841 After flush: 12100841
Start: 12104842 Test#: 3024 StatP: 0 Before flush: 12104842 After flush: 12104842
Start: 12108844 Test#: 3025 StatP: 0 Before flush: 12108844 After flush: 12108844
Start: 12112846 Test#: 3026 StatP: 0 Before flush: 12112846 After flush: 12112846
Start: 12116848 Test#: 3027 StatP: 0 Before flush: 12116848 After flush: 12116848
Start: 12120850 Test#: 3028 StatP: 0 Before flush: 12120850 After flush: 12120850
Start: 12124852 Test#: 3029 StatP: 0 Before flush: 12124852 After flush: 12124852
Start: 12128854 Test#: 3030 StatP: 0 Before flush: 12128854 After flush: 12128855
Start: 12132856 Test#: 3031 StatP: 0 Before flush: 12132856 After flush: 12132857
Start: 12136858 Test#: 3032 StatP: 0 Before flush: 12136858 After flush: 12136859
Start: 12140860 Test#: 3033 StatP: 0 Before flush: 12140860 After flush: 12140861
Start: 12144862 Test#: 3034 StatP: 0 Before flush: 12144862 After flush: 12144862
Start: 12148863 Test#: 3035 StatP: 0 Before flush: 12148863 After flush: 12148863
Start: 12152864 Test#: 3036 StatP: 0 Before flush: 12152864 After flush: 12152864
Start: 12156865 Test#: 3037 StatP: 0 Before flush: 12156865 After flush: 12156865
Start: 12160866 Test#: 3038 StatP: 0 Before flush: 12160866 After flush: 12160866
Start: 12164867 Test#: 3039 StatP: 0 Before flush: 12164867 After flush: 12164867
Start: 12168868 Test#: 3040 StatP: 0 Before flush: 12168868 After flush: 12168868
Start: 12172869 Test#: 3041 StatP: 0 Before flush: 12172869 After flush: 12172869
Start: 12176870 Test#: 3042 StatP: 0 Before flush: 12176870 After flush: 12176870
Start: 12180871 Test#: 3043 StatP: 0 Before flush: 12180871 After flush: 12180871
Start: 12184872 Test#: 3044 StatP: 0 Before flush: 12184872 After flush: 12184872
Start: 12188873 Test#: 3045 StatP: 0 Before flush: 12188873 After flush: 12188873
Start: 12192874 Test#: 3046 StatP: 0 Before flush: 12192874 After flush: 12192874

I only need to make the 2x Photons talk now. :slight_smile:

I won't call you crazy.
I have reported a similar issue with half-duplex mode in the past
https://github.com/particle-iot/firmware/issues/1276

This shoule be resolved with 0.7.0, but I've never come round to testing it.
However, there might be similar gremlins at work.

Also, how long are your wires?

emcee,

Could you please post a photo of your test setup with particular attention to the serial loopback? When I have a chance I’ll throw my scope on and see if I can simulate this.

I occasionally get garbage characters, but I attribute those to buffer overflows/errors not EMI.

1 Like

ScruffR, LapSpokane, appreciate the feedback. I’ll definitely post a pic of my setup.

Wondering if it boils down to an interaction between things like board dielectric properties and serial buffer dB threshold(s) in 0.7.0 firmware …

emcdee,

I generally stick to the Occam’s Razor Corollary of troubleshooting, that testing one’s way up through the most mundane explanations to the more difficult is the best way to find the problem. :wink:

1 Like

Roger, I figured I did that by slowly rebuilding the code without taking online examples for granted. :slight_smile:

And as promised here are the pics -

  • Notice it's only 2 Photons (ignore both the ADC breakout board on the left and the RS-485 on the right, neither are hooked to in/out pins)
  • It's shown with Tx to Rx and Rx to Tx (white red wires)
  • At one point I used the pictured resistor and then the yellow wire (separately) as a self-talk jumper (Tx to Rx on 1 Photon, inspired by ScruffR)

Going forward with the RS-485 breakout board, do I need to control the RTS (ready to send) pin? I'd like to think the breakout board will happily do its half-duplex thing so long as it's powered up (I believe they tie DE, RE legs together internally), but didn't see for sure in the docs...

You need to share grounds between the two photons!

2 Likes

Strongly agree, with a caveat. Ground is common to the USB connector shield on the photon. As long as everything comes from a common USB bus, that could/should be OK. If they are different busses, ground potential may differ slightly. I would do what peekay123 recommends, but also connect Vin and disconnect one of the USB cables. Then power and ground come from a single point and no current loops through a noisy computer.

My biggest piece of advice will not sound nice. It is not meant to offend. It's just my experience and that of others.

Don't do development on breadboards, particularly comms and signal work. Just give them up. That may not be the issue here, but breadboards are simultaneously wonderful and the *^$@! devil. The contacts wear and get loose, there is all kinds of stray impedance in them, wires come loose unbeknowst to you, power leads pop out then ground out against expensive things and blow them up. Soldering is a skill you should have in any event. It takes longer to do, but nearly always results in less problems and easier troubleshooting. You can spend hours troubleshooting your circuit when it was a faulty breadboard.

I know those sweet breadboards came with your Photons from Particle and I realize that I'm spending your money, not mine, but do your development on soldered perfboard. Adafruit's Perma Prototos are excellent. They're not cheap, but the quality is good and its not easy to make them yourself for less through a board house. You can socket your photons so that they are not permanently soldered in.

Adafruit Perma-Proto Full-sized Breadboard PCB - 3 Pack! : ID 590 : $19.95 : Adafruit Industries, Unique & fun DIY electronics and kits

When I get a chance, I'll wire up a couple of Photons and see if there is anything untoward riding on RX and TX.

Wow, thanks for the notes folks!

Just to clarify I basically saw cross-talk on one Photon before self-talking (Tx, Rx pins open), but can easily see grounds interfering between 2 devices (and plan to use the ground = 3rd wire on RS-485 wire runs). And I wondered about the breadboards, in fact I have a precision soldering iron (w/ tips) and will put to use.

Else just need to confirm on the RTS pin functionality (whether I need to control or not).

Cheers, happy 4th!

You will need to manage RTS for sure. I spent a few weeks working with a similar RS485 board thinking the RTS would manage itself. Big mistake. Manage it. This will make things work. As serial tends to mock things up, you might want to include a checksum and use a packet approach.

Roger, thanks for the insight, might as well make it right first time. Let me know if you recommend any good reading in particular (packet approach). :slight_smile: