Not sure if general was the right category but here goes.
I know in the documentation it specifics that the internal serial buffer (Or Rx buffer ?) can hold a maximum of 64 bytes. So if I was communicating between two Cores using Serial1 (Tx Rx), and from one Core I was using Serial1.write() to send over 62 bytes, there is now 62 bytes in the Rx buffer right? But now with the other Core if I was using:
while (Serial1.available()) { Serial1.read(); }
Wouldn’t the # of bytes then reduce to 61, 60, 59 etc. for every iteration of the while loop? That way if I send more bytes over, while this loop is going, it will continue to read so long as their are bytes to read. This is how I understand the Rx buffer to work. If I’m wrong can anyone please clarify?
Because as it stands if I send 62 bytes over Serial1 and use the above loop to read from the Rx buffer I can only read 61 of those bytes! I’ve reproduced this several times and am positive that 61 bytes are being sent over. Is there something I’m missing, doesn’t Serial1.read() remove the bytes from the buffer to make room for new incoming bytes?
@UST, the serial buffer is 64 bytes long and DOES clear out as bytes are sent via the serial port (1 or 2). If the buffer is full and a write is attempted, the firmware will wait until buffer space is available. The question is whether you are counting your bytes correctly going into the buffer. On the read side, the code looks right to read the corresponding number of bytes. Can you share your code for the sending side?
For sure ! I was just about to do a test firmware code eliminate any suspicions!
But this is how I’m doing it for now on the sending side :
/
/read sensor data continuously, including timestamp, date etc. which equals 62 bytes long
while(data.available())
{
char stuff= data;
Serial1.write(check);
// Serial.print(check); Used this to confirm "stuff" was working to get all my intended data
}
////////////// Then on the reading side /////////////////
String string = "";
while(Serial1.available())
{
char stuff= Serial1.read();
string += String(stuff);
Serial.print(stuff); // similar to just printing "string" after the loop is
//done, but just checking if everything is coming in
//is correct, but it cuts off bytes
}
I even tried something else to ensure all bytes were coming through, but just created a whole slew of issues
I should also mention that Serial1.begin is set to 115200 baud
the snippet above will digest every byte transmitted, but in practice, you aren't going to do that(or is is is less-likely). Unless you are talking about married couples, it is not that often that you want to completely disregard every communication that gets transmitted! You will likely be evaluating the byte or parsing the data into a C string which you will later evaluate/manipulate.
as @peekay123 pointed out, you want to post your code to see if folks can help out. Even better... explain what you are trying to do, even if is just the learning exercise).
@UST, as @bko pointed out, you must make sure Serial is set to 115Kbaud. Also, you are assuming your sending side is only sending 62 bytes but you have no counter. I would add a counter in the while() of the sending side and then print it out after the while is done to confirm your send count.
On the receive side, don’t use a string. Simply do a Serial.write(stuff). Add a counter on the receiving side as well to confirm your receive count.
In both send and receive code, make “stuff” a uint8_t since the Spark firmware defines “char” as a signed byte.
@bko I do have my Serial (USB) set to 115200 same as Serial1; didn’t realize Serial could go higher, should I set it higher?
@peekay123 Okay, I was just about to post my code as @BulldogLowell suggested (which I can still do?) but I was receiving similar errors to what I was describing. But I’ll take your suggestions in to see if it fixes the issue!
Also @peekay123 , on the sending side I was doing:
Serial.print(stuff);
Everytime it went through the loop, then just copying the what came up on PuTTY into a character counter - I can add a counter instead, which I guess is easier
@UST, Serial.print() and Serial.write() work a little differently. You should use Serial.write() to print out exactly what you send. Same applies on the receive side.
this will slow down the while loop, which isn’t a bad thing on the receiving end, if you want to trap a slug of bytes.
I’d try to put a little delay in here just to test that you don’t have another issue:
String string = "";
while(Serial1.available())
{
char stuff= Serial1.read();
delay(5); // slow this down and try to get the entire slug of data from the sender in one turn of the while loop
string += String(stuff);
Serial.print(stuff); // this is slowing the while loop, but maybe not enough
{
I’ve put a counter in before, 62 bytes are being sent, I’ve also tried slowing down the receive side BEFORE the while loop but it didn’t seem to help. However, I’ve created an exact reproducible scenario for my issue (code below) but I will in a few minutes try slowing it down WITHIN the while loop to see if that helps.
/*--SEND CODE--*/
void setup ()
{
Serial.begin(115200);
Serial1.begin(115200);
}
void loop()
{
send();
delay(5000);
}
void send()
{
Serial1.write("Today's date is 2015/02/05 10:10:00");
Serial1.write(',');
Serial1.write("1");
Serial1.write(',');
Serial1.write("2");
Serial1.write(',');
Serial1.write("3");
Serial1.write(',');
Serial1.write("data received A.O.K.");
Serial1.write("Do we get to here?"); //line never completes
}
Now Recieve Side, this ALWAYS prints everything up to “A.O.K.D” but doesn’t print “o we get to here?” Two different tests, first one had a weird result, second was the more predictable result, images below!
We do see 63 byte transmitted correctly and the implementation for USART serial does make use of % SERIAL_BUFFER_SIZE (which only allows for results 0…63) a lot.
I haven’t checked this thesis yet, but if the modulo gets used as a count somewhere in there, this might account for the missing 64th byte.
e.g.
int USARTSerial::available(void)
{
return (unsigned int)(SERIAL_BUFFER_SIZE + _rx_buffer.head - _rx_buffer.tail) % SERIAL_BUFFER_SIZE;
}
// and
inline void store_char(unsigned char c, Ring_Buffer *buffer)
{
unsigned i = (unsigned int)(buffer->head + 1) % SERIAL_BUFFER_SIZE;
if (i != buffer->tail)
{
buffer->buffer[buffer->head] = c;
buffer->head = i;
}
}
But the rest of the string(“o we get to here?”) isn’t accounted for, so it’s more than just the 64th byte right? This is why I originally thought that the buffer wasn’t updating or bytes weren’t being removed from it … I’m at a loss I’m still trying to look around to see if I can find anything on it
Edit: Or even why is it printing the rest of the string sometimes but not within the same line/function iteration.
The RX buffer is full and since there is no handshaking the sender wouldn't know how long to wait till the receiver has made space for more bytes.
And since the sender does not run out of TX buffer it'll just blast all the extra bytes into nirvana.
Edit: About this
If the async sending and receiving by chance happens at the same time you are lucky and the RX buffer gets emptied while it gets filled from the other side hence accepting more than 64 byte.
But normally it stops at the “A.O.K.D” … which seems like there is some type of “Hey you are full, I’ll wait” type of communication going on.
Other times it posts the rest of the string (as in the first PuTTY picture) but on a separate line (which tells me it was another function iteration)
After I put the “delay(5)” within the while loop as @BulldogLowell suggested, I get the overflow happening where it prints up to “A.O.K.D” then IMMEDIATELY prints the start of string again "Today’s date is … " but doesn’t even complete it …
@ScruffR you’re saying this is due to the RX buffer becoming full? Doesn’t Serial1.read() pull what’s in the buffer out, thus making more room? Also just saw your edit I’ll look at that code now. Saw you’re other edit @ScruffR, so the reciever is waiting for things to get full? So basically things aren’t happening at the same time.
@ScruffR, from what I see in spark_wiring_usartserial.cpp, the write() function will wait for the serial queue to have room before adding new bytes to it:
// If the output buffer is full, there’s nothing for it other than to
// wait for the interrupt handler to empty it a bit
// no space so or Called Off Panic with interrupt off get the message out!
// make space Enter Polled IO mode
@UST, you should slow down the send side as well to allow the send queue to empty.
Unless you have only one Core being sender and its own receiver you are facing an async setup which does get a bit tricky to figure out especially since there are things going on in the background.
@peekay123, yes this is true but I think only for the TX buffer, but the TX Core has no understanding of the fill state of the RX buffer of the other Core - or am I wrong here?
@peekay123, by adding a 10ms delay on both the send and recieve side, these are the results I get:
Where every 6 iterations, it seems to reprint the same sent string, sometimes missing characters, sometimes adding the ones that should be following.
@ScruffR, I didn’t know the technically reasoning for it, but this was my original-original thought, that the two Cores were not synced up correctly. As in the receiving/reading core was reading faster than the other could send, causing it to cut off But then when I realized .read() should be taking bytes out of the buffer, I figured as long as one Core is writing as the other Core is reading then it should be okay, cause the buffer shouldn’t overflow … but now I don’t know.
@ScruffR, neither side is aware of the other. However, from what I can see, the receive firmware uses a ring buffer and it will drop received characters if the buffer is full!
@UST, in your test, you need to pull the receive data as fast as possible but delay between chunks of sent data to allow the receiver to service its buffer.
- I can add a counter instead, which I guess is easier
