TCP or UDP issues consolidation

Troubleshooting
1

Hey community!

It seems like the discussion about TCP and UDP has been ongoing regular on discourse and I would love to spend some time looking at how the issue can be improved.

Disclaimer: I’m not at all an expert in networking stuff. Just trying to provide some help/testing :smile:

The Spark firmware team has quickly put together the new Ti patch 1.14 and this is untested and unverified to resolve any issues yet. The patch works and the core is able to connect successfully to the :cloud: after some host driver changes. That’s about it.

My aim is to consolidate a series of test cases that allows me to test the network before and after the patch. If you have any, or would love to help, drop a note here!

Please note that purely experimental for now and purely something i want to look into during my free time :wink:

Ping @bko @peekay123 @wgbartley @Hootie81 @psb777

3 Likes
2

Some Github issues:

3

I am by no means a TCP or UDP expert. I only know enough to be dangerous to myself and others around me. I can basically do plain-text, and that’s pretty much it.

However, I’m not afraid of using tcpdump or ngrep or netcat to help test, but only if someone can give me something to test!

1 Like
4

Hi,
My issues aren’t listed above and @bko has been helping me through testing but anyone else testing my code cant be a bad thing. Having the same problem with UDP and TCP getting pretty sure now that it’s not the code it’s either hardware or internet connection my end (I’m on a satallite link wifi network)
UDP:http://community.spark.io/t/udp-problems-with-simple-program-router-side-problem/8271/12
TCP:http://community.spark.io/t/tcp-libraries-unreliability/8231

Any more help is much appreciated.

1 Like
5

I took a quick look. Are you using a Spark core to send and another to receive?

6

Yeah at the moment, thanks for any help

7

@bko, do you know where i can find some non-working UDP code? :smiley: :smiley:

8

A set of test cases is a great idea. A number of simple ones need to be proposed. Each of the following to be done in the three modes. But the testing does rely upon understanding what TCP and UDP are supposed to do, and the differenced between them. E.g. Newly received UDP packets should flush out old ones if the receive buffer is full. Newly received TCP packets should be discarded if the receive buffer is full. UDP packet boundaries must be maintained but ordering is not, packets are allowed to be lost and/or duplicated. TCP packet boundaries are not maintained, the byte stream is presented to the user code in strict order, no duplcations, no losses.

UDP tests to be done in each of the 3 modes, MANUAL, SEMI and AUTO.

(1) UDP broadcast coded as plainly as possible. None of the known tricks employed. Sending 100 500char packets in quick succession. Note that UDP does not have flow control so we should expect to see packets lost. But we know this test will fail after only two or three packets because we won’t be reading the packets we have ourselves sent. And NORMAL mode won’t work at all because there is no cloud/no ping.

(2) Spark sends a UDP packet by one call to UDP.beginPacket(), several to UDP.write(), and one call to UDP.endPacket(). Only one datagram should be received on the recipient Linux / Windows / whatever box.

(3) Use one of the many utils (or write one on Linux, trivially, ask me) to send UDP datagrams variable lengths, Spark must respect the packet boundaries, one UDP.read must read one datagram only, even though the length is unknown. This test will not work.

(4) Test return codes. E.g. an error should be returned if a UDP packet greater than 576(?) chars is attempted to be sent - we know that the old fashioned limit is all we can expect to work.

And then we relax these tests, making these easier for Spark, step by step. E.g. variations on (1)

(1a) Send a packet only every 100ms. (1b) Send shorter packets. (1c) Set the local port to another, or read the socket to drain it, and to test it is being sent(!). (1d) Call ping first

etc etc

ping @ScruffR @Bdub @SomeFixItDude @Raldus @mdma

A search for UDP on the forum quickly gives these in the search results window:
Simple UDP program breaks the core
Unreliable UDP: crashes/freezes when sending at high frequency
UDP received DGRAM boundaries lost: read(), parsePacket(), available() all broken
UDP Broadcast problems with simple application
Strange UDP bug
Beehive Monitor (UDP, Sleep, Thermistor, WiFi antenna, ADC speed, RAM)
[Solved] UDP broadcast occasionally resets system
[Solved/Workaround] Spark Core can’t send UDP broadcast packets without cloud connection
UDP + red LED restart loop
UDP problems with #include "spark_disable_cloud.h"
UDP broadcast only works when connected to the cloud, why?
UDP problem’s with simple program. Router side problem?
I’m having a problem getting UDP traffic from my spark cores
UDP issues and workarounds

Numerous UDP bugs are consolidated here: https://community.spark.io/t/udp-issues-and-workarounds/4975

1 Like
11

Using simple test code, I managed to use the spark core as a UDP server and Mac as UDP client and vice versa.

Both showed reliability thus far in Automatic mode though.

Can ýou cook up a simple spark test code to test (3)?

Just note that these tests are before the new patch. I’m trying to find a repeatable case where it fails consistently.

1 Like
12

@kennethlimcp, what do you mean by reliability?

(A) Because UDP does not provide flow control one should expect occasional or even frequent packet loss if (1) many packets are sent in a short time and/or (2) the receiver is slow in reading the packets. Were this packet loss to occur then the UDP communications could not necessarily be said to be unreliable.

(B) UDP does not guarantee that packets are not duplicated, nor does it guarantee that packets are not re-ordered. If either of these things occur then UDP could not necessarily be said to be unreliable.

© What UDP guarantees is that if(!) the packet is received that it is received intact, correct and complete, and on its own. [Others have claimed here in the forum there is no checksumming on UDP packets, and if you dig into the arcane detail of the UDP specs you can see that it is possible not to set the checksumming flag. As far as I know all implemented versions of UDP do checksum the packets, and Berkeley sockets - upon which practically every implementation UDP is based: Python, Windows, Perl, Unix etc - checksums UDP.] If a packet is received and it is not what was sent, then UDP is unreliable. A received packet must correspond exactly to a sent packet. Otherwise UDP is unreliable.

So, what are you testing? :smile:

13

Test (3): Code can be found here http://community.spark.io/t/udp-received-dgram-boundaries-lost-read-parsepacket-available-all-broken/3800

This code sets up the Core as a slow reader of UDP packets. It delays 1000ms between reads. This allows the fast sender to send multiple packets. They should be queued up as separate packets on the Core but the UDP datagrams are instead concatenated together as if they are a TCP bytestream. It is impossible, on the Core, to say: give me the next packet only.

That this is a problem should be obvious when you recognise that a common set of apps for UDP have a central server receiving packets from many sources. Each must be dealt with separately and each requires a reply to each of the clients. On the Core the packets cannot be distinguished from one another, and only the sender of one of all the concatenated packets can be determined.

14

@kennethlimcp - I suggest we start with TCP, side-stepping discussions of protocol unreliability and the incessant hair-splitting that accompanies it.

The TCP case is particularly interesting how that the ACK/NAK responses have been pushed up to cc3000 host events, allowing applications to assist with flow control or increase buffers etc.

1 Like
15

@mdma, this is a large amount of functionality to test, and so one has to start somewhere, and if that is with TCP, so be it. But there really is no hair-splitting going on: UDP is as I describe it. And it is impossible for @kennethlimcp or anyone to test that UDP (or TCP) works if there is no functional definition against which to test. I am merely trying to correctly specify the function of UDP so that we will know it works, or does not.

I ought not to try to re-write what others have written better than I: From User Datagram Protocol - Wikipedia

Comparison of UDP and TCP

Transmission Control Protocol is a connection-oriented protocol, which means that it requires handshaking to set up end-to-end communications. Once a connection is set up, user data may be sent bi-directionally over the connection.

Reliable – TCP manages message acknowledgment, retransmission and timeout. Multiple attempts to deliver the message are made. If it gets lost along the way, the server will re-request the lost part. In TCP, there's either no missing data, or, in case of multiple timeouts, the connection is dropped.
Ordered – If two messages are sent over a connection in sequence, the first message will reach the receiving application first. When data segments arrive in the wrong order, TCP buffers delay the out-of-order data until all data can be properly re-ordered and delivered to the application.
Heavyweight – TCP requires three packets to set up a socket connection, before any user data can be sent. TCP handles reliability and congestion control.
Streaming – Data is read as a byte stream, no distinguishing indications are transmitted to signal message (segment) boundaries.

UDP is a simpler message-based connectionless protocol. Connectionless protocols do not set up a dedicated end-to-end connection. Communication is achieved by transmitting information in one direction from source to destination without verifying the readiness or state of the receiver. However, one primary benefit of UDP over TCP is the application to VoIP where latency and jitter are the primary concerns. It is assumed in VoIP UDP that the end users provide any necessary real time confirmation that the message has been received.

Unreliable – When a message is sent, it cannot be known if it will reach its destination; it could get lost along the way. There is no concept of acknowledgment, retransmission, or timeout.
Not ordered – If two messages are sent to the same recipient, the order in which they arrive cannot be predicted.
Lightweight – There is no ordering of messages, no tracking connections, etc. It is a small transport layer designed on top of IP.
Datagrams – Packets are sent individually and are checked for integrity only if they arrive. Packets have definite boundaries which are honored upon receipt, meaning a read operation at the receiver socket will yield an entire message as it was originally sent.
No congestion control – UDP itself does not avoid congestion, unless they implement congestion control measures at the application level.

1 Like
16

I’m performing an extremely simplistic test where the core behaves as a Server OR Client. Observing the output of the transmission to determine what’s the packet loss rate in a given time interval.

That’s what I’m doing to approach the problem as a total beginner who might want to use the TCP/UDP function and the kind of test case that I would personally run. :wink:

Once again, if you want to help, give me some example code and tell me what’s the use case and how I can determine the issue. That would help a beginner like me a lot!

Also, if you are keen to test, please do so! I’m happy to work with you as this is my own initiative during my limited free time :slight_smile:

I have yet to receive any test case up to this point in time except for doing a simple UDP server vs client transmission

17

Packet loss:

Packet loss in UDP does not signify a problem, necessarily. By definition of the protocol.

Packet counts in TCP cannot be used reliably as all you are guaranteed by TCP is that the bytestream arrives intact, in order, no losses, no duplications. But the number of received packets need not match the number of sent packets. Strange but true.

What signifies success in UDP is that those packets which do arrive, even if duplicated, must arrive intact and complete and accurate - a received datagram must be the same as one of the sent datagrams. Packets can be lost or duplicated or re-ordered, however.

What signifies success in TCP is that the bytestream received is in order and correct byte for byte. Number of received packets is unimportant.

18
19

@psb777,

not sure if i’m testing it correctly but i observe that UDP is observing the rule of one datagram and not joining them together.

I have a code running on the core:

UDP udp;

unsigned char buf[50];


void setup() {
    for(int i = 0; i < 50; i++)
      buf[i] = 0x00;
    udp.begin(remotePort);
    Serial.begin(9600);
}


void loop() {
   if(udp.parsePacket() > 0){
      udp.read(buf,50);
      for(int x = 0;x < 50; x++){
        Serial.write(char(buf[x]));
        buf[x] = 0x00;
      }
      Serial.println();
      }
}

The result is as follows sending a UDP packet of length > 50 (12345678901234567890123456789012345678901234567890AAAAAAAAAAAAAAAAAAAAAA)

12345678901234567890123456789012345678901234567890
AAAAAAAAAAAAAAAAAAAAAA............................
12345678901234567890123456789012345678901234567890
AAAAAAAAAAAAAAAAAAAAAA............................
12345678901234567890123456789012345678901234567890
AAAAAAAAAAAAAAAAAAAAAA............................
12345678901234567890123456789012345678901234567890
AAAAAAAAAAAAAAAAAAAAAA............................

I see no data behind the As during the 2nd time the udp buffer is read while the server is still transmitting.

Sending UDP packet of much longer length of 100+ showed the rule being obeyed as well.

Length of 200+ seemed pretty acceptable with no data from the 2nd datagram joined to the 1st.

This test code seems fairly decent IMO for UDP basic testing and i’ll switch over to TCP testing tomorrow and see what i can observe :smiley:

@bko please critic on my code and what changes should i make to have a more fair and comprehensive test :wink:

20

What is the content of the sent packets? Is it “1234567890” or is it 5 of those, 50 chars?

If you read your UDP packets as quickly as they are sent (or more quickly) then you will never see UDP packet concatenation. What you need to do is slow down so that there is time for two or more packets to arrive before you loop around to read another. I suggest that the last line of your loop() function should be delay(1000);. This should show the problem, if it still exists. Of course, the sender must be unconstrained to speed (or less constrained). I suggest you set it up to send 1 packet every 250ms. Or that you let it run full speed, but then I think you’ll possibly get buffer overflow which may mask the problem we’re trying to detect.

Also, you may wish to send smaller packets than the length you read - otherwise how will you ever see packet concatenation?

And, you should read the number of characters specified by parsePacket(). That will show that parsePacket is incorrectly returning the no of chars in the UDP buffer, not the no of chars in the 1st packet in the buffer.

You can also try reading UDP.available() chars but that too incorrectly returns the length of all the packets in the UDP buffer, not the length of the 1st packet.

So, try this, the sender sending small packets, quickly, or one pkt every 250ms, sorry cannot test for compilation errors right now:

UDP udp;

unsigned char buf[1000];

void setup() {
    udp.begin(remotePort);
    Serial.begin(9600);
}

void loop() {
   int pktlen, readlen;
   pktlen = udp.parsePacket();

   if(pktlen > 0){
      Serial.write("packet length:");
      Serial.write(pktlen);
      Serial.println();
      readlen = udp.read(buf,pktlen);
      Serial.write("read length:");
      Serial.write(readlen);
      Serial.println();
      for(int x = 0;x < readlen; x++){
         Serial.write(char(buf[x]));
      }
      Serial.println();
   }
   delay(1000);
}
21

I’m trying to test one parameter at one time so checking a few at one go might not be that appropriate.

Following your suggestion, i modified my code to do delay(3000) each loop and send 1234567890 at 1 second intervals.

The result seems fine except that the core gets knocked offline probably due to buffer overflow:

1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................
1234567890........................................

A more relevant testing to what you mentioned is sending a packet of length greater that your buffer at a faster interval than you read it will reveal the issue.

Sending 12345678901234567890123456789012345678901234567890 ABCDEFGHIJKLMNOPQRSTUVWXYZ that is length 76 with a buffer of only 50 results in:

12345678901234567890123456789012345678901234567890
ABCDEFGHIJKLMNOPQRSTUVWXYZ........................
12345678901234567890123456789012345678901234567890
ABCDEFGHIJKLMNOPQRSTUVWXYZ........................

This shows that the next packet is not being read incorrectly to overwrite any data unread in the previous loop. Sounds like a test case with the new patch :smiley:

Using your example, i managed to read the correct length each time for the packet i sent and output all the data in 1 datagram per loop printout

22

I’m unsure I entirely understand what you have written above, and I cannot see your code, but it seems perhaps the latest TI patch has made a significant improvement to the UDP functionality. If so that will be very very welcome. But I would be very interested in you running my code, so we can see what parsePacket() returns.

Where can I find the latest TI CC3000 firmware so I can try it myself?