# TCP Client not reading anything

**URL:** <https://community.particle.io/t/tcp-client-not-reading-anything/2427>\
**Category:** Troubleshooting\
**Created:** [January 28, 2014, 2:22am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427 "2014-01-28T02:22:08Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Iv4n](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/iv4n/32/1968_2.png) [@Iv4n](https://community.particle.io/u/Iv4n)\
**Post date:** [January 28, 2014, 2:22am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/1 "2014-01-28T02:22:08Z")

</div>

Hey guys,

I had a working code running in my Spark Core. The compile-server2 changes where rolled out and re flashed my spark core to change the brightness and update the I2C address of my LCD.

However, the TCP Client stopped working, I’m not able to read any returned http post or get. I have tried my old version which I posted online. [http://pastebin.com/7FXZ11QJ](http://pastebin.com/7FXZ11QJ) and also an example from @Coffee to get the date ([https://github.com/synox/spark-core-examples/blob/master/http\_timeapi.cpp](https://github.com/synox/spark-core-examples/blob/master/http_timeapi.cpp)). Note that I have tried with diferrent websites and it still doesn’t return anything.

Has anyone else experience the same issue? Can someone confirm it’s not happening to you? With a tested code example would be awesome.

---

<div class="post-metadata">

**Author:** ![Coffee](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/coffee/32/1989_2.png) [@Coffee](https://community.particle.io/u/Coffee)\
**Post date:** [January 28, 2014, 3:47am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/2 "2014-01-28T03:47:07Z")

</div>

I have the same problem, my code stopped working. See my other thread:

> **[TCPClient.available() does not return length](https://community.particle.io/t/tcpclient-available-does-not-return-length/1169/5)**
>
> look at the following example: https://gist.github.com/synox/b049a88bb533d7c3ec4c it works when using a delay(2000) before reading the response. without the delay it doesn’t work. any idea how to make the coder better? Can someone please post a...

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 28, 2014, 3:48am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/3 "2014-01-28T03:48:56Z")

</div>

I have this feeling with the [auto escaping of strings.](https://community.spark.io/t/bug-in-ide-using-json-in-string/2038/8)… something is broken within strings even though they compile. Can you try to monitor the request from the server side somehow and see if it it getting there, and if so if it’s botched at all?

Or spit your strings out on Serial to see what they are encoded as.

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 28, 2014, 4:10am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/4 "2014-01-28T04:10:47Z")

</div>

Ok guys, looks like the compiler is stripping out single backslashes from strings.

**EXAMPLE CODE:**

```cpp
void setup() {
    pinMode(D7,OUTPUT);
    digitalWrite(D7,HIGH);
    Serial.begin(9600);
    while(!Serial.available());
}

void loop() {
    Serial.println("HTTP/1.0 pass");
    Serial.println("http://www.domain.com/ pass");
    Serial.println("c:\directory\ fail");
    Serial.println("c:\\directory\\ pass");
    Serial.println("{haha} pass");
    Serial.println("\"abcdefg\" pass");
    delay(1000);
}

```

**OUTPUTS:**

```
HTTP/1.0 pass
http://www.domain.com/ pass
c:directory fail
c:\directory\ pass
{haha} pass
"abcdefg" pass

```

So until @Dave sees this, you can escape your backslashes like `"\\"`

---

<div class="post-metadata">

**Author:** ![jgoggins](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/jgoggins/32/20654_2.png) [@jgoggins](https://community.particle.io/u/jgoggins)\
**Post date:** [January 28, 2014, 5:00am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/5 "2014-01-28T05:00:26Z")

</div>

Thanks @BDub for distilling this

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 28, 2014, 5:18am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/6 "2014-01-28T05:18:44Z")

</div>

No problem, only thing is I don’t think @Iv4n 's code has any important uses of backslash that might make it fail.

One thing I would do @Iv4n instead of client.read(); is something like this:

```cpp
// wait 5 seconds or less for the server to respond
uint32_t startTime = millis();
while(!client.available() && (millis() - startTime) < 5000); 

```

I’m going to keep looking for the issue… brb

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [January 28, 2014, 5:23am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/7 "2014-01-28T05:23:59Z")

</div>

I am also having trouble that started tonight (before 2014-01-28T00:17:52-05:00). My problem is not directly related to backslash, but to other quoted chars. I have been successfully parsing an html string that is structured as field=“value” and my parser is suddenly failing. I think this is related to my using strtok to look for the double quote char in a string as “”". This worked before but now fails in a way I don’t understand yet.

The preprocessor giveth and the preprocessor taketh away, I guess. 🙂

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [January 28, 2014, 6:00am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/8 "2014-01-28T06:00:35Z")

</div>

I replaced my web page parser with just:

> void serialEvent() {  
> char inChar = myTCP.read();  
> Serial.print(inChar);  
> }

In my loop(), I do:

> while (myTCP.available()) {  
> serialEvent();  
> }

And I do not see the full contents of the web page like I used to. The first 604 chars are completely missing. I get 504 chars, with some missing in between and then nothing. The chars I do get are mostly good, except for some missing areas. And it consistently does the same thing over and over.

Using curl and wc, I find that the web page I am fetching is 2767 bytes long, so I am only getting a small part of the page now.

Did anyone see a recent change to TCP buffer management that might be causing this?

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 28, 2014, 7:53am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/9 "2014-01-28T07:53:12Z")

</div>

I’m also seeing issues with TCP Client reads… if anyone can run this example and tell me what I’m doing wrong I’d appreciate it. I’m basically missing chunks of data, and I’m waiting plenty of time to get further chunks from the server. I’ve even increased the timeout in http\_get() to 5 seconds and got the same response.

NOTE: When you run this you have to have your serial monitor closed, wait for the blue LED to come on, then open your monitor and press enter. It will pull data down from [http://graph.facebook.com/sparkdevices](http://graph.facebook.com/sparkdevices) and then timeout, and return to the main loop where it flashes the led until the next 10 second mark, and repeats the request. One thing to note client.connected() is always true until we call client.stop() so I didn’t use that for a timeout… had to use an actual timer. In the future client.connected() should be show the socket closed when the server terminates the connection, which should be after all of the data is sent.

I’m also seeing weird things with Serial TX… I need to put delays in there after large amounts of data or I don’t seem to see the data appear in the serial monitor.

```cpp
#include "application.h"

TCPClient client;

String http_get(char const * hostname, String path) {

  if (client.connect(hostname, 80)) {
    client.print("GET ");
    client.print(path);
    client.print(" HTTP/1.1\r\n");
    client.print("HOST: ");
    client.println(hostname);
    client.print("\r\n\r\n\r\n");
  } else {
    Serial.println("\r\n\r\nConnection Failed!");
    client.stop();
    return NULL;
  }

  int i = 0;
  Serial.println("\r\n\r\n\r\nReading Facebook Data......");
  uint32_t lastRead = millis();
  while ((millis() - lastRead) < 1000) {
    while (client.available()) {
      char c = client.read();
      Serial.print(c);
      if(i++ > 50) {
        delay(100);
        i = 0;
      }
      lastRead = millis();
    }
  }
  client.flush();
  client.stop();
  return "";
}

void setup() {
  pinMode(D7, OUTPUT);
  digitalWrite(D7, HIGH);
  Serial.begin(115200);
  while (!Serial.available()); // After Core D7 LED turns on, open serial monitor and press enter!
}

uint32_t nextTime = millis(); // next time to contact the server
int count = 19;
bool state = 1;
void loop() {
  if (millis() > nextTime) {
    nextTime = millis() + 500UL;
    if (++count == 20) {
      count = 0;
      http_get("graph.facebook.com", "/sparkdevices");
    } else { // toggle the led while we wait for 10 second mark
      state = !state;
      digitalWrite(D7, state);
    }
  }
}

```

This is what I get:

```
Reading Facebook Data......
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: private, no-cache, no-store, must-revalidate
Content-Type: text/javascript; charset=UTF-8
ETag: "38d35b9d9ec8dd27ed2ce90819473c873663d089"
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Pragma: no and pre-order the Spark Core and accessories at www.spark.io","category":"Computers\/technology","description":"Spark makes using and creating connected devices easy. The Spark Core an Arduino-compatible, Wi-Fi enabled development kit pairs with the Spark""},"products":"Spark Core, Spark Cloud","talking_about_count":395,"username":"sparkdevices","website":"http:\/\/www.spark.io","were_here_count":0,"id":"427120970660412","name":"Spark","link":"http:\/\/www.facebook
```

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [January 28, 2014, 12:33pm UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/10 "2014-01-28T12:33:10Z")

</div>

There is another [thread](https://community.spark.io/t/repeat-http-requests-to-server-help/938/10?u=bko) where there is similar discussion and a patch to TCPClient that has fixed the problem for some. I don’t have time to test this myself, but I thought you would want to know.

I hope that @zachary can get this fix applied quickly–I am dead in the water right now.

---

<div class="post-metadata">

**Author:** ![Iv4n](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/iv4n/32/1968_2.png) [@Iv4n](https://community.particle.io/u/Iv4n)\
**Post date:** [January 28, 2014, 2:21pm UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/11 "2014-01-28T14:21:01Z")

</div>

That’s great guys, thanks a lot for the fast feedback and workarounds, in my case I was able to fix the code looping until available() returned something:

```
String http_get(char const* hostname, String path) {

    if (client.connect(hostname, 80)) {
            client.print("GET ");
            client.print(path);
            client.print(" HTTP/1.0\r\n");
            client.print("HOST: ");
            client.println(hostname);
            client.println();
            client.flush();
    } else {
            Serial.println("connection failed");
            client.stop();
            return NULL;
    }

    int bytes = 0;
    while(!bytes && client.connected()) {
	    bytes = client.available();
	    delay(200);
    }
	Serial.println("bytes:");
	Serial.println(bytes);

	for (unsigned int i = 0; i < bytes; i++) {
		char c = client.read();
			Serial.print(c);
		if (c == -1) {
		    	Serial.println("oops");
			break;
		}

		buffer[i] = c;
	
	}
	client.stop();

    String response(buffer);
    int bodyPos = response.indexOf("\r\n\r\n");
    if (bodyPos == -1) {
            Serial.println("can not find http get reponse body");
                            return NULL;
    }
    return response.substring(bodyPos+4);

```

}

B​Dub’s code seems more elegant as it doesn’t use delay that much and as he mentioned connected() always returns true which makes mine redundant. I’ll change to that if in the following weeks we don’t get the fixed merged into compile-server2.

---

<div class="post-metadata">

**Author:** ![kwyjib0](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/kwyjib0/32/916_2.png) [@kwyjib0](https://community.particle.io/u/kwyjib0)\
**Post date:** [January 28, 2014, 10:05pm UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/12 "2014-01-28T22:05:45Z")

</div>

@BDub not really answering but having a problem using your code.

I set up a web service that queries [http://openweathermap.org/](http://openweathermap.org/) and then maps temperature and conditions to RGB codes (I want to create a moodlight based on the weather.)

the web service url is: [http://roman-mueller.ch/api/weather](http://roman-mueller.ch/api/weather)

As you can see it returns something like this:

```
"W-[000,000,139][144,238,144] - Temp: -2,3°C; Color [DarkBlue] - Condition: SkyIsClear; Color [LightGreen]"

```

So when I use your code from above just using my url instead of [graph.facebook.com](http://graph.facebook.com) and /api/weather instead of /sparkdevices I get the following answer:

```
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET

```

Looks like I get all the header data but not my weather data… any idea?

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 28, 2014, 10:18pm UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/13 "2014-01-28T22:18:27Z")

</div>

Yeah I don't know why everyone is messing around with TCP Clients right when it's currently broken, but it's popping up a lot today! See my post and a temporary fix by weakset here:

> **[HTTP Request issue](https://community.particle.io/t/http-request-issue/2436/2)**
>
> Hi @Jeroen3, this is something I encounted yesterday as well… and the solution is here… https://github.com/spark/core-firmware/pull/67 If you build your code locally you can just replace those two edited files and it should work. I don’t think...

---

<div class="post-metadata">

**Author:** ![Dave](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/dave/32/181_2.png) [@Dave](https://community.particle.io/u/Dave)\
**Post date:** [January 28, 2014, 10:26pm UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/14 "2014-01-28T22:26:39Z")

</div>

Hey Guys,

Popping in to say I fixed some issues with the pre-processor stripping out stuff it shouldn’t. I’ve changed the approach it takes, so that shouldn’t be a side-effect anymore. Please keep sending me any code you find that doesn’t compile like you think it should, and we’ll add unit tests for it! 🙂

Thanks!  
David

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [January 29, 2014, 3:23am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/15 "2014-01-29T03:23:39Z")

</div>

I just want to report back a minor success. Remember in the early days we all noticed that TCPClient.available() was returning a boolean rather than a count of bytes? Well the recent changes fixed this and it must be related to the TCPClient problems.

My old code for the boolean case looked like this and extracted one byte per call of loop():

> ```
> while (myTCP.available()) {
> serialEvent();
> }
> 
> ```

This code stopped working sometime last night when the new TCPClient rolled out.

Now my code looks like this:

> ```
> int bytecount = myTCP.available();
> while (bytecount>0) {
> serialEvent();
> bytecount--;
> }
> 
> ```

And this works **much** better.

I thought I was being a good cooperative-multitasking doobie by giving control back to the TCPClient every byte, but I think it does not like that! Now if the available() method says there are N bytes available, I process them all at once and only then give control back by letting loop() go round again. Maybe this prevents premature buffer reuse in the TCPClient, I don't know.

But it does work much better now--maybe you guys can try it too.

---

<div class="post-metadata">

**Author:** ![zach](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/zach/32/20714_2.png) [@zach](https://community.particle.io/u/zach)\
**Post date:** [January 29, 2014, 4:45am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/16 "2014-01-29T04:45:09Z")

</div>

Thanks @bko for the report! Here are the commits for the recent changes to TCPClient:

> <https://github.com/particle-iot/device-os/commit/71da1f1c3ccc603da8791ba3f12cbc30d6c44161>

> <https://github.com/particle-iot/device-os/commit/79db884dee971e3ed018ae4c003881ebdfb361a2>

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 29, 2014, 5:02am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/17 "2014-01-29T05:02:14Z")

</div>

What does serialEvent() do? If you let it fall through your loop() over and over again, how do you know when you are done with the transfer?

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [January 29, 2014, 5:07am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/18 "2014-01-29T05:07:08Z")

</div>

Hi @zach, Thanks for the pointer to commits. Here is what I think is going on. The new code in TCPClient.available() checks the number of bytes received on line 131 and then sets the `_offset` to zero and the `_remaining` to the number of bytes. When TCPClient.read() is called, it checks if there are bytes remaining (`_remaining > 0` on line 154) and if so, it gets the byte incrementing the `_offset` pointer and decrementing the `_remaining` count.

The problem arrises if I call TCPClient.available(), then call TCPClient.read(), and then call TCPClient.available() again. In the first call to available, `_offset` and `_remaining` are set correctly. I then get one byte with the call to read() and `_offset` and `_remaining` are updated. But then when I call available() again, the current values of `_offset` and `_remaining` are ignored and overwritten with `_offset = 0` and `_remaining` = new bytes received.

I think the proper thing to do in TCPClient.available() is to **add** the number of new bytes received to the current value of `_remaining` and to only force `_offset` to zero when \_remaining is zero.

I would love to send you a pull request, but I don’t have a build sandbox setup here.

I hope this helps!

---

<div class="post-metadata">

**Author:** ![bko](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bko/32/20704_2.png) [@bko](https://community.particle.io/u/bko)\
**Post date:** [January 29, 2014, 5:15am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/19 "2014-01-29T05:15:00Z")

</div>

Hi @BDub,

My serialEvent() is my one-character-at-a-time state-machine HTML/XML parser. It starts off with:

```
void serialEvent() {
    char inChar = myTCP.read();
    if (tagFlag==false && dataFlag==false && inChar == startMatch[matchPtr]) {
        tagFlag = true; 
....

```

So it consumes one char from the TCPClient.read() for every call. With my changes for the TCPClient.available(), it is called in a while loop for as many bytes are available. In my old code, I called it one or more times per loop() when available was true, since parsing the web/rss feed doesn’t have to be done quickly.

The problem seems to be that if you call TCPClient.available() and it tells you there are N bytes, you need to call TCPClient.read() N times before you call TCPClient.available() again, or you will lose data.

---

<div class="post-metadata">

**Author:** ![BDub](https://sea2.discourse-cdn.com/flex026/user_avatar/community.particle.io/bdub/32/5137_2.png) [@BDub](https://community.particle.io/u/BDub)\
**Post date:** [January 29, 2014, 6:33am UTC](https://community.particle.io/t/tcp-client-not-reading-anything/2427/20 "2014-01-29T06:33:45Z")

</div>

@zach @satishgn I don’t pretend to understand the low level driver yet but I have a rough view of what select() and recv() do, but what is the theory behind making TCPClient.available() check for available bytes AND read them into a buffer, vs. letting TCPClient.read() do the actual reading from low level stuff?

One reason I thought might be because you thought buffering 512 straight away is faster, and then let users peck away at the data in read() and buffered read() at their leisure.

One issue I see is currently we are not automatically getting our socket to close when the server is done sending data, so TCPClient.connected() is always true unless we call TCPClient.stop(); But we don’t really know when the data is done… so we don’t really know if we should try to close the socket ourselves or not. That said, there is a way to know when the server closes the socket… the recv() method will return -57 when it happens. I guess some things can be done in either as long as you’re polling them both and it doesn’t matter… but it seems confusing when you go read the code.

Anyhoo… if the server socket closing was detected, \_sock could be set to SOCK\_MAX\_NUM and then TCPClient.connected() would be useful. Something like this:

```cpp
uint32_t lastRead = millis();
  while (client.connected() && (millis() - lastRead < 3000)) {
    while (client.available()) {
      char c = client.read();
      lastRead = millis();
    }
  }
  client.stop(); // technically this is redundant if we don't timeout of the main while loop

```

This is pretty much necessary for chunked transfers, so you’ll know to keep waiting because there is another chunk possibly coming.

[Next page](https://community.particle.io/t/tcp-client-not-reading-anything/2427.md?page=2)