TCP Server and Client Example Socket Programs "D7-ON" please

How do you currently check for localIP? If it's still as shown above, you might want to try this instead.

  while(!WiFi.ready)    // in non-AUTOMATIC this might happen
    SPARK_WLAN_Loop();
  SPARK_WLAN_Loop();    // once more to allow for IP transfer from CC3000 to Core
  ip = WiFi.localIP();  // now it should be there

That was much better for getting the local IP. Not seeing any speed improvement from shutting down the cloud, either way gets about a command every 70 ms, so I think I will stay cloud connected.

I still think this can be made a lot faster, so I am going to look at the connection code for VoodooSpark. Even with telnet I am having trouble sending bytes. Not really sure how to send something like 0x01, 0x07, 0x01. If anyone has working VoodooSpark code I would be very interested in a link to it. Not much to go on on the VoodooSpark page at https://github.com/voodootikigod/voodoospark#getting-started

Everything makes sense until I get to the line that says

With the IP Address and TCP port information, use your favorite language or TCP client to connect to the device (even telnet will work) and send it the necessary BINARY protocol commands to trigger the desired API interactions as defined in our API Command guide.

Looks like I am continuing the voodoospark issues at https://community.spark.io/t/solved-voodoospark-setup-troubles/9091/14

There are a couple TCP server examples (in javascript and ruby) over here. They’re servers that run on the computer, not clients as you’re talking about, but just wanted to point to a simple reference example of using “your favorite language… to connect to the device”

1 Like

I sure wish I could get it going, looks very simple and a great Hello World program. Unfortunately when I try to install os

var os = require('os');

.

npm install os

It does not work, has this node package name been changed. Anyway the code made sense and the os package seemed to only grab the IP so I did manage to get some kind of connection, but only using port 8080, which then caused some HTTP over head. Anyway, I connected from the spark to the node server but 7h did not light D7. it did not work. I don’t think cloud9 or my service provider allows port 9000.

This connects to a node js server and returns the following

successfully connected
HTTP/1.1 200 OK
access-control-allow-origin: *
content-type: text/plain
content-length: 59
date: Sun, 17 May 2015 01:41:11 GMT

Thank You For The Cross-Domain AJAX Request:

                                            Method: GET

.

The .ino file is

TCPClient client;


char server[] = "my-server-url.com";


bool myUsbSerialDebugOn = true;      // set to false when not hooked up to USB


int connectToMyServer(String myNothing) {

  digitalWrite(D7, HIGH);
  if (client.connect(server, 80)) {
      client.write("GET / HTTP/1.1\r\n");
      client.write("Host: my-server-url.com\r\n");
      client.write("\r\n");
      digitalWrite(D7, LOW);
      if (myUsbSerialDebugOn){
         Serial.println("successfully connected");
       }
      return 1; // successfully connected
  } else {
     digitalWrite(D7, LOW);
     if (myUsbSerialDebugOn){
         Serial.println("failed to connect");
      }
     return -1; // failed to connect
  }

}

void setup() {

      pinMode(D7, OUTPUT);

      Spark.function("connect", connectToMyServer);
      digitalWrite(D7, HIGH);
      delay(25000);
      digitalWrite(D7, LOW);
      
      
    if (myUsbSerialDebugOn){
       Serial.begin(9600);             // Initalize the USB Serial port
       while(!Serial.available()) SPARK_WLAN_Loop();
       Serial.println("Hello Computer");
    }

}

void loop() {
  digitalWrite(D7, LOW);
  if (client.connected()) {
    if (client.available()) {
        digitalWrite(D7, HIGH);
        char myIncoming = client.read();

      
      if (myUsbSerialDebugOn){  
          Serial.print(myIncoming);
        // delay(2);
      }
      
      
      } else {
        //digitalWrite(D7, LOW);
      }
    }
}

The Node js file is

// Include our HTTP module.
var http = require( "http" );
 
 
// Create an HTTP server so that we can listen for, and respond to
// incoming HTTP requests. This requires a callback that can be used
// to handle each incoming request.
var server = http.createServer(
    function( request, response ){
 
 
        // When dealing with CORS (Cross-Origin Resource Sharing)
        // requests, the client should pass-through its origin (the
        // requesting domain). We should either echo that or use *
        // if the origin was not passed.
        var origin = (request.headers.origin || "*");
 
 
        // Check to see if this is a security check by the browser to
        // test the availability of the API for the client. If the
        // method is OPTIONS, the browser is check to see to see what
        // HTTP methods (and properties) have been granted to the
        // client.
        if (request.method.toUpperCase() === "OPTIONS"){
 
 
            // Echo back the Origin (calling domain) so that the
            // client is granted access to make subsequent requests
            // to the API.
            response.writeHead(
                "204",
                "No Content",
                {
                    "access-control-allow-origin": origin,
                    "access-control-allow-methods": "GET, POST, PUT, DELETE, OPTIONS",
                    "access-control-allow-headers": "content-type, accept",
                    "access-control-max-age": 10, // Seconds.
                    "content-length": 0
                }
            );
 
            // End the response - we're not sending back any content.
            return( response.end() );
 
 
        }
 
 
        // -------------------------------------------------- //
        // -------------------------------------------------- //
 
 
        // If we've gotten this far then the incoming request is for
        // our API. For this demo, we'll simply be grabbing the
        // request body and echoing it back to the client.
 
 
        // Create a variable to hold our incoming body. It may be
        // sent in chunks, so we'll need to build it up and then
        // use it once the request has been closed.
        var requestBodyBuffer = [];
 
        // Now, bind do the data chunks of the request. Since we are
        // in an event-loop (JavaScript), we can be confident that
        // none of these events have fired yet (??I think??).
        request.on(
            "data",
            function( chunk ){
 
                // Build up our buffer. This chunk of data has
                // already been decoded and turned into a string.
                requestBodyBuffer.push( chunk );
 
            }
        );
 
 
        // Once all of the request data has been posted to the
        // server, the request triggers an End event. At this point,
        // we'll know that our body buffer is full.
        request.on(
            "end",
            function(){
 
                // Flatten our body buffer to get the request content.
                var requestBody = requestBodyBuffer.join( "" );
 
                // Create a response body to echo back the incoming
                // request.
                var responseBody = (
                    "Thank You For The Cross-Domain AJAX Request:\n\n" +
                    "Method: " + request.method + "\n\n" +
                    requestBody
                );
 
                // Send the headers back. Notice that even though we
                // had our OPTIONS request at the top, we still need
                // echo back the ORIGIN in order for the request to
                // be processed on the client.
                response.writeHead(
                    "200",
                    "OK",
                    {
                        "access-control-allow-origin": origin,
                        "content-type": "text/plain",
                        "content-length": responseBody.length
                    }
                );
 
                // Close out the response.
                return( response.end( responseBody ) );
 
            }
        );
 
 
    }
);
 
 
// Bind the server to port 8080.
server.listen( process.env.PORT, process.env.IP );
 
 
// Debugging:
console.log( "Node.js listening on port " +  process.env.PORT);

Here is another socket node js server that sends back something interesting

// Require HTTP module (to start server) and Socket.IO
var http = require('http'), io = require('socket.io');

// Start the server at port 8080
var server = http.createServer(function(req, res){ 
	
	// Send HTML headers and message
	res.writeHead(200,{ 'Content-Type': 'text/html' }); 
	res.end('<h1>Hello Socket Lover!</h1>');
});
server.listen(Number(process.env.PORT),process.env.IP);
  
// Create a Socket.IO instance, passing it our server
var socket = io.listen(server);

// Add a connect listener
socket.on('connection', function(client){ 
	
	// Create periodical which ends a message to the client every 5 seconds
	var interval = setInterval(function() {
		client.send('This is a message from the server!  ' + new Date().getTime());
	},5000);
	
	// Success!  Now listen to messages to be received
	client.on('message',function(event){ 
		console.log('Received message from client!',event);
	});
	client.on('disconnect',function(){
		clearInterval(interval);
		console.log('Server has disconnected');
	});
	
});

The output on the Spark Core looks like

successfully connected
HTTP/1.1 200 OK
content-type: text/html
date: Sun, 17 May 2015 04:15:29 GMT
transfer-encoding: chunked

1c
<h1>Hello Socket Lover!</h1>
0

Just found this link which will help

So @ekbduffy @omarojo @chuank have you done anything lately with your websocket? I just found your thread. I have solved the whole not flashing issue with a 25 second delay at setup and the webpage IP visible issue using localstorage.

But cant keep a connection open. Will look at your thread.

Hi @rocksetta, I haven’t worked on the websockets client for a while. In our thread, you might have noticed that we had to make modifications to the generic websockets library to catch malformed JSON data specific to Spacebrew – there was a lot of data truncation happening due to buffer issues.

Essentially, connections can be opened (and maintained) with no issue.

What was problematic for us was that the buffer in TCPClient was constantly being overrun, causing SOS hard faults, and we had to kludge our way around this by finding ways to flush the buffer / reset the Core before the hard faults. I do not recall ws connections being closed unceremoniously – the spacebrew server (which sends out ‘heartbeat’ pings at regular intervals) never had issues with connection dropouts.

1 Like

This looks great. So I am going to try to get a node js server to generate a webpage that I can click a button that lights D7 on the Spark Core as a websocket client, a different button shuts off D7. As a websocket micro-controller Hello World program. Any help would be appreciated. I am fine with javascript not yet comfy with Node JS sockets.

I am making this so that I can teach high school kids how to make a webpage (Phonegap APP conntrolled) Rover (RC car hacked to work with the Spark Core). Kind of like my failed Kickstarter Shopping Drone

To re-cap:

Using the cloud we can get spark-webpage communication that averages about 1000ms.

see github and video at

Using the TCP server we can make a very easy webpage to spark controller that takes about 100 ms. Unfortunately this only works on the local network and will be a pain to get working fully on the internet.(Mac Address port forwarding), and very different at home, at School and using a cell phone hotspot. But still fast and maybe what I will use.

So I am now working on the Spark Core as a Websocket client. Hopefully this will get solved.

A webpage that has become very useful for testing Spark Functions is the following. It uses local storage to store your spark credentials and then you just change “my-main” to be your spark function name and enter parameters if needed. See image. the link is at:

http://rocksetta.com/spark-core-photon/spark-control-basic-web-page.html

Thanks @bko, good idea to use the core as a client to access a internet based server. This is a horribly bad hack that seems to work. Not fully finished but interesting how easy it is to trick the websocket protocol to keep the socket open using basic TCP.

Here is the serial output

successfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒A▒A▒A▒A▒A▒A▒A▒A▒A▒Asuccessfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B

Here is the .ino file to flash to the core

TCPClient client;


char server[] = "my-cloud9-server.c9.io";


bool myUsbSerialDebugOn = true;      // set to false when not hooked up to USB


int connectToMyServer(String myNothing) {

  digitalWrite(D7, HIGH);
  if (client.connect(server, 80)) {
      client.write("GET / HTTP/1.1\r\n");
      client.write("Host: my-cloud9-server.c9.io\r\n");
      client.write("Upgrade: websocket\r\n");
      client.write("Connection: Upgrade\r\n");
      client.write("Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n");
      client.write("Sec-WebSocket-Version: 13\r\n");
      client.write("\r\n");

      digitalWrite(D7, LOW);
      if (myUsbSerialDebugOn){
         Serial.println("successfully connected");
       }
      return 1; // successfully connected
  } else {
     digitalWrite(D7, LOW);
     if (myUsbSerialDebugOn){
         Serial.println("failed to connect");
      }
     return -1; // failed to connect
  }

}

void setup() {

      pinMode(D7, OUTPUT);

      Spark.function("connect", connectToMyServer);
      digitalWrite(D7, HIGH);
      delay(25000);
      digitalWrite(D7, LOW);
      
      
    if (myUsbSerialDebugOn){
       Serial.begin(9600);             // Initalize the USB Serial port
       while(!Serial.available()) SPARK_WLAN_Loop();
       Serial.println("Hello Computer");
    }

}

void loop() {
  digitalWrite(D7, LOW);
  if (client.connected()) {
    if (client.available()) {
       
        char myIncoming = client.read();
        if (myIncoming == 'A'){ digitalWrite(D7, HIGH);}
      
      if (myUsbSerialDebugOn){  
          Serial.print(myIncoming);
        // delay(2);
      }
      
      
      } else {
        //digitalWrite(D7, LOW);
      }
    }
}

And here is the node js file on a cloud 9 server https://c9.io

var WebSocketServer = require("ws").Server
var http = require("http")
var express = require("express")
var app = express()
var port = process.env.PORT || 5000

app.use(express.static(__dirname + "/"))

var server = http.createServer(app)
server.listen(port)

console.log("http server listening on %d", port)

var wss = new WebSocketServer({server: server})
console.log("websocket server created")


wss.on("connection", function(ws) {
  var id = setInterval(function() {
        console.log("send ping")
    //ws.send(JSON.stringify(new Date()), function() {  })
      ws.send("A",function() {  })     // "A" = blink D7, "B" = don't blink 
    
  }, 2000)

  console.log("websocket connection open " )

  

  ws.on("close", function() {
    console.log("websocket connection close")
    clearInterval(id)
  })
})

I still don’t get how to program sockets using node js. Seems really confusing how to find simple DOCS for node modules “ws” “http” “express” and I need to work on a web page controller instead of my 2 second interval timer. However, with this setup D7 blinks every 2 seconds. When I switch the send command to send something other than “A” D7 does not blink. It appears that the connection is staying live and the Core seems reasonably stable.

I think I am getting the main web page working. The code is very easy. Still have to test it on the spark Core

<script>

var host = location.origin.replace(/^http/, 'ws')
var ws = new WebSocket(host);
ws.onmessage = function (event) {
    document.getElementById('myDiv1').innerHTML += event.data + '<br>'; 
};

 ws.onconnection = function (event) {
    ws.send('something');
}; 


</script>

<input type=button value="send D7-ON = A" onclick="{
   ws.send('A')
}">...

<input type=button value="send D7-OFF = B" onclick="{
   ws.send('B')
}"><br><br><br><br>

<input type=button value="mouseover D7-ON = A" onMouseOver="{
   ws.send('A')
}">...

<input type=button value="mouseover D7-OFF = B" onMouseOver="{
   ws.send('B')
}"><br>


 Test with: <a href="https://www.websocket.org/echo.html">https://www.websocket.org/echo.html</a> 
   remember to use  <br><br>wss://my-cloud9-server.c9.io<br><br> in the box<br>

<h1>Sent every 4 s</h1>
<div id="myDiv1">  here<br>  </div>

Presently using a github site at

NOW I CAN TEACH ROBOTICS! It looks like my Spark Core WebSocket works (In all its hacked, insecure, un-encrypted glory), will have to try it on my Rover. I am getting speeds of about 10 ms per command. Actually difficult to estimate the speed, things seem to falter around 3 ms, anyway much better than my TCP local WiFi server ~100 ms or the general cloud server ~1000ms. So I am very happy.

P.S. @zachary or @peekay123 any word on perhaps sending a class set (15) Spark Cores or Photons. You know I am going to keep asking every time I spend weeks on solving a problem that others could use with SparkDevices :smile:

Just want to say that the DOCS at Particle.io are a heck of a lot better than the DOCS for Node JS, still haven’t found anything really good to explain how to use express, ws, net, socket.io or http. I just tried using about 30 different Node JS webSockets and cut and pasted until things worked. I bet someone who knows what they are doing could tidy up my code dramatically. I haven’t really worked on sending data from the spark to a webpage since the Cloud 1 second delay is fine for my needs. I just need to be able to SEND commands to the Spark Core with little latency, if I am going to be able to drive a Rover with any kind of control.

My new Github site is at https://github.com/hpssjellis/Particle-Spark-Core-Photon-Websocket-Hack

I think this threads is close to being finished. Thanks for the help @ekbduffy, @omarojo, @chuank, @ScruffR, @kennethlimcp, @Dave, @BDub, @efatsi and @bko.

4 Likes

That sounds great @rocksetta! May I suggest disabling the Particle cloud connection to get even more reliable websocket traffic (Spark.disconnect()) while your TCP ws connection is going. I found that to help, but ymmv. Your Cores will light up green when that happens, instead of breathing cyan.

If you do that, you’ll have to rig up either a small h/w switch to enable/disable Spark cloud connectivity (to allow OTA firmware updates), but that can also be done by sending that command via your ws connection.

Will be interested in your further tests, as you potentially queue more data sent out your ws socket. The problem @ekbduffy and I faced was despite the low latency of a local ws connection, when the data stream got too long (due to Spacebrew’s implementation of JSON formatted messages), the Cores went into the hard faults as described in our posts. It was a case of incompatibility between the Core’s limited-but-understandeable TCP buffer size and the expectations of the Spacebrew message implementation.

@rocksetta, you need to PM @Steph regarding the Cores/Photons for your class :smiley:

Good ideas @chuank. I can use a cloud based function to disable the cloud and a ws command to enable the cloud. I always give a 25 second delay minimum at the start of my programs to re-flash the core after I screw it up, strangely a 20 second delay is not always enough.

Hopefully I will not have any long data streams as I plan to stick with single character controls. Even analog write data can be chunked to A=0, B=50, C=100, D=150, E=200, F=255, which for controlling a small vehicle should be fine. Small letters can have different meaning, so that is enough control for my students.

P.S. I did try extracting the analog amount from a ws command such as C255, it should not be hard to do but it strangely did not work when I tried to use the number as an integer. The above idea works fine for me.

This was something I meant to open a topic for a while myself, but always made my way around that issue by either increasing a delay or waiting for a serial or push button input.
So I'd "highjack" your post - if you don't mind, since it might become handy for you too :wink: - to ask the pros like @peekay123 or @mdma, if there is a way to know when a OTA download is running (slowish magenta blink, before rapid blink while actually flashing)?
The thing with SEMI_AUTOMATIC is, that user code keeps running during the OTA download phase and hence any WiFi.off() or other command that disconnects the cloud will execute and mess up the OTA flashing attempt.
But if there was a way to know, that a download is going on, user code could take care for that.

I guess there are several threads where users do suffer from this, but might not be aware of it.
e.g. this locked thread, where the OP thinks his code was flashed, but never will be

The plan is that we will have system events that your firmware code can subscribe to so that you know when an OTA is about to start.

1 Like

Hi rocksetta,

Please tell me the software tool you used to capture this serial output

successfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒A▒A▒A▒A▒A▒A▒A▒A▒A▒Asuccessfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B

Many thanks

I feel your pain @sam_desd . I do not like the USB serial port communication, I prefer turning on and off D7 as proof of concept (I also think it screws up some programs). It is a pain to get going on windows someone needs to make a youtube video showing how to do this. I tend to brute force program, (try lots of things and see what works) and I was trying to get DFU mode working and got the windows drivers for serial installed. Still not sure what I did. The starting link is at

http://docs.particle.io/core/connect/#connecting-your-device-using-windows

and a thread is here

.
.

I tried to get a batch file working to install it on windows but got frustrated with it, but the github site has lots of information.

That is part of why I wrote my youtube IOT Rant

Anyway if you do get USB-serial working, a few extra steps are needed when using it. First connect your core by usb to your computer, on windows, it takes about 30 seconds for the computer to recognize that the driver is connected, the computer should beep. I load "device manager" so I can see which COM port it works on. Then load PUTTY.exe and enter your COM port and 9600.
Then hit the enter key. You should see "Hello Computer"

Many CRobot programs use some way to switch back and forth from this mode.

#define DEBUG = 1  // debug mode on

I find that confusing so I use

bool myUsbSerialDebugOn = true;      // set to false when not hooked up to USB

Then if using windows the Particle Core needs to wait for you to push the enter key so it uses the code

if (myUsbSerialDebugOn){
   Serial.begin(9600);             // Initalize the USB Serial port
   while(!Serial.available()) SPARK_WLAN_Loop();
   Serial.println("Hello Computer");
}

Which continually repeats until you hit that key. Not sure what it does if you are not connected to the computer.

After that in my program I show the code you asked about by the following code

   char myIncoming = client.read();

  if (myUsbSerialDebugOn){  
      Serial.print(myIncoming);
    // delay(2);
  }

Note the commented out //delay(2); This is why I do not like serial, I never know if the serial delay needed to print the information to the screen is messing up the program. Once I had to just get rid of the serial and try it and the darn D7 lite, suggesting the serial was messing with the microcontroller.

I only used the serial in this program to find out what the heck was being sent back and forth to the node js server.

Good luck. I may make videos about DFU and Serial but it is low on my list of things to get ready for September 2015.

What is this supposed to do? ( I understand the connecToMyServer fx implementation, which is connecting as client to websocket i9 server...right?) but what does the assign to connect method really does?

I think you are referring to

void setup() {

      pinMode(D7, OUTPUT);

      Spark.function("connect", connectToMyServer);
      Spark.function("stop", stopMyServer);

which is in the github file at https://github.com/hpssjellis/Particle-Spark-Core-Photon-Websocket-Hack/blob/master/socket.ino which gets flashed to the core (or Photon).

Really good question, because as I look at my badly commented code, I can't see any way to actually activate these two functions. What I have probably done is forgotten to include my standard way of activating a function using my website at unofficial-Particle.io Photon (formerly spark-core)-teaching-website.html or more specifically the webpage at IoT Rant

Enter your ID and access_token and then just change the Function Name:--my-main to be either "connect" or "stop" to activate the server. I think I may have used this for testing and it is not needed once everything is actually going. Sorry about that. I will look into it and see if it is needed or not.