TCP Server and Client Example Socket Programs "D7-ON" please

Hi @rocksetta, I haven’t worked on the websockets client for a while. In our thread, you might have noticed that we had to make modifications to the generic websockets library to catch malformed JSON data specific to Spacebrew – there was a lot of data truncation happening due to buffer issues.

Essentially, connections can be opened (and maintained) with no issue.

What was problematic for us was that the buffer in TCPClient was constantly being overrun, causing SOS hard faults, and we had to kludge our way around this by finding ways to flush the buffer / reset the Core before the hard faults. I do not recall ws connections being closed unceremoniously – the spacebrew server (which sends out ‘heartbeat’ pings at regular intervals) never had issues with connection dropouts.

1 Like

This looks great. So I am going to try to get a node js server to generate a webpage that I can click a button that lights D7 on the Spark Core as a websocket client, a different button shuts off D7. As a websocket micro-controller Hello World program. Any help would be appreciated. I am fine with javascript not yet comfy with Node JS sockets.

I am making this so that I can teach high school kids how to make a webpage (Phonegap APP conntrolled) Rover (RC car hacked to work with the Spark Core). Kind of like my failed Kickstarter Shopping Drone

To re-cap:

Using the cloud we can get spark-webpage communication that averages about 1000ms.

see github and video at

Using the TCP server we can make a very easy webpage to spark controller that takes about 100 ms. Unfortunately this only works on the local network and will be a pain to get working fully on the internet.(Mac Address port forwarding), and very different at home, at School and using a cell phone hotspot. But still fast and maybe what I will use.

So I am now working on the Spark Core as a Websocket client. Hopefully this will get solved.

A webpage that has become very useful for testing Spark Functions is the following. It uses local storage to store your spark credentials and then you just change “my-main” to be your spark function name and enter parameters if needed. See image. the link is at:

http://rocksetta.com/spark-core-photon/spark-control-basic-web-page.html

Thanks @bko, good idea to use the core as a client to access a internet based server. This is a horribly bad hack that seems to work. Not fully finished but interesting how easy it is to trick the websocket protocol to keep the socket open using basic TCP.

Here is the serial output

successfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒A▒A▒A▒A▒A▒A▒A▒A▒A▒Asuccessfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B

Here is the .ino file to flash to the core

TCPClient client;


char server[] = "my-cloud9-server.c9.io";


bool myUsbSerialDebugOn = true;      // set to false when not hooked up to USB


int connectToMyServer(String myNothing) {

  digitalWrite(D7, HIGH);
  if (client.connect(server, 80)) {
      client.write("GET / HTTP/1.1\r\n");
      client.write("Host: my-cloud9-server.c9.io\r\n");
      client.write("Upgrade: websocket\r\n");
      client.write("Connection: Upgrade\r\n");
      client.write("Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n");
      client.write("Sec-WebSocket-Version: 13\r\n");
      client.write("\r\n");

      digitalWrite(D7, LOW);
      if (myUsbSerialDebugOn){
         Serial.println("successfully connected");
       }
      return 1; // successfully connected
  } else {
     digitalWrite(D7, LOW);
     if (myUsbSerialDebugOn){
         Serial.println("failed to connect");
      }
     return -1; // failed to connect
  }

}

void setup() {

      pinMode(D7, OUTPUT);

      Spark.function("connect", connectToMyServer);
      digitalWrite(D7, HIGH);
      delay(25000);
      digitalWrite(D7, LOW);
      
      
    if (myUsbSerialDebugOn){
       Serial.begin(9600);             // Initalize the USB Serial port
       while(!Serial.available()) SPARK_WLAN_Loop();
       Serial.println("Hello Computer");
    }

}

void loop() {
  digitalWrite(D7, LOW);
  if (client.connected()) {
    if (client.available()) {
       
        char myIncoming = client.read();
        if (myIncoming == 'A'){ digitalWrite(D7, HIGH);}
      
      if (myUsbSerialDebugOn){  
          Serial.print(myIncoming);
        // delay(2);
      }
      
      
      } else {
        //digitalWrite(D7, LOW);
      }
    }
}

And here is the node js file on a cloud 9 server https://c9.io

var WebSocketServer = require("ws").Server
var http = require("http")
var express = require("express")
var app = express()
var port = process.env.PORT || 5000

app.use(express.static(__dirname + "/"))

var server = http.createServer(app)
server.listen(port)

console.log("http server listening on %d", port)

var wss = new WebSocketServer({server: server})
console.log("websocket server created")


wss.on("connection", function(ws) {
  var id = setInterval(function() {
        console.log("send ping")
    //ws.send(JSON.stringify(new Date()), function() {  })
      ws.send("A",function() {  })     // "A" = blink D7, "B" = don't blink 
    
  }, 2000)

  console.log("websocket connection open " )

  

  ws.on("close", function() {
    console.log("websocket connection close")
    clearInterval(id)
  })
})

I still don’t get how to program sockets using node js. Seems really confusing how to find simple DOCS for node modules “ws” “http” “express” and I need to work on a web page controller instead of my 2 second interval timer. However, with this setup D7 blinks every 2 seconds. When I switch the send command to send something other than “A” D7 does not blink. It appears that the connection is staying live and the Core seems reasonably stable.

I think I am getting the main web page working. The code is very easy. Still have to test it on the spark Core

<script>

var host = location.origin.replace(/^http/, 'ws')
var ws = new WebSocket(host);
ws.onmessage = function (event) {
    document.getElementById('myDiv1').innerHTML += event.data + '<br>'; 
};

 ws.onconnection = function (event) {
    ws.send('something');
}; 


</script>

<input type=button value="send D7-ON = A" onclick="{
   ws.send('A')
}">...

<input type=button value="send D7-OFF = B" onclick="{
   ws.send('B')
}"><br><br><br><br>

<input type=button value="mouseover D7-ON = A" onMouseOver="{
   ws.send('A')
}">...

<input type=button value="mouseover D7-OFF = B" onMouseOver="{
   ws.send('B')
}"><br>


 Test with: <a href="https://www.websocket.org/echo.html">https://www.websocket.org/echo.html</a> 
   remember to use  <br><br>wss://my-cloud9-server.c9.io<br><br> in the box<br>

<h1>Sent every 4 s</h1>
<div id="myDiv1">  here<br>  </div>

Presently using a github site at

NOW I CAN TEACH ROBOTICS! It looks like my Spark Core WebSocket works (In all its hacked, insecure, un-encrypted glory), will have to try it on my Rover. I am getting speeds of about 10 ms per command. Actually difficult to estimate the speed, things seem to falter around 3 ms, anyway much better than my TCP local WiFi server ~100 ms or the general cloud server ~1000ms. So I am very happy.

P.S. @zachary or @peekay123 any word on perhaps sending a class set (15) Spark Cores or Photons. You know I am going to keep asking every time I spend weeks on solving a problem that others could use with SparkDevices :smile:

Just want to say that the DOCS at Particle.io are a heck of a lot better than the DOCS for Node JS, still haven’t found anything really good to explain how to use express, ws, net, socket.io or http. I just tried using about 30 different Node JS webSockets and cut and pasted until things worked. I bet someone who knows what they are doing could tidy up my code dramatically. I haven’t really worked on sending data from the spark to a webpage since the Cloud 1 second delay is fine for my needs. I just need to be able to SEND commands to the Spark Core with little latency, if I am going to be able to drive a Rover with any kind of control.

My new Github site is at https://github.com/hpssjellis/Particle-Spark-Core-Photon-Websocket-Hack

I think this threads is close to being finished. Thanks for the help @ekbduffy, @omarojo, @chuank, @ScruffR, @kennethlimcp, @Dave, @BDub, @efatsi and @bko.

4 Likes

That sounds great @rocksetta! May I suggest disabling the Particle cloud connection to get even more reliable websocket traffic (Spark.disconnect()) while your TCP ws connection is going. I found that to help, but ymmv. Your Cores will light up green when that happens, instead of breathing cyan.

If you do that, you’ll have to rig up either a small h/w switch to enable/disable Spark cloud connectivity (to allow OTA firmware updates), but that can also be done by sending that command via your ws connection.

Will be interested in your further tests, as you potentially queue more data sent out your ws socket. The problem @ekbduffy and I faced was despite the low latency of a local ws connection, when the data stream got too long (due to Spacebrew’s implementation of JSON formatted messages), the Cores went into the hard faults as described in our posts. It was a case of incompatibility between the Core’s limited-but-understandeable TCP buffer size and the expectations of the Spacebrew message implementation.

@rocksetta, you need to PM @Steph regarding the Cores/Photons for your class :smiley:

Good ideas @chuank. I can use a cloud based function to disable the cloud and a ws command to enable the cloud. I always give a 25 second delay minimum at the start of my programs to re-flash the core after I screw it up, strangely a 20 second delay is not always enough.

Hopefully I will not have any long data streams as I plan to stick with single character controls. Even analog write data can be chunked to A=0, B=50, C=100, D=150, E=200, F=255, which for controlling a small vehicle should be fine. Small letters can have different meaning, so that is enough control for my students.

P.S. I did try extracting the analog amount from a ws command such as C255, it should not be hard to do but it strangely did not work when I tried to use the number as an integer. The above idea works fine for me.

This was something I meant to open a topic for a while myself, but always made my way around that issue by either increasing a delay or waiting for a serial or push button input.
So I’d “highjack” your post - if you don’t mind, since it might become handy for you too :wink: - to ask the pros like @peekay123 or @mdma, if there is a way to know when a OTA download is running (slowish magenta blink, before rapid blink while actually flashing)?
The thing with SEMI_AUTOMATIC is, that user code keeps running during the OTA download phase and hence any WiFi.off() or other command that disconnects the cloud will execute and mess up the OTA flashing attempt.
But if there was a way to know, that a download is going on, user code could take care for that.

I guess there are several threads where users do suffer from this, but might not be aware of it.
e.g. this locked thread, where the OP thinks his code was flashed, but never will be

The plan is that we will have system events that your firmware code can subscribe to so that you know when an OTA is about to start.

1 Like

Hi rocksetta,

Please tell me the software tool you used to capture this serial output

successfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒A▒A▒A▒A▒A▒A▒A▒A▒A▒Asuccessfully connected
HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: Upgrade
sec-websocket-accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B▒B

Many thanks

I feel your pain @sam_desd . I do not like the USB serial port communication, I prefer turning on and off D7 as proof of concept (I also think it screws up some programs). It is a pain to get going on windows someone needs to make a youtube video showing how to do this. I tend to brute force program, (try lots of things and see what works) and I was trying to get DFU mode working and got the windows drivers for serial installed. Still not sure what I did. The starting link is at

http://docs.particle.io/core/connect/#connecting-your-device-using-windows

and a thread is here


.
.

I tried to get a batch file working to install it on windows but got frustrated with it, but the github site has lots of information.

That is part of why I wrote my youtube IOT Rant

Anyway if you do get USB-serial working, a few extra steps are needed when using it. First connect your core by usb to your computer, on windows, it takes about 30 seconds for the computer to recognize that the driver is connected, the computer should beep. I load “device manager” so I can see which COM port it works on. Then load PUTTY.exe and enter your COM port and 9600.
Then hit the enter key. You should see “Hello Computer”

Many CRobot programs use some way to switch back and forth from this mode.

#define DEBUG = 1  // debug mode on

I find that confusing so I use

bool myUsbSerialDebugOn = true;      // set to false when not hooked up to USB

Then if using windows the Particle Core needs to wait for you to push the enter key so it uses the code

if (myUsbSerialDebugOn){
   Serial.begin(9600);             // Initalize the USB Serial port
   while(!Serial.available()) SPARK_WLAN_Loop();
   Serial.println("Hello Computer");
}

Which continually repeats until you hit that key. Not sure what it does if you are not connected to the computer.

After that in my program I show the code you asked about by the following code

   char myIncoming = client.read();

  if (myUsbSerialDebugOn){  
      Serial.print(myIncoming);
    // delay(2);
  }

Note the commented out //delay(2); This is why I do not like serial, I never know if the serial delay needed to print the information to the screen is messing up the program. Once I had to just get rid of the serial and try it and the darn D7 lite, suggesting the serial was messing with the microcontroller.

I only used the serial in this program to find out what the heck was being sent back and forth to the node js server.

Good luck. I may make videos about DFU and Serial but it is low on my list of things to get ready for September 2015.

What is this supposed to do? ( I understand the connecToMyServer fx implementation, which is connecting as client to websocket i9 server…right?) but what does the assign to connect method really does?

I think you are referring to

void setup() {

      pinMode(D7, OUTPUT);

      Spark.function("connect", connectToMyServer);
      Spark.function("stop", stopMyServer);

which is in the github file at https://github.com/hpssjellis/Particle-Spark-Core-Photon-Websocket-Hack/blob/master/socket.ino which gets flashed to the core (or Photon).

Really good question, because as I look at my badly commented code, I can’t see any way to actually activate these two functions. What I have probably done is forgotten to include my standard way of activating a function using my website at http://rocksetta.com/spark-core-photon/ or more specifically the webpage at http://rocksetta.com/spark-core-photon/spark-control-basic-web-page.html

Enter your ID and access_token and then just change the Function Name:–my-main to be either “connect” or “stop” to activate the server. I think I may have used this for testing and it is not needed once everything is actually going. Sorry about that. I will look into it and see if it is needed or not.

Correct, that is what I was referring to.
Basically you are assigning events or function calls on spark api to your own functions I think… Just not sure why.

Just tried it. It works great but I did leave out a major step. The Photon needs to be activated first to receive the websocket calls. So you have to activate the “connect” function before the Photon will receive information from the NodeJS server. I will put a dedicated webpage on the github site. Youi will need your CoreID and Access_Code for the page to work. You could also use the Client to activate the function.

Note D7 will show you that the activation worked, with a quick flash.

The new webpage is called activation.html and is now on the github site. Very sorry for the confusion.

https://github.com/hpssjellis/Particle-Spark-Core-Photon-Websocket-Hack/blob/master/activation.html

Here is what the activation.html webpage looks like

Note: It can be on your computer and does not have to be a live webpage.

Come to think of it I don’t think that step is needed except that the websocket will probably go to sleep if it isn’t used for 20 seconds or so. You could probably make a loop to activate the socket if it isn’t being used.

I was just trying to prove that the thing worked, so my code is probably not very efficient. Have fun. Hope it works for you.

THx