Local cloud a.k.a. spark-server updates

[lbt]

Hey @Brewskey - thanks for working on this
I’ve gotten myself a simple Docker image up and am trying to run your clone inside it.
It’s running but I get:

Connection from: ::ffff:10.0.0.102 - Connection ID: 189
Handshake failed:  error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error { cache_key: '_189', deviceID: null, ip: '::ffff:10.0.0.102' }
1 : Device disconnected: Error: error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error { cache_key: '_189', deviceID: '', duration: undefined }
Device startup failed: error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error
Connection from: ::ffff:10.0.0.125 - Connection ID: 190
Handshake failed:  error:04065084:rsa routines:RSA_EAY_PRIVATE_DECRYPT:data too large for modulus { cache_key: '_190', deviceID: null, ip: '::ffff:10.0.0.125' }
1 : Device disconnected: Error: error:04065084:rsa routines:RSA_EAY_PRIVATE_DECRYPT:data too large for modulus { cache_key: '_190', deviceID: '', duration: undefined }
Device startup failed: error:04065084:rsa routines:RSA_EAY_PRIVATE_DECRYPT:data too large for modulus

I’m using node v6.10.0
and use ./node_modules/.bin/babel src/ -d lib

I have a mix of photons and cores - I think the photons give the pkcs decoding error and the cores give data too large for modulus.

Any ideas?

[Brewskey]

Can you post the issue in github? My engineer can look into this if I don’t get to it in time.

I’m confused about the Photon not working but the Core makes sense as I haven’t done a ton of testing with it.

Also, post what firmware version are you using on the Photons/Core.

[lbt]

Done: https://github.com/Brewskey/spark-server/issues/124

[prospect]

@Brewskey echoing the ‘thanks’ for working on this as well. Do you guys have plans to implement customers? I’m hoping to generate separate access tokens for each user without requiring them to setup a separate account. As well, I see that a json file gets generated for each user. Just thinking about scale; will this stay as-is or are there plans for a database of some kind?

[Brewskey]

@prospect - we looked into the amount of work needed to implement firmware versioning and it turns out that the organizations/customers/firmware versioning are somewhat dependent on each other. It would be a few month’s worth of work to get the bare minimum completed. We also do not need this functionality as we have our own database handling our customer info.

Anyways, the way we refactored spark-server should allow you to use custom implementations of all the stores so you can hook into your existing database or REST API fairly easily and use that.

If you still want to use customers in the current implementation you can do a pull request.

[chenc]

@sud335 @ScruffR @Brewskey @Dave @lbt Although i haven’t look into your code in detail, I will be really appreciate if you could give me some suggestion for the bug in original spark-server.

There will be a crypto transform error (bad decrypt) occurs when sending data frequently to from device to spark-server. I believed i know where the problem is, but not sure how to fix it.

The error is originated from the implementations of CryptoStream.prototype._transform in CryptoStream.js, where the cipher.end() cannot be done if the device send data too frequently. The detail can be referred to following simplified code.

try {
    var cipher = this.getCipher(callback);
    cipher.write(chunk);
    cipher.end();  // exception occurred here
    cipher = null;
    if (!this.encrypt) {
        this.vi = new Buffer(16);
        chunk.copy(this.vi, 0, 0, 16);
    }
} catch (ex) {
    // error occurred!  - bad-decrypt
}

After dig into these codes, I noticied that the error is originated from Handshake.js where the following callback will be called when new stream arrives:

that.secureIn.on('readable', function () {
    var chunk = that.secureIn.read();
    ......
    that.routeToClient(chunk)
});

routeToClient() routine basically route to the various processing function.

routeToClient : function(date) {
    var that = this;
    process.nextTick(function() {that.client.routeMessage(data);});
}

So as a summary when program enters the callback of that.secureIn.on, there may be two things waited to be executed:
* the next data stream as currently the device sends data at very high frequecy;
* the sending routine, which is routed by process.nextTick()

From my understanding, the second thing should have higher priority because it is routed by process.nextTick(), meaning it will be executed after finishing current loop and before I/O stream (since the cipherStream is a customized transform stream).

However the fact is that in some cases the first ting will be executed before the sending routine, causing the exception of crypto stream transform error, which disagrees how process.nextTick() work…

Please let me know if any one got some ideas, or is it because of the bug in process.nextTick() of node.js (i’ve tried on node 0, 4, 5, 6, 7… node 0 is best, but still will occur very infrequently…) Thanks for the help!

[Brewskey]

@chenc - are you using the Brewskey fork of spark-server?

It looks like what you posted is older code and we have added in a ton of bug fixes with our branch – it just hasn’t been merged back into the main Particle repository.

[chenc]

Thanks for your reply!
Yes indeed!, it is the original spark-server code.
I have looked briefly into your code, which have HUGE difference to original one. We think it can save much of time by modifying the minor bugs than migrating to new system. So it will be really appreciated if you have any ideas on the way for solving this bug.

[lbt]

@chenc Have you seen:

Solved the CryptoStream errors causing the node version incompatibilities, now works with up to node 12.5

I’ve pulled it and the other useful commits I’ve found into a particle-master-patched branch here:

Fix "routeMessage got a NULL coap message" issue.

I’m using this reliably until I can find time to debug some issues about claiming my old cores (photons are fine) on Brewskey’s branch

@chenc I would also say it’s worth looking at Brewskey’s code if you use Photons - the migration effort was minimal for me. I do have some issues with Cores but I’ve not had time to dig yet. I guess I’m saying don’t think of it as a new system, just a refactoring of the old code to allow scaling and support for more of the API.

[Brewskey]

@lbt - hey, I forgot about your spark core problems. Can you create a new issue on github with some basic repro information?

I’m going to be adding some improvements to the code for the next few weeks so I should be able to dig into that.

[chenc]

@Brewskey @lbt Super thanks for all of your suggestions!

Although I have not fully solved the Crypto stream transform error, which I think may be caused by the bugs of nodejs Process.nextTick() function, the performance seems much better than original server!

Another thing is that I thinks I found a small bug of Brewskey’s code after looks at it whole day. Our usage is to defined a customized event, for example we already have the public event, private event and another myCustomizedEvent! The server code will fail next data stream comes in. I felt the solution is to add a return in the following segment in ChunkingStream.js for avoiding process.nextTick() being called twice. It will be appreciated if you could give some hints!

  if (_this._incomingIndex === _this._expectedLength && _this._incomingBuffer) {
      _this.push(_this._incomingBuffer);
      _this._incomingBuffer = null;
      _this._incomingIndex = -1;
      _this._expectedLength = -1;
      _this.process(remainder, callback);
  } else {
      process.nextTick(callback);
      return; // THIS IS SUGGESTED SOLUTION!! :smile: 
 }

  if (!remainder && callback) {
    process.nextTick(callback);
  }
};

[Brewskey]

Oh wow, nice catch there.

If you want to give my your firmware to test with I can have one of my developers test with it and investigate.

[chenc]

@Brewskey Big thanks for your help, but I am afraid i can’t, as the project is currently unpublished… However I do hope to make some contributions to this open source

After digging into the code and comparing with the old code regarding these, I though the problem is in ChunkingStream.js file (however personally it is may more or less related to the bug of process.nextTick() which is the native nodejs functions).

Basically in ChunkingStream.js file, you may see following code segment:

_this.process = function (chunk, callback) {
    // do somethings
    process.nextTick();
}

This snippet is called by transform stream:

_this.transform = function (chunk, encoding, callback) {
    // do many things
    // call process() which is above function
}

So very similar to my previous post, after doing process.nextTick() there will have 2 possible things to be done,

• the envent (public, private, or myCustomized one)
• or the next transform crypto stream

However since the process.nextTick is used, theoretically the event shall have higher priority than crypto stream (I/O). But for some how, it release the next crypto stream before the event, which is not what process.nextTick() should do…

It can be observed that if the packet is huge and frequency is very high, the situation gets much more worse. It my test case the server it supposed to handle 10~20 requests for every 1 seconds with the size of each requests being around 2~3KB.

By the way, currently my test node is V6 and V4 node.

[chenc]

@lbt Yes, I do use these paddings on upgrading server to higher version node, however it only works for low frequency, small packet size.

If the serer need to handle larger number of requests with each packet size being ~kB size, CryptoStream transform error will occur!

[suhajdab]

I’m considering switching to local cloud for my smart-home particle gadgets, but looking at the github repo (last touched 2 years ago) made me unsure whether I should pursue the idea. On the other hand the forum seems to be active on the topic, so I turn to you:
Does anyone know if the spark-server would run on an rpi with jessie + nodejs 7 and particle devices with the latest firmware?
Thanks in advance!

[ScruffR]

@suhajdab

but looking at the github repo (last touched 2 years ago) made me unsure whether I should pursue the idea

That's exactly why @Brewskey has forked it and given it major overhaul ()

GitHub - Brewskey/spark-server: An API compatible open source server for...

[suhajdab]

Brilliant!! Thanks!