Javascript getEventStream returns poorly formatted data

Cloud
1

In my Node application, I want to subscribe to all of the events being published from my Cores. This is pretty straightforward with the API wrapper, after logging in I just say

Spark.getEventStream(false, 'mine', dataProcessing);

For debugging purposes, dataProcessing is just logging the data right now:

function dataProcessing( data ) {
  console.log(">>>> New data received");
  console.log(data);
}

I was expecting to get the data in the same format as I do from the Spark CLI, (spark subscribe mine) which is like:

{"name":"hour","data":"22","ttl":"60","published_at":"2014-12-04T22:58:03.705Z","coreid":"54ff75066667515136142367"}

But alas, not the case. Here’s a screenshot of what I get

Some times the event and the data comes as one String, sometimes it comes as two, and neither of them is valid JSON. This is a huge pain to do String processing on.

There’s another thread about this and an open GitHub issue, but both of those are a couple months old and haven’t gotten any attention. I’ll keep investigating and am happy to make a pull request, but I can’t figure out why this is happening.

2

Sometimes I also get this

Why return :ok? Return a JSON object that says like {"status" : "success"} or something, this seems like an unnecessary edge case to make

3

Looking at the Spark.js source for getEventStream and comparing it to the getEventStream method in SparkCLI, there’s a small difference:

// Spark.js
if (callback) {
  requestObj.on('data', function(event) {
    event = event.toString().replace('\n', '');
    if ((event !== '') && (typeof(callback) === 'function')) { callback(event); }
  });
}

vs.

// Spark CLI
if (onDataHandler) {
  requestObj.on('data', onDataHandler);
}

Why is the event.toString().replace necessary? And why wouldn’t you check typeof(callback) === 'function' in the same line as if (callback), i.e. if (callback && typeof(callback) === 'function')

4

Hi @louis_,

The Spark CLI does slightly more work than SparkJS in parsing out the results from the Server-Sent-Events endpoint on the API to turn the messages into something more useful. You can see the javascript that is parsing those results here:

https://github.com/spark/spark-cli/blob/master/js/commands/SubscribeCommand.js#L123

I’ll open an issue on SparkJS to remind myself to upgrade it to process the events into something more natively useful than the raw format. :slight_smile:

Thanks,
David

5

Hmm, I have some time so I’ll look into this now.

6

Awesome, thanks @Dave

7

All fixed! If you pull the latest version, you should be good to go. :slight_smile:

Thanks,
David

3 Likes