1,500+ SparkCores backend arquitecture

So I plan to manage lots and lots of SparkCores modules with my account.
And I need each and every sparkcore to report a variable value frequently to a server to be stored in a DB.
For later consumption via Mobile Apps.

The question here is…
1- Should I make my server ask via REST API to each and every sparkcore its variable value? (this will be done very frecuently), and take advantage of the sparkcloud features. I think this might have to be done async and could be very consuming for the server to cycle through this requests so frequently.


2 - Should I make each and every SparkCore POST its variable value to the server ? loosing sparkcloud features.
(BTW can this be done with the TCPClient Library ? )


3- Make all my MobileApps clients request via SparkCores RESTAPI directly to individual cores ?? this means, 1 core might have to handle hundreds of GET requests via SparkCloud.

FYI im tracking GPS positions.

And last question…
How hard would it be for SparkCore Team to manage a batch purchase of more than 1,000 sparkcores ?

Thanks, I :heartpulse: sparkcore


Hi @omarojo,

There is a feature in development that will be released in a couple of weeks called Spark.publish() that will suit your purpose perfectly. Each of your 1,500 devices would publish an event every time you wish to share data. For instance:

Spark.publish("temperature", "70.5");

You could then have your server subscribe to these events by listening for Server-Sent Events; this would function much like listening to the firehose from Twitter.

We can definitely handle bulk purchases! We just need a bit of advanced notice if we don’t already have them in inventory.


One would need “batch update” feature here to update all cores at once :smiley:


I think that the best way is to drive all of this information directly to your server with TCP Client library. I’ve already used this Lib - it’s pretty good even to HTTP requests.

1 Like

@zach will this be the EventSource API as standardized as part of HTML5?

@zachary will have to confirm but yes I believe Server-Sent Events == EventSource

Yes indeed @binaryfrost—that’s exactly it. Some resources:


As i said previously - best way is to use TCP client.
I think it’s good to use Node.js and my websocket client https://community.spark.io/t/websockets-client/3090 :slight_smile:


@omarojo this is very interesting. You have a good list of options, and I’ve considered something similar with many deployed devices. I think you should look at your goal and capability for your system/business. That is, hosting your own servers/app that Spark Core access means you need to be sure you have a very robust and capable system. It can take a bit to build this, but it offer the confidence that you control the entire content delivery system.

On the other hand, relying on Spark to provide the cloud based portion for the cores, at least at this point, allows you to focus on your actual product or solution to the customer. Use what they have (or will have with the server-sent events) for now and get your your system working to prove its viability.

Keep us posted, sounds exciting!


@omarojo The spark publish would serve a near real-time need. Is that a requirement?
Lots of businesses make the mistake of demanding real-time when it’s actually not truly needed.
There are some extra elements to consider for both polling and publishing data.

  1. How do you know when they’re not working?
  2. How is that different than when the vehicle is simply off.
  3. Is it ok to have data be up to a few minutes old? 5 minutes? 10 minutes?

Just a few things to consider. I’ve designed systems for “always” online kiosks. They had to push activity data as well as a heartbeat when there was no activity just so the mothership knew everything was ok.

This project sounds very interested. Let us know which way you choose to go.

1 Like

Questions 1 & 2 are very interesting, Im just assuming that if core is not reporting gps position then the core is not working or haven’t retrieved gps position or GSM vehicle network is failing or spark core wifi reconnection is failing too.
this will certainly have to be rethought. Vehicles are supposed to be off only and just only when they are in terminals, obviously can’t rely on that.

Question 3: I do need realtime position, because knowing if vehicle is 200~300 meters away from previous position is crucial for the project. The plan is to be able to see realtime position just like you see yourself using google maps in your mobile.
You know… seeing vehicle in one position and then watching how it disappears and reappears miles ahead is just not what I want. This is going to be used for passengers. In Mexico, public transport has no time schedules like in USA or Canada. Passengers have to wait in bus stops for the bus to pass, it could be at any moment and you never know if a bus is coming or not. This project is suppose to help passengers know ahead of time where the buses are and not have to waste time in bus stops.

Thanks for all your replies, as @Swieter said I would probably be using SparkCloud for proof of concept, using Server-Sent events. I thought spark cloud publish methods will then have to be HTTP POSTED to my own server but I didn’t know Server-Sent events could be used too, instead of HTTP POST which might be slow. My other option is to use common web sockets connected directly to my server and not to rely on spark cloud.

I like the use case. Very interesting.

Have you every seen Glympse? It’s a great mobile app. It looks like you’re basically trying to build this.
It might bring some good inspiration if not actually be your platform. I’m not sure how useful their developer features are.

I used Glympse years ago… didn’t know there were SDKs. I will check them out. Thanks.