Graphing Variables (or post to Xively)

I’ve built a project based on Spark which measures air quality. I would now like to graph the data somehow. It seems my choices are:

  1. Post a variable periodically to Spark Cloud. However, then I have to write a server process somewhere to query the variable via Spark Cloud and either graph it myself or post it to Xively.com
  2. I could make a connection directly to Xively.com (which looks a bit harder to do but avoids any more server work).

It seems to me that the easiest solution would be for SparkCore to provide a graphing function but I’m guessing that’s not possible if you don’t record historical data.

Am I missing anything? Is there a better solution?

@denishennessy , in the short term your best bet is likely something along the lines of option #1. Option #2 would be very difficult because you’d need to implement the https protocol to run on the Core (aka https://api.xively.com/) (Please let me know if there is a non-HTTPS way to hit xively)

For now, rather than posting a variable to Spark Cloud from the core periodically, you’ll want to write a server script that polls the core via the Spark Cloud API to get the value of the variable then hit xively to graph it. Though we’re planning on adding the capability of pushing data from a core to the cloud sometime in the next couple of months, this functionality is not quite there yet, so polling is the way to go.

that does seem to be the options. If you are up for it you could try to port the xively arduino library and that may save you some pain (or cause more). I’m hoping to take on a similar project but I’m unsure of the route i will take.

ideally the best scenario would be to able to store data in the spark cloud but my guess is thats a long was off but would be an awesome feature. I might even be willing to pay for a feature like that. (hint to dev team)

@tidwelltimj that is an interesting idea, noted and shared with the spark dev team, thanks for sharing!

As far as I can see, the Xively.com Arduino library doesn’t require HTTPS (it uses HttpClient). I may have a go at porting that.

that’s true for embedded programing they allow authentication through just your secret key and device ID it doesn’t have to be on HTTPS. I have the library running on an UNO right now and its very fast to upload.

I’ve created a number of web pages using HTML and Javascript that pull data out of a cloud (Xively or others) using their Javascript APIs and plot that data directly in the web page using great, freely available plotting packages such as Flot (http://www.flotcharts.org). Xively’s built-in charting is nice but ultimately I wanted something I could more completely control in terms of layout, behavior, etc.

It can take a little work to handle retrieving large swaths of data from some cloud services as they limit the number of data points retrieved in a single request. (Xively does this, for example.) You’ll therefore need to issue multiple requests of the appropriate maximum size then combine the data into a single set for plotting. All this is fairly straightforward in Javascript, but does add extra structure to the code.

1 Like

Using a Python script and Highcharts (Java) I’ve created a Frankenstein setup for reading a value from the core and writing an auto-refreshing HTML page containing a graph display. This requires running the script repeatedly in order to refresh the values and the HTML page - I’m using ‘watch’ on a linux server:

watch -n 2 python core_value_chart.py

Assuming the process is running and the core is online, you should be able to see the results here.

Apologies in advance for the state of the code. I’m sure it’s not pretty.

Some questions, since we’re on the topic here:
Is there a better way to do this? I’d prefer the web page caused the Python code to execute - instead of having the script re-run repeatedly. Having the page update without a hard refresh would also be nice. Any thoughts on simple ways to achieve those goals?

I’m guessing it would be pretty simple to modify this setup to write values from the core to a local file, then graph those over time - useful for the air quality monitor mentioned at the start of this thread.

Look forward to hearing feedback on this topic. My goal (as hinted by the page) is to create a remote monitor system for my sailboat - so I know battery voltage levels, bilge level, cabin temp, etc.

The voltage being displayed on the page is actual voltage being taken from the Core. The other two items in the graph are just hard-coded.

Very cool! For now, it looks like you’re going about this the right way. As @jgoggins alluded to, we are adding a feature for the Spark Core to publish events that you can subscribe to so that you don’t have to poll the Core. In the meantime, however, this looks like a good way to go.

The only other suggestion I would have is to put the polling time interval in the Python script, but I’m not a Python guy so I don’t know how Python handles that (in JavaScript it would be setTimeout()).

If you want the page to auto-refresh, look at using jQuery (a $.ajax call) or perhaps a WebSocket to your server, which could send updated information as it comes in.

Tinkerman has posted a mini-review of a few services :