I’ve been having some fun with the Photon and reading off a couple of sensors. However, so far, I’m using it like I used my Arduinos, just logging data over serial and storing strings in a text file. What I’d like to do is be able to get that data over the web, but I’m struggling with a way to find a way to do so with data that is being collected this quickly.
I’m taking sensor values every 10ms. Each time, I’m printing a 38 character line. I’ve had some logging sessions that resulted in files that are several megabytes in size. I was thinking either storing chunks of data then uploading them somewhere or attempting to transfer the strings in realtime. The overhead for both seems pretty large, but I wouldn’t know for sure.
Am I asking for too much? What would be the best way to log this much data over the web?
I did a sample project that can transfer data at that rate without too much difficulty. You might be able to get some ideas from that.
The example is for a local server on a home network, but the technique also works over the Internet. The example is a little more complicated because it also includes the second step which also allows you to stream the data from the server to a web browser in real time, but you can skip that part if you don't need it.
That’s just what I needed, thank you! I actually, eventually, wanted to be able to stream to a web browser, so it’s great that you’ve done that, as well. A few weeks back I was attempting to set up a Python TCP server, but I was having issues piecing things together to work with the Photon. I had to focus on other parts of the project, so I dropped it for the time being. I think your node.js sample will do a much better job.
So, I’m using it to log data off a piezo element or an electret microphone (testing both). They are basically listening for drops of water and trying to determine the size of them. All these values are getting graphed so I have a visual of all these drops. 10ms seems to be the sweet spot between catching these drops and quantity of data.