Spark.publish is an awesome method, I tried it and it works really fine. I have only a doubt: what’s the best practice if I need to be sure that no event will be lost? For example, if I need to log all spark events in a database, I can listen to these events from a server. But what happens if my server go down? I will lose all the events triggered during the server downtime…
I don’t know if the event approach is the best if I need to be sure to log all events…
What do you think? Is there a way to have a very reliable events log?
Any help or idea is really appreciate!
Thank you!
can you have your data logger subscribe to the Spark.publish( ) events?
You could hook up an SD card to which your data gets logged. Then have it check for the server every once in a while. If the server is up and running, then transfer a whole bunch of data at once.
Tank you @Moors7, I thought to something like that. I could send a response to the spark for each event and, if the spark doesn’t receive the response, it will save the event on the SD card or anywhere. But it would be nice if the spark cloud would record a small events log, available through the Spark Cloud Api. Something like this:
GET /v1/events[/:event_name]/log
that returns the last x events. What do you think? Could be it a nice feature?
Another approach if data quality is that much of a point and you’d have a dedicated server to talk to would be TCP.
There you’ll have a protocol that caters for that.
Have a look at the TCPServer/TCPClient classes.
In the future the TTL setting could be used to make sure the event stays out there for a longer period of time, I think. Local storage will at least eliminate faults in the receiving end.
@peekay123, that’s true, but there isn’t an aweful lot you can do with that when your server is offline 
The online event gets published almost immediately, whereas the offline event may take a couple of minutes, in which you could lose some other events. The offline event won’t get published if your Core regains connectivity before the disconnection is checked, so there’s a bit of a grey area. Although you could argue that you won’t lose any events when your Core is offline, since they wont be sent in the first place…
@Moors7, I agree. However, for guaranteed event capture, it may be better to avoid Publish and Subscribe, as servicing time and availability is not guaranteed. Even with TTL, how deep would publish events be queued? Similar to IFTTT, you are not guaranteed a specific “sampling” interval. In this case, I agree with @ScruffR that a TCP solution and dedicated server is best.
2 Likes