How to view all past historical publishes for a device


#1

Hello,

Looking for a way to view all historical publishes for a device registered to a product.

Unfortunately as I have documented here and here, the IFTTT integration is not reliable and/or refuses to function across multiple devices and/or publish events.

We have also set up our own custom server that uses the webhooks integration to push incoming publishes to it, however even this isn’t enough since the webhooks need to be registered to specific publish event names (i.e. “trans” or “dbase”, meaning that we would have to setup 62^12 number of webhooks to be able to ensure that all possible matching Publish event name character sequences have an associated webhook setup for them.

So, it would seem that Particle doesn’t have any way to support this. Is that correct?


#2

Why would you have so many Publish event names? An Event name should describe the event, which in most cases with be the name of a state or set of states like “Tank_Level”, “Temperature” or “Status”, it certainly wouldn’t be normal to give it a name that is is in fact a variable. Particle doesn’t store your events, is up to your 3rd party service to be sufficiently robust to catch them all.
Surely you could structure the publish event in a different way so that the unique identifier is part of the payload rather than the name? However I rather think your use case is bordering on the commercial side of IFTTT, as you possibly want to do things they want to charge for. The free part of IFTTT is for hobbyists to play about and developers to experiment with and potentially decide they want to use the commercial side of it, where they get multiple platforms, users and all that sexy stuff. At least that’s the impression I get…
IFTTT is not the only service out there, depending on what exactly you want to achieve there are plenty of examples on here of ways to get data into a google sheet. You can also store historic data in Azure, Ubidots , Losant (which like IFTTT fully integrates with Particle) and countless others.


#3

What I need is a sensible workflow that will allow me to add new event names my ever-growing application and have those new event names captured without having to create a whole new product webhook or a whole new IFTTT applet (which I would have to do for every device).

Currently, the only workflow that I am aware of that would allow me to capture a new publish event type via IFTTT is:

  1. Create an app with a new publish event name
  2. Go to IFTTT and create a new applet for each individual device for which I want to log this new event type.

So, if I have 100 devices, then I need to manually create 100 new IFTTT applets, since you can’t simply create a single applet that captures all devices or all publishes of a certain type. AFAIK IFTTT applets can only be setup on a per-device basis.

Is there a better alternative?


#4

Using the right tool for the job :wink:
At a 100 devices, you’re out of the hobby realm of IFTTT, and you should start looking at dedicated data storage/visualization solutions. @Viscacha mentioned several valid solutions, and they’re worth checking out.
Alternatively, a custom solution in the form of a simple node.js script that subscribes to all your events should work fine as well. You can do some pre-processing and send it off to where ever it needs to go.

If you could give us a more detailed description of what your end-goal is, we could advise more accurately, rather than the broad “I wanna catch everything, but IFTTT is uncooperative” :wink:


#5

if you have the resources available you could establish a local cloud. there was a thread i read today about a local spark server. each publish in the particle cloud could send a notice of activity to your local spark server which would catalog it for a historical database you want. may not even need the local cloud, just a database to log activity. a local cloud would allow you to manipulate the data in the same fashion as the particle cloud to some extent.


#6

My end goal is to be able to view all of the past publishes that all of my Electrons have published.

I think I have found a solution that works for now:

Setup a single webhook targeting ThingSpeak to fire on events with the “e/” prefix, and change all my publishing event naming conventions to have an “e/” prepended onto them. Won’t work at scale due to limits but should be good enough for now.