Hello, My team is having some issues with intermittent connectivity for sending instructions to the device or getting information back.
Typically we push instructions to the device with a function call via the particle cloud api.
Is there any built-in retry here?
Is there any way for our remote system to know whether the device received the data?
Our current means of handling this is that we’re having the device publish an event with its latest instructions; however, intermittent connectivity means that this packet can get lost, so we’re left with nothing.
Another thing we’ve done do try to make the communication more durable is having the events request information periodically (currently once per minute.)
The order of operations for how we’re achieving this is
a) the device publishes an event
b) the event triggers a webhook to a remote system (uni-directional)
c) the remote system invokes the same function call as above for pushing data
For practical purposes, we can assume that the connection between particle cloud and the remote server will not go down, but both (a) and (c) are subject to connectivity issues.
So, similar to above, is there any for our remote system to know that the RPC happened successfully, and if not, will it retry automatically (and at what rate)?
Another line of reasoning is that we could switch the “get me my data” calls from event->hook->function to event->hook->subscribe to the hook-response. I’ve read that the subscriptions will retry up to 3 times. How frequently do the retries take place? Is there any way for us to know whether the device got the hook-response? (ie, could we potentially set up another integration and kick off another webhook to try again?)
Open to other ideas here! Trying to handle these scenarios is eating a lot of our time.
PS
Just to clarify, when I say “instructions”, I don’t mean that we’re trying to rely completely on remote operation, but rather update the devices parameters by which it operates.
Oh, the other thing I was wondering is when a device tries to publish an event, is there any way for us to know whether particle cloud received the event? Will it automatically retry?
Also, sorry if any of my terminology is off here – my main focus is on the remote system and this is all very new to me.
Typically we push instructions to the device with a function call via the particle cloud api.
Is there any built-in retry here?
Is there any way for our remote system to know whether the device received the data?
When you make a function call, there is a retry at the CoAP level. You can tell if the device received it if you get a 200 HTTP result back.
Is there any way for us to know whether the device got the hook-response? (ie, could we potentially set up another integration and kick off another webhook to try again?)
There is no way to know if the device got the hook response.
The most common solution is to have the device event trigger a webhook or SSE event handler to your server. Your server uses the Particle API to send a function call back to the device to indicate that the event has been received.
If the device firmware does not receive the function call, it resends the event. It's recommended that you include a unique serial number in the event because it's possible for a duplicate event to be received at your server, so you should take that into account.
There's an example of doing this with node.js and SSE in the guaranteed delivery example here.
When publishing an event, if you check the result from Particle.publish and it returns true, the event was received by the Particle cloud. It does not indicate that the webhook was sent or received, only that the event was received by the cloud.
There are two retries before the timeout that occurs at 20 seconds. Even if the Particle.publish returns false, the event could still be received by the cloud, so code defensively in your server to deal with duplicate events.
It’s also possible to both check the result and be asynchronous by using a Future.
It seems like your application is similar to how the configuration service works on Tracker devices. The cloud maintains the current configuration settings (either fleet-wide or per-device) and they are synchronized with the device whether online or offline.
When updated in the console, the configuration settings are sent to online devices using Particle function calls. The reason is that the cloud can know if the transmission succeeded with a function, it cannot with a publish. Also, trackers are often unclaimed product devices, and unclaimed devices cannot subscribe to events, so publishing to tracker devices is not practical.
When a device comes online, it publishes a hash of its current configuration saved in the flash file system. The configuration service monitors for this event, and either sends an acknowledgement or a new configuration using a function call.