Product Webhooks, Private Events & Device Claiming

Cloud
, ,
1

Hello,

So after hours of debugging and researching the forums I have come to realized that:

  • All webhook responses/calls backs are private
  • Private events cannot be sent/recieved within a product
  • The only way to recieve events are by claiming the device

This is base on the following posts and replies by @rickkas7:

My problem is very similar to that of Is there a way to setup and claim multiple electrons at once? - #10 by rickkas7.

A summary of the issue is:

  • I need to recieved private events
  • I can only recieve private events when a device is claimed
  • I need to claim devices and I would like devices claim in bulk as part of my assembly and manufacturing process. I want this process to be scalable for a large number of devices.

Solve any ONE of the above points and I will be happy.

My suituation is as follows:

  • I have bought a large number of particle (~300 Borons/Argons/Xenons/Electrons)
  • We have a manufacturing process where by particles are integrated into our custom parts making up our product, boxed up and shipped to customers
  • If we want to claim the particle, we have to do this before we sent this to the customer. This means that all devices will have to be...
    • Powered on
    • Configured to connect to the cloud (in the case of Argons/Xenons)
    • Wait for connection (sometimes 10-15mins for Boron's first connect)
    • Individually claimed (probably by particle-cli, scripting this is no problem it is really the above points)

My question is as follows:

  • How is this claiming process scalable for large numbers of devices? Or are there any alternatives? Configuring each Argons and Xenons on the manufacturing line is not impossible but a HUGE hassel and adding time to assembly/manufacturing. But for Boron devices? The time taken to first connect is really not scalable.
  • Even after claiming the devices, it takes a hard power on and power off after the claim for me to start recieving private web hook response. A power cycle by the reset button or just waiting for the claim to take effect does not seem to work. If we push claiming to the customer's side, this is for one, a crappy customer experience, and secondly unnecessary since the customers do not interface with particle's console in anyway.
  • All devices are currently claimed under 1 single account which feels unnessary. This also feels like a point of failure if the account is also accidentally misconfigured. Is particle looking at sending private response's on a product scope at all? @rickkas7 said in Is there a way to setup and claim multiple electrons at once? - #12 by rickkas7 (quote below), is there a timeline for this? Large deployment is EXACTLY my issue here...

That being said, a way to use product devices without claiming is something that’s being investigated, as it would eliminate a step that’s a pain for large deployments. I don’t have a time frame for that feature, however.

2

There is currently no support for private events for unclaimed product devices. As the feature is not on the current roadmap, there is no schedule for it. It has not been rejected as a feature, it just hasn’t been prioritized and scheduled yet.

Claiming to a single account is the easiest way for most cellular product creators, but claiming to customer accounts is supported and is an option.

The having to force disconnect after claiming to receive private events is believed to be a bug. That will probably be fixed sooner.

1 Like
3

@rickkas7 thanks for the response and for insights into the roadmap and scheduled. I am disappointed that currently there is not even a priority, but can only give the benefit of the doubt that Particle is working on "more important" items. It is what it is. Even you mentioned in the previous post that being able to use the device unclaimed would:

I am not sure how particle's feature priortizing works but if there is an upvote system, count my vote in for implementing this feature. I am hoping that large deployment features and improvements are somewhere on their priority. =(

If this is fixed and the transition from "unclaimed" to "claimed" is really quick, then perhaps we can look at the possibility of automatically claiming devices on our first connect. Do you happen to know if this issue is being fixed soon? This might influence whether I suck it up, put in the extra hours and claim my devices for a few weeks, or implement some work around.

After thinking about this issue for a abit, I think I have a work around for my particular solution. But this is only because I have some control over the implementation of our web's backend system. The work around I have in mind is something I just thought of an hour or 2 ago so it is still in its "theory" stages and may have loop holes, so if anyone sees a large flaw please let me know.

Anyway. essentially we could implement our own public call back. It is proven that unclaim device can still recieve webhook calls and data, so a custom call back could be implemented through another web hook. This call back would be triggered on each publish from the backend and basically mimic the web call back with the relevent data (i.e. 200 OK, 400 Bad Request, 404 etc...). This webhook could even handle the response of multiple web hooks if we throw in an identifier for each hook into the data recieved. Off the top of my head, maybe something like:

{ 
    "id":123,
    "hookName":"myHookThatIsNotLargeDeploymentFriendly",
    "response":200,
    "message":"someMessageThatIsProbablyOptionalAndUsedForErrors"
}

Note that ID should always be unique to EACH publish event whose response is dependent on this hook to allow you to spam publish if need be.

The downside of this would be...

  • This is limited to web APIs that you only have control over. Any standard 3rd party APIs will not work unless routed through your backend. I guess similar to what particle is doing with webhooks.
  • Now you have this overhead of this additional response callback and probably higher data rates...

One positive thing is that, maybe we don't have to bother subscribing to multiple hooks or any other infact. We could just subscribe to this one for all callbacks.

I'll let you guys know if I end up trying this and if it works or fails.

4

I created a proof-of-concept auto-claimer that does not require an external server. It only requires two webhooks and a little device firmware. The best option is still to claim the devices during manufacture and test, however this is another option that is available.

The basic idea is that the device keeps a flag in EEPROM to know if has been claimed. If it has not been claimed, then it publishes an event that triggers a product webhook to claim the device. After publishing the event, and a delay, the cloud connection is reinitialized and then the claiming tested by using the test event. That will only succeed if the claiming worked.

The full documentation and library are here:

GitHub - rickkas7/DeviceClaimRK: Particle firmware library to help with device claiming

4 Likes
5

@rickkas7 wow! Thanks! I will definately take a look at this as well as try this out myself!

1 Like
6

@rickkas7 spent some time studying the library you provided. I've just got started but so far so good. Just a few questions I thought I ask.

The access token used for the "deviceClaim" and and "deviceTest" API, you mentioned in your git docs.

Do not attempt to use the token from the Web IDE (build.particle.io) as that token does not have product permissions and will not work.

Form my understanding we are claiming a device to a user account. Are you sure product permissions matter at this point? I did try the web IDE token and it does work but of courses ended up following you instructions to create a token via the CLI. Just a curiousity question.

For the account you mentioned:

This account must also be a product team member.

Currently I have a few products under an organization for the new dashboard layout. Is it okay that the account is under the organization instead of the individual product? I am assuming that being in the organization would give you premissions and access to all products but let me know if this is not the case. So far just being under an organization seems to work for claiming.

For the disconnect event you did a publish to spark/device/session/end. I was speaking to @marekparticle on the side a few days ago and he recommanded

PUT https://api.particle.io/v1/products/PRODUCTID/devices/DEVICEID/disconnect?access_token=PRODUCT_ACCESS_TOKEN

or

curl -X PUT https://api.particle.io/v1/devices/<DEVICE_ID>/disconnect?access_token=<ACCESS_TOKEN>

One URL is product specific and one URL is user specific. A side from the access token difference, he mentions it shouldn't matter since they achieve the same functionality. Is there a difference between the URLs he recommanded verses the URL you used? Or do they all do the same thing.

7

Form my understanding we are claiming a device to a user account. Are you sure product permissions matter at this point?

You're right, a regular user token would work here because I ended up using user-based API endpoints for both webhooks.

Currently I have a few products under an organization for the new dashboard layout. Is it okay that the account is under the organization instead of the individual product?

That should be fine.

For the disconnect event you did a publish to spark/device/session/end . I was speaking to marekparticle on the side a few days ago and he recommanded

The disconnect API endpoints disconnect from the cloud side. Since you want to disconnect from the device side, you use the spark/device/session/end. There are various situations where either makes sense, but since the device firmware knows it just sent the claim event, it's a logical place to put the disconnect.

If you had your own server monitoring device online events for unclaimed devices, then using the cloud-side disconnect would make sense because you'd need to initiate the disconnect from the cloud side.

8

@rickkas7 thanks! You've anwsered all my questions. I've finished studying your library and I'm in the process of integrating it into my code. I also did a proof of concept with the other work around where we mimick the call back through another call. This works but having the device unclaimed poses a security risk as any one who finds out the device ID is able to claim it. We will probably go with your proposed solution so I'm marking your post as the solution to this thread.

One last question:

Since we acknowledge this is an issue, is there a public git hub ticket or something I can follow? When this issue is resolve, I would also like to remove the disconnect step from my own code.

1 Like