[FYI] webhook responses - the 512B chunks not coming in order any longer

Hi guys,
I’m just letting know that if your webhook response is longer than 512B than the chunks of the response no longer comes in order, which was a case so far (at least for me). More than last 2 years my devices downloaded config assuming that the chunks come in order. Yestarday devices started to report that the config is malformed. Of course it’s my bad. I should have just from the start take into account the index of the chunk.

I’m just posting this message to inform the rest of our community!

3 Likes

Hi @gkujawsk, good to hear from you. We have an implementation where chunks can arrive out of order.

Our message order has never been guaranteed, but the nature of the old broker meant that messages almost always turned up in order. Having now moved to a cluster of servers and message de-duplication, there is a high likelihood messages will turn up in any order. Whichever broker queue finishes first will send the message to the stream, resulting in an unordered response. The change was made as this offers much more reliability, with the downside being an unordered queue.

An ordered response for TCP devices was almost always guaranteed, but for Cellular devices it has always been a jumble.
A feature in the works is blockwise transfers that will solve this problem for everyone. I expect the block-wise transfer feature in the first half of 2020. Blockwise transfer will open a stream down to the device and our device service will send ordered chunks to the device. The current implementation sends data as it arrives.

Our documentation team has been tasked with updating the relevant docs so users are not caught unaware. I am sorry that you have been affected by this.

1 Like