Tips for Avoiding publish() Process from Blocking Other Critial Funcitons

Hello,

I have a small fleet of M404s running Device OS 5.9.0. I have a system that turns a pump on and pumps a fluid until a target volume is reached, as recorded by a flow sensor (hall effect). I've noticed that some funcitons in the loop() will not execute properly while the device is trying to publish an event, which can take several seconds. The clicks from the flow sensor are still recorded, since I'm using an interrupt, but other functions in the pump loop, like converting those clicks into gallons and checking if that quantity has reached the target volume to then turn off the pump, seem to be getting blocked by the publishing process.

Code is set up something like this. This is simplified for readability. Please let me know if additional details would be helpful.

Pump pump;

void setup() {
    pump.begin()
}

void loop() {
    // stuff
    pump.loop();
    // more stuff
    tryPublish();
    // even more stuff
}

void tryPublish() {
    if 10 minutes have passed since the previous publish {
        Particle.publish();
    }
}

Pump::loop() {
    gallons_pumped += convertClicksFromFlowSesnor();
    if (isOn() && isTimeToPump()) {
        gallons_pumped = 0;
        turnOn();
    }
    if (isOff() && gallons_pumped == target) {
        turnOff();
    }
}

I was wondering what is the expected blocking behaviour for Particle.publish(), and how can I avoid issues like accidentally pumping more that the target because the loop can't run until the device has finished publishing?

We usually turn on the pump once or twice a day and run it for about 1-2 minutes max. We typically publish every 10 minutes. The liklihood of trying to publish an event while the pump is on is not that great, but it still happens every now and then. I've implemented some code to not start publishing while the pump is running, and I'm wondering if there is a way to check if the device is actively trying to publish so that I can implement something similar to not start pumping until the device has finished trying to publish an event. In some cases, I need to publish while the device is pumping, but this is usually for debugging or real-time monitoring, and I often force the device to publish early and on-command by using a Particle Function to trigger the process. If needed, I could implement different functionality (i.e., omit the check if the pump is running) in this Particle Function to allow me to publsih while the pump is on.

I'm also wondering if I could implement an additional thread to run the pump loop or publishing process ot avoid this blocking and that seems like a robust and reliable option.

I'd appreciate any advice!

Yes, Particle.publish can block. if you check for Particle.connected() before publishing, most cases will be taken care of, but there are still some edge cases, and it could block for up to 10 minutes in the worst-case scenario.

Some options:

  • Move your pump start/stop code to its own thread. This is what I would do, because it is the least likely to be blocked unexpectedly. Make sure you don't have any Particle* or Cellular* calls in this thread, because they can block.
  • Switch to using CloudEvent in Device OS 6.3.0 and later which is designed to be asynchronous.
  • Move the publish to its own thread. You don't have to use this library, but BackgroundPublishRK shows how to do it.
2 Likes

Thanks @rickkas7! Do you recommend any of those options as preferable over the others?

They are in order of preference, best first. The best option is to put your sensitive pump code in its own thread. If it uses only things like digitalRead(), digitalWrite(), millis(), etc. it should be safe from any blocking. It can use delay(1) to yield CPU to other threads as mentioned in the threading explainer.

Great, I'll give this a try. Thanks!

Hm, you could set a flag when your pump is on and check the flag to delay the publish by some amount of time until the pump has finished :thinking:.

I do something similar already. I check to see if the pump state equals PUMPING, and I don't start publishing if that's true, but I haven't yet implemented something to check if the system in the process of publishing before starting the pump. And ideally, I'd still like to publish, or try to publish, while pumping in some cases.