Semaphores / Thread coordination

I am using the new multi-threading capability you all added sometime recently – awesome!

One question: I’m running into problems where I have multiple threads writing to the Serial port for debugging, and they are overwriting each other.

I need some sort of way to coordinate the threads through a semaphore or mutex or some other coordination mechanism.

Example:

Thread 1:

while (true)
{
    Serial.println("Hello World");
    delay(100);
}

Thread 2:

while (true)
{
    Serial.println("Another Line");
    delay(50);
}

Output on serial will be garbled when they overlap:

“Hello Another WoLinerld”


Thanks in advance!

I don’t have too much experience with embedded threading… But il give you some ideas…

Locks seem appropriate… Every strategy will have it’s drawbacks unfortunately :blush: In this case DeadLocks… But locks might be what you need however you might loose some messages if one thread wants to write a message and the terminal is bussy with another thread printing…

Another possibility might be a queue of messages that will get printed to the terminal by a third thread (queue manager) that reads the queue and write the messages to the serial port and remove the msg from the queue… You might have 2 queues, one for each thread and then the queue coordinator will read periodically to see if there are pending messages to be sent from any queue.

However theres the issue of reading and writing to a queue (potentially an array) at the same time (When the thread that wants to print adds to the queue, and when the queue coordinator reads the array or queue to TX to terminal. In my experience in PCs and Mobile devices this is possible as long as only one thread and only one WRITES and another only reads… specially on single core processor this shouldn’t be an issue, but I can’t be sure on the particle.

Good luck and let me know what you ended up doing!

Exactly, so where is your actual question?

Have you read something like this and tried something (e.g. mutex)?
http://en.cppreference.com/w/cpp/thread

I wrote most of that API. I know it is still incomplete, so I would appreciate some feedback on what’s missing.

The thread API is not documented yet because as soon as it becomes publicly documented, it will be a lot harder to change the API to improve it.

That being said, there is currently a class called CriticalSection that prevents the OS from swapping another task while that object is in scope (just create an object and it will prevent task switches inside the current set of brackets). It doesn’t disable hardware interrupts so the name is likely to change to something like SingleThreadedSection.

Here’s how you use it:

void print1() {
    CriticalSection cs;
    Serial.println("Hello World");
}

// Thread 1
while (true)
{
    print1();
    delay(100);
}

void print2() {
    CriticalSection cs;
    Serial.println("Another Line");
}

// Thread 2
while (true)
{
    print2();
    delay(50);
}
2 Likes

jvanier,
I’d be happy to provide feedback. Can you provide some pointers on where to look? I found some docs and looked at some of the firmware code on github, but I know I’m missing some pieces.

Thanks!
Jason

This is the file with the thread primitives usable from the user application:

Here are the thread and RTOS primitives usable from the system firmware:

The main question is which of these RTOS concepts would be useful to enough people to add them to the user API.

As I’m building out an application I’m finding that its been a bit complex handling events / input both local and from the web. Basically I have an internet enabled device that can be controlled from a web based console, but the device itself has interactive elements. I’ve packed it full of peripherals so there’s a lot of things to connect and many of them operate independently.

For me, in addition to the threads, the conditionals and mutexes would really simplify my development. I work around the lack of them right now using the software timers but it’s a bit of a hack and turning into a rats nest.

Can the mutex be made safe against priority inversion?

1 Like

Yes, FreeRTOS mutexes have priority inheritance to avoid priority inversions. I haven’t tried it myself however.

You can try the os_mutex_create, os_mutex_lock, … functions.

There’s no documented user API for this stuff yet. This kind of feedback is great though to figure out what that API should be.

In system firmware, we are using the std::thread, std::mutex etc. which are then mapped to the HAL layer. I’m contemplating making this available to the application so that developers have a familiar and well-understood API complete with reference docs. A detracting factor is that this isn’t as friendly as a typical Wiring API, but then multi-threaded programming with mutexes isn’t in the beginners realm.

Mutexes can be tricky to get right - a simpler approach is to use channels.

3 Likes

I’ll give channels a look. Has anyone integrated that with the photon yet with success? I couldn’t find any reference to the library in my cursory searches.

Regarding my background, I spent a couple of years developing a fairly complex chassis/system manager and I/O transport in vxworks, and more recently a highly distributed storage cache.

For what I’m building right now its quite a bit simpler so maybe channels might be the right way to go.

@jbeacon, have you seen the new 0.4.9 features?
https://docs.particle.io/reference/firmware/photon/#task-switching

Sure enough these are not nearly as powerful as the mentioned standard tools, but might be worth a look anyhow.

1 Like

@ScruffR, Thanks! I was able to play around with the threads and synchronization a bit. From what’s there I can make something work. Support for conditionals would be handy however. I’m still ramping up on how the firmware is put together, but I see that I can write to the hal layer directly to experiment with things.

I’m curious on what is all involved in pulling the conditional logic into one of wrapper classes?

1 Like

ok… So I’ve been playing with the threading for a week now and just demystified for myself how the firmware was built/linked etc. Using threads really cleaned up my implementation a bit and kept things humming along even though I have some device polling, so that was a big improvement.

A couple of questions though…

  1. Any plans to sync the version of FreeRTOS for the photon to 8.2.2 (as it is for the electron)?
  2. How keen are you to take a patch that allows for a timed wait on a mutex? (And also supports releasing the mutex within an ISR)
  3. Would there be reluctance to exposing conditionals as well? The current implementation doesn’t allow for a timed wait on the cv, but this would be fairly trivial to implement using the task notify routines in FreeRTOS 8.2 …

Reading through the forums it seems to me that threading isn’t at the top of most people’s lists and I respect the desire to keep all the touch points simple (and portable) moving forward. To me this is one of those areas where it would be nice to have access to a bit more of the underlying API but not require a custom firmware build.

2 Likes

Pull Requests are always welcome :wink:

1 Like

On the Photon, the FreeRTOS version is set by WICED. Their latest SDK does update the bundled FreeRTOS to 8.2.1, and we are waiting on them to provide us a version of the SDK that we can redistribute.

Even when FreeRTOS is upgraded in WICED, it’s a good idea to refrain from accessing FreeRTOS directly and instead use the abstractions in the HAL interface. The reason is that we may need to switch to ThreadX/NetXDuo for IPv6 support. This is still presently undecided/unexpored territory and not set in stone at all, but throwing it out there so you see the reason for abstraction.

Happy to take a PR on timed waits on mutexes. For acquiring/releasing from an ISR, I suggest not creating a new function but instead using the isISR() function in the implementation so that we can call the appropriate FreeRTOS function based on whether we are in an interrupt context or not.

Hi,
Is there any news on FreeRTOS 8 on Photon? I just wanted try out the RTOS Task Notifications API and discovered that the version of FreeRTOS is still 7.x. Is WICED every going to be updated for the Photon or did the sale to Cypress put and end to the plans? It’s unfortunate to miss out on all those features…

That’s still the plan. We haven’t received the approval yet. It’s been a frustratingly slow process. We’ll definitely update the community when we have a new version of WICED available for use.

2 Likes

Is the Thread API already online? If so, where can I find it? Thanks!

Hi there

I need to use a mutex on my SPI port as I have SPI radio and another SPI device.

I am running the radio in it’s own thread.

I have tried creating a mutex
using https://github.com/spark/firmware/blob/develop/wiring/inc/spark_wiring_thread.h

and also using the os_mutex create functions in
spark/firmware/blob/develop/hal/src/stm32f2xx/concurrent_hal.cpp

While the code compiles and runs the first time the thread tries to obtian the lock, it goes to SOS blinky mode.

I believe that the mutex is not actually getting created so I have a null pointer that is getting passed around, do you guys have any ideas or example code on how to get it working.

Here is what I have.

Thanks Marshall

//initial declarations

os_mutex_t mutex;
Mutex  spi_mutex(mutex);

//this is called from a seperate thread
void radio_receive_loop(void){
  spi_mutex.lock();
  SPI_swap_devices(SPI_lora);
     //do other radio stuff
   spi_mutex.unlock();
}



void loop(){
   		spi_mutex.lock();
		SPI_swap_devices(SPI_sign);		
             //do sign stuff
		spi_mutex.unlock();
    }

Oh it was easy

I had the declaration incorrect

Had this
os_mutex_t mutex;
Mutex spi_mutex(mutex);

Should be this
Mutex spi_mutex;

Now to see if the mutex actually works!