Sending real-time audio data using particle photon

[brontoluke]

I'd like to use a Particle Photon to send real-time data collected using a microphone over to another Photon that will receive the data and output a graphical visualization of the collected data, for example turning on and off some LEDs according to the mean amplitude that has been recorded in a given interval of time.
I don't need a very high resolution, I just need 100 samples per second and some packet losses are fine, but I need a stable and reliable connection that doesn't break.

I read of some examples successfully using UDP on the Core but with some issues with the code running on the Photon

Receiving UDP packets on the Photon

Hey, I’ve got a very simple bit of code that listens for UDP packets and uses them to turn the LED attached to digital pin 7 on or off. This code works fine on my two Cores, however when I run the same code on my Photon it never receives any UDP packets. Is there anything different I need to do for the Photon to receive UDP messages? Here is my code: int led = D7; // UDP Port used for two way communication unsigned int localPort = 12348; // An UDP instance to let us send and receive packets…

and of some problems with communication interruptions

Robust UDP communications

I have eventually tracked down a UDP error that occasionally interrupted UDP communications with a Core/Photon server. I cannot offer a cure but I have identified an additional try/except test at the client end that seems to keep things running. I’ve been using a Core and more recently a Photon as a UDP server to send monitoring data to a PC or Raspberry Pi. The Particle devices work very reliably but because they occasionally go offline (as far as UDP is concerned) while they renegotiate WiFi…

So now I'm wondering which could be the best move, so far I think I could go for:

1. try to implement a UDP server/client model between two Photon, the one connected to the mic being the server, the other receiving and displaying data being the client
2. try to implement the same model but using TCP protocol to avoid communication breakings
3. try to implement UDP or TCP communication but with a server in between (for example a NodeJS application) that collects data from the first Photon and send those to the other Photon, in order to have a bit of a futher control over what's going on during the data exchange
4. all other proposals are welcome

what do you suggest me?

thanks in advance,
Luca

[ScruffR]

I’d go with UDP and since the threads you quoted are quite dated there shoukd be less issues with the most recent firmware.

What signals are you intending to sample?
Are you sampling with higher resolition and only send the information derivec from the samples or do you intend to transfer the raw audio data?

[brontoluke]

thanks for your reply.

I’m going to sample an audio signal, mainly voices and ambient soundscape at high resolution but I will do some data transformation on the sending device in order to send 100 samples per second (1 every 10ms) instead of sampling raw data at higher data rate.

one more consideration, probably I will need to go for an Electron to be used as the server (no WiFi connection where I will record the sounds), but I think it should be the same as a Photon as far as UDP communication is considered.

[ScruffR]

If you are going for the Electron, UDP is definetly the better choice when it comes to data consumtion.
But with such high transfer rate you might still run into the data limit for Particle SIMs.
Even with a minimum packet size of 32 byte you’ll go through 1MB in less than an hour.

[davidgatti]

@ScruffR is right, now everything seams to work fine. I even created a nice article/tutorial where I put everything that I learned about TCP and UDP. https://github.com/davidgatti/IoT-Raw-Sockets-Examples. I hope the examples will help you have a quick start

Also check what David (not me) did on Hacksterio with Particle and Audio: https://www.hackster.io/middleca. This should give you a general idea how to go about it.

If you get something working, don’t forget to share it. I’m also very interested with audio over the internet.

[phec]

@brontoluke, I can’t offer a complete solution and wish I had something with rock solid microcontroller to web access. The UDP workaround has been quite reliable but it does mean that sometimes there are a few minutes with no communication while the Photon/router/Cloud access is renegotiated.
I haven’t found that TCP gives me higher availability.
For a reliable point to point radio link (for my Beehive Monitor) I have replaced the Photon/WiFi link with a much simpler solution using a Teensy and a Hope LoRa transceiver - the range is better too and I can use ARM’s DSP to do real time audio analysis on the Teensy as it has stacks more memory than the Photon. Of course I have lost the easy web access but that is not too difficult to obtain at the server end using a Raspberry Pi/LoRa shield.
Next up is an Electron when I have sorted out some data filtering to reduce the size of the payload. Then I’ll be able to talk to the bees from a long way away.

[brontoluke]

Dear all,
first I implemented a basic UDP sender/receiver but then I realized that I could use the publish/subscribe cloud functions. Indeed Particle.publish() can send up to 1 message per second with optional data of max 255 byte and I needed to transfer a hundred of data per second where each data is a 8-bit number.
The only thing I have to handle is to encode a sequence of 8-bit numbers (the payload) as a Particle String (or as a C const char *) and then to convert those back to an array of integers.

Both the basic UDP sender/receiver and the Pub/sub sender/receiver are on my repository on github

lucasguanci/two-photons-data-transfer

As I complete the code I will publish the updates.

Best, Luca