Audio transmit and playback

Hi all!

I need to send audio (low quality) from a web page or app to my spark and need the spark to play it back…

I can figure out all the app part and the spark hardware part…

Does anyone have a suggestion as to compression / de compression and receiving the file on the spark end?

Anything that points me in the right direction is extreamly appreciated.

To start with, I wouldn’t worry too much with encoding, and simply try streaming the audio as 8-bit, 8Khz, and see if you can get that to work. The encoding only adds complexity which you can deal with once you’ve got the basic audio rendering working.

Once 8kHz, 8-bit audio is working, you then know you can then look to increase the sample rate or sample width until the system starts jittering. Once you reach that point, you then know the maximum bandwidth the spark can handle. The next step is to then use a codec to squeeze more information into the bandwidth available to improve audio quality (if necessary.)

Given the limited horsepower on the spark, I would suggest you look at either mu-law encoding, or differential-PCM encoding. Both can reduce the size by roughly a factor of 2.

As to hardware to play the audio, you will need at least an 8-bit DAC. I’ll leave it to the hardware gurus to recommend one, but I would probably go for something simple like this MCP4725.

On the software side, the audio rendering can work like this:

  • a sound buffer, which is is a shared circular buffer to store sound data
  • the main loop pulls data from the network and pushes it to the sound buffer. (This operation blocks if the sound buffer is full.)
  • an interrupt driven audio renderer that is set on a timer to fetch the next sample from the sound buffer and send that to the DAC.

With the circular buffer, audio data is pushed in by the main loop, and then read out again by the audio renderer.

I coded a circular buffer in flash memory, you could adapt this to work with a buffer in RAM, if you don’t find a circular buffer implementation elsewhere.

I hope that gets you started! Good luck!

EDIT: Eh, sorry I see now that you said you can handle the app and hardware! :slight_smile: Still, the codec details are there which is what you wanted!

3 Likes

All good input @mdma!

I only have one thing to add… I recently (yesterday) was at a STM seminar for the STM32F401 Discovery Kit DSP functions… and there was a nice demo of voice recognition and also audio record and playback. All supported on this $15 board.
http://www.st.com/web/catalog/tools/FM116/SC959/SS1532/LN1848/PF259098

Now the STM32F401VCT6 is a bit beefier than the Spark Core’s STM32F103CBT6, but you may be able to gleam some useful things from this example project:
https://dl.dropboxusercontent.com/u/41117656/Audio_playback_and_record_STM32F401_DISCOVERY_BOARD.zip

If you need to use small files, low quality WAV is easy to work with… but MP3’s are not (you need a DSP or external encoding/decoding IC such as a VS1053).

4 Likes

Yeah, definitely stay away from MP3 - I wrote an MP3 decoder once - the Cosine Transform alone would be too much for the spark - the Cortex M3 doesn’t have the DSP instructions that might make MP3 decoding feasible on an embedded device.

2 Likes

Wow!!! Extremely useful data!!

I can handle the app side… Not so much the firmware or hardware side in terms of DAC, etc… But I can handle an opamp and a speaker :smile:

Thanks. I am grateful for your help.

This will be kind of hard for me… So I will keep poking your brain for answers :smile:

Pardon my newbie question. I am more a software guy…

How would you connect the VS1011E-L ( : https://www.sparkfun.com/products/8126 )
To the Spark-Core…

I am trying to figure the pinouts and it looks like I might need more GPIO than I have available. And also a regulator to convert 3.3 to 1.8v it appears.

I only need the audio part. No Midi though…

Any clues?

Sorry for the vague question but I am lost.

it looks like it uses SPI, and a few other lines… 7 all up but you probably wont need all of them depends on what else you have connected to your core. The datasheet is a bit confusing and i don’t really understand the different SPI modes.

Plus you gain 4GPIO from it too :smile:

not sure where the 1.8v comes from the datasheet shows 2.7 to 3.6v inputs, one each of the digital/analogue sections

i would definitely start with it on a breakout board like this to get it up and running

https://www.sparkfun.com/products/8819

I need 2 Pins connected to my core additional to the required to this chip.

I tried looking at the breakout and I understand some of it.. I also checked a Shield for arduino but it's using all 12 pins!!! I am lost with the data sheet too

Thanks for your help!

Hey @frlobo I’m trying to do something similar where I stream audio from soundcloud into my speakers using the spark, How far did you get with your project? Any chance you could share your code?

I got some parts sourced from SparkFun to try some prototypes. I bought a set of mono amps and an MP3 decoder chip…

I haven’t started working. I am worried about the buffer sizes for the streaming… I am thinking hard changing to a Raspberry Pi for this application since I think it will be more robust for the application.

http://www.element14.com/community/community/raspberry-pi/raspberry-pi-bplus?CMP=KNC-USA-RPI-BPLUS

Il keep you posted on my advances. Please do the same!

2 Likes

@frlobo and @ish, any new progress on streaming audio. I just found this post and saw it is a little old but I am curious as I have a need to send audio to a web app. Looks like the Photon will make the process easier with better hardware.

I didn’t pursue this any further, its a bit over my knowledge!. But please if you find anything… Let me know!

@ish

did you make any progress on this? I’m working on a similar project where i need to send audio to my spark core. Not so much of a software person so can you let me know what you have in your code?

any help is appreciated

I’m downloading an MP3 from a web server and playing it using an MP3 shield. The unfortunate reality is that the Spark Core has some issues with it’s TCP receive buffers. This means that you need to transfer files in small chunks, which reduces the speed drastically. I haven’t seen reliable transfer rates that would allow for audio streaming.
There may be a CC3000 firmware update for the Core that will fix this, and it will also likely be fixed on the Photon.

@rsteckler, the issue with the TCP receive buffer is ALL in the CC3000 and I don’t believe there is a fix. So chunking is necessary to ensure that buffer does not overflow. The Photon uses a Broadcom wifi chip and runs FreeRTOS so I am pretty sure the issue will not be there. @mdma is the best guy to comment on either of these. :smiley:

1 Like

Hi All.

@mdma Just checking if you have a comment on this matter with the photon. Can I send audio (flc or similar simple compression, and play it back using the photon)… Or is the power simply not there? I am looking for voice. Not HIFI Audio…

Looking to receive directly in the spark, and playback directly thru the spark thru a small amp and PWM pins… Not with an Arduino MP3 shield…

I’m pretty sure you could use some form of compression, even if it’s something like u-law compression or ADPCM :slight_smile: But I imagine we could do better than that. Years ago I coded and optimized a mp3 decoder in java that ran fine on a 133MHz system - so while it’s not clear if we can stretch to mp3, I’m pretty confident we could achieve some form of audio compression on the Photon.

I’d love to have the time to look into this - sound engineering is so much fun!

2 Likes

I did an audio player library that uses two digital pins, no need for extra hardware besides a speaker, resistor, capacitor and wires.

1 Like

This is Great!! Thanks