32 Channel Strain Data Acquisition System - Cloud connected

Hi guys,

Lovely community here. So i got into the bandwagon of the Photon. I am planning to build an entire data acquisition built around the Photon. Would really appreciate it much if i were to get help from all of you. It would be a learning experience for all of us. So basically here are my project details and list of components being used.

A 32 Channel Data acquisition system with 30 sensors measuring strain (momentarily using a metal gage strain sensor of 120Ohm being fed to a quarter bridge circuit) and 2 temperature sensors(PT100). I would like an acquisition speed of atleast 1000Samples/Sec per channel. So here is my idea as of now.

32 Sensors connected to its each wheatstones bridge. The amplifier is done using the XR18910/XR10910, which is a 16:1 bridge Mux interface with selectable gain. I have attached link https://www.exar.com/content/document.a … rt=XR10910

The XR18911 can be controlled via I2C link. it does all the amplification necessary. I will need to use 2 XR10910 or 4 XR18911(8:1 sensor interface). And then finally controlled by our Photon which converts the analog data to digital. and with the help of wifi to transmit it to a computer where it can be collected in an excel file along with time of capture.

Could you guys kindly help me out on how i am to proceed with programming of the Photon. I would really prefer to use Arduino IDE guys, as that seems to simplify things. Also if there is any improvements that could be added to the circuit. Kindly let me know. Please let me know if you need more details(Picture attached). Thank you :smile:

@fieryfire, let’s do the math:
30 channels * 2 bytes (12bits/ADC) * 1000 s/sec = 60KB/sec

Each sampling cycle will require I2C to configure the muxes, then DMA the ADC data to RAM and out to the server. Since the muxes need to be switched, you will not be able to DMA 1000 samples of all channels at the same time. You will end up with “slices” of data from each channel in the one second sampling interval. Every millisecond, you would need to sample 4 channels at a time, switch all four muxes, sample again and so on for all channels. This is a classic data muxing problem. If you need true 30-channel simultaneous sampling, you will need to reconsider your sampling strategy and hardware.

Communications will require a direct TCP connection to a (local) server (see @rickkas7’s audio sampling topic on high-speed sampling and sending to server) since all Particle cloud functions are not designed to handle this type of data bandwidth. At 60KB/s, that may be pushing the Photon limits.

4 Likes

Hi Peekay, Thank you kindly for the response. I was carefully considering the case. I was at the impression that the muxing from the I2C was directly related to the clock speed of the driver. So the photon acting at 120MHz should be to switch at 120 million times per sec ? or atleast half that, or no?

Morever carefully taking into the conditions of muxing, demuxing and storage of the information, not counting wifi transmission, should it not be possible to achieve a sampling rate of merely 1000Samples/Sec? How about i divide the systems to 8 channels each with its own micro-controller. in such a case isnt it plausible to achieve this speed. Would any other recommendations be considered? Like the ESP32 for example?

@fieryfire, unlike an Arduino device which dedicates itself mostly to running the user app in loop(), the Photon runs FreeRTOS to manage the WiFi and cloud firmware as well as the user app. On the Photon, loop() is given a 1ms time slice, thus running at a maximum of 1KHz. Looking only at the system clock is deceptive. Instructions may run at that speed but not peripherals (ADC sampling, I2C, SPI, etc.).

Having a micro-controller dedicated to sampling 8 channels each and connecting to the Photon as I2C slaves is still not feasible since, at 400KHz, you will not be able to transfer 60KB of data each second.

Do you need simultaneous sampling (ie time aligned sampling) for all 30 channels or simply consecutive samples for all 30 channels every second?

Regardless of the processor you use, the data sampling and aggregation issues remain. If you use SPI instead of I2C, you could (in principle) read data faster. You could use an RPi zero W coupled with 30 SPI based ADCs to do the sampling simultaneously and buffer it to RAM (which the RPi has plenty of) and send it out via TCP.

1 Like

That is quite unfortunate to hear about the 1ms time slice, that is a bit of a limitation to Photon.

And about the I2C speed, i was at the impression that it runs along the speed of the clock set by the micro controller.

And yes, i would definitely need each of the channels to have 1000 Samples/Sec, without counting for the 2 temperature channels of course, which could be done once every minute or more.

As for the last paragraph, I am a bit unaware of the terms RPi. I do know that SPI is another form of I2C, i have not had much experience on this. But it appears that the amplifier chip XR18910 does not support the SPI interface. But for now, i can do with simple serial transmission via USB.Adding the wireless transmission can be added later on.

I2C has predefined clock settings and the STM32 chips support 100kHz (Standard Mode) and 400kHz (“Full Speed” Mode).

RPi is short for Raspberry Pi

1 Like

The question I was asking was whether you needed all 30 channels to be sampled at the exact same time (synchronous) as opposed to simply being sampled at any time during the 1 second interval.

The 1ms FreeRTOS time slice is not a limitation. It simply indicates that loop() runs at 1KHz. @rickkas7’s high speed data sampling of a single ADC input uses DMA coupled with a hardware timer thus bypassing loop(). However, in your case, you have multiple ADC inputs and muxed signals which imposes switching time. Looking at the XR18910 shows that the I2C bus max speed is 400KHz, the same as the Photon. Essentially, you are left with this type of timing:

  1. Switch to input 1 on MUX1 via I2C
  2. Switch to input 1 on MUX2 via I2C
  3. Switch to input 1 on MUX3 via I2C
  4. Switch to input 1 on MUX4 via I2C
  5. Sample ADC0 to ADC3 sequencially (DMA may not be useable)
  6. Store in 60KB RAM buffer
  7. Repeat for all channels

In theory, you could use SparkIntervalTimer to fire off an interrupt at intervals slightly greater than the time it takes to do one pass. The timer ISR would, using a simple FSM, cycle through all mux inputs on successive interrupts. On the last pass, it would disable the timer interrupt and set a flag to be used in loop(). Another timer could be used for the 1 second sampling marker. It would fire at 1s intervals to re-enable the sampling timer interrupts. This could also be done with a Software Timer.

When the “sampling done” flag is detected in loop(), it would (ideally) flip the target RAM buffer to a second buffer (to prevent overwriting by the ISR) and send the first buffer full of data via TCP or UDP. There is no hope in heck that you can send a continuous stream at 60KB/s over the internet due to latency, etc. You will need a local server to receive the data.

The second RAM buffer is the problem. The Photon doesn’t have enough RAM available to the user for two 60KB RAM buffers. Another possible approach is to create a circular buffer. Code in loop() would continuously look for available data in the buffer and send it via TCP or UDP. It is possible in this configuration to lose data if data goes out slower than it gets put in by the ISR. Synchronizing between loop() and the ISR is also tricky (though possible IMO).

What’s missing out of all this is the addition of any time indexes in the data, adding to the buffer size, sampling time and transmission time.

Your comment about front end having separate micro-controllers to capture data is the start of an alternate solution. The reason I indicated SPI is that it is a much simpler bus which is un-addressable. Devices are selected, one at a time, using a chip select line (ie GPIO pin) for each device. However, SPI can run in excess of 30MHz so it is suitable for high speed data transfers (eg. OLED or TFT displays). As before, the Photon’s RAM constraints and TCP/UDP send time still apply.

I mentioned a Raspberry Pi since its processor runs faster, has more memory and can run python or nodejs since it runs Linux. However, precise timing is trickier. The ESP32 is relatively new so I can’t comment on its ability to handle your needs.

2 Likes

400KHz would be enough to drive 8 to 16 sensors at a time at 1KSamples/Sec? Ah forgive my ignorance, I hadnt known the acronym for raspberry Pi.

Thank you for the extremely well presented arguement. Your comment on the time indexes taking a huge chunk of bus bandwidth was also a cause of concern for me.

To answer your question, Yes i would definitely need the channels to be sampled at the exact same time (sychronously). But a few microseconds gap between the sampled data is not an issue as far as i am concerned, this would not be an issue. And precisely the time stamp is most definitely required during the transmission.

Might i suggest that instead of going the huge channel count. may i divide into 4 boards of 8 channels.

That is 8 sensors-> 1 XR18910(8:1 mux amplifier), with one Photon/ESP32 controlling this XR18910 and then transmission towards a system.

In the above case my only cause of concern is how to manufacture multiple boards of 8 channels each to transmit data over wifi to a single computer. Or is it even possible that i can connect 4 ESP32/Photo to the same wifi network and store the information there. And i would let the time stamp of the capture be the time recorded in the cloud?

Would you kindly make me understand the feasibility of the solution?

@fieryfire, let’s get back to basics with these questions:

  1. Do you need to have continuous sampling at 1KHz with data flowing to a server (stream) or only for a short duration (burst) of a few seconds and then send the data?

  2. Is the data going to a local server or a cloud server. If cloud, which server?

To answer to the questions -

  1. Yes, i would need 1Khz with data flowing ( initially i plan to do via serially). But i would like to set a trigger mode as in , when the vibration occurs, i capture the information and send the bursts of information at 1KSamples/Sec to a serve or serially . This should reduce the burden of the photon perhaps?

  2. I am very ignorant of this fact of a local serve or a cloud server. I was first starting to send data to a local system first. and once that work is complete, i could begin work on to a local server or cloud server. As i understand, our institution does have a cloud server, but i need to understand the specifics of how to connect with it and store the information there. Or perhaps i could utilize a third party application like Blynk to perform this?

@fieryfire, you’ll need to beef up your understanding of data acquisition. I reiterate that sampling 30 channels of (12 bit or 2 bytes) data, including a time stamp (assuming 32 bits or 4 bytes) for each channel, at 1000 samples per second represents about 180,000 bytes of data per second. The serial port on most micros will run at 115,200 baud or 11,520 bytes per second. So, every second, data in vs data out would lag by 168,480 bytes! This is not a workable model!

Working backwards, at a perfect 11,520 B/s, that represents 64 samples per second, somewhat off of the 1 kilo samples/sec you are looking for.

Using Blynk won’t help you either. The issue here is data collection and aggregation at your target rate.

Even with a “trigger condition”, you would most likely need to monitor the gauges at full speed anyway. The only difference is you are not capturing data.

The synchronous aspect of your requirements also dictates a non-hierarchical data acquisition approach. The multiplexers you want to use preclude that synchronicity. I’m not sure how to say this but I don’t believe that Particle is the right platform here. In fact, streaming 180KB/s of data over wifi is not a good approach IMO. Designing a proper solution requires quality engineering time and is beyond the scope of this forum.

Hi @fieryfire

I think you need to rethink the 1000 Samples/s part of your idea. That sampling rate would allow you to resolve frequencies up to 500 Hz under the Nyquist criterion which is in the band of musical notes. So unless your strain gauges are very small and connected to violin strings, I don’t think you have thought this through. Mechanical systems have momentum that limits the achievable vibration frequencies.

If you are looking for transient rather than steady-state response, you should should consider having a trigger type event that makes your device briefly record all channels as fast as you need into as much RAM as you can spare and then stops. After the recording you can send the data out a more leisurely pace and then get set up for the next transient event. You can turn-off the Particle Cloud connection while you are recording and then turn it back on when you need to send data for the most speed. Not a simple program to write, but possible.

1 Like

Dear Bko,

Thank you for the insight. That is exactly what i am trying to do, a trigger across the 32 channels would be a fantastic idea. I have a feeling that not all of the channels would be triggered at once even across the same surface. And until the trigger is set, no values are transmitted.

And the idea is excellent as well. Although i only have limited knowledge in coding Arduino, and im yet to begin RTOS as peekay mentioned. On how to record information over to the RAM is still a question. I am guessing in order to make things easier, i would have dedicated channels of ADC or limit 8 channels to one ADC, which would make things much more easier?

I understand better now. That means that i am unable to utilize the serial port for sure. How about using XBee Wifi? would it be a plausible that if i limited from 32 channels to 8 channels. i add the information of which set (ex. set 1 for 1st 8 channels set 2 for 2nd 8 channels and so on) which i assume should not be a huge burden on the ADC and the transmission. My only concern is that in order to make this happen, i would need 4 different photons to transmit to one receiver… or 4 different xbees to communicate to one receiver Xbee.

I was at the impression that wifi could transmit over 10Mb/s of information. For 180KB/s shouldnt be much of a problem would it?

Could you kindly suggest me an alternate approach i could take? instead of a particle? perhaps another solution?

Hello,

From this thread and talking to peekay123 this sounds like you might need a FPGA solution custom engineered for this. This sounds like something that our partners services can help with! I can also leave this post open for two weeks to see if anyone from the community would like to help out as well.

Kyle

1 Like

This topic was automatically closed after 14 days. New replies are no longer allowed.