Google-funded ‘super sensor’ project brings IoT powers to dumb appliances

this is an interesting project, powered by… a Particle Photon!
isn’t this amazing???

6 Likes

Interesting to see they just integrated the P0 module on their board vs the surface mount Photon. Make sense from a cost perspective.

Pretty cool use for a P0.

Nice features to add to the Google Home.

I wonder how they use the P0 to fuse all the sensor data and send it to their cloud for analysis. I’m sure that’s not done on the P0 itself.

I tend to agree. I bet they send all sensor data - unprocessed - to the cloud and analyze it there.

@gusgonnet, so why a Particle P0? There is no HTTPS/TLS so they must be using raw TCP or UDP since Particle.publish() is too slow.

if they are able to send all the data from the sensors in one publish payload, wouldn't it be enough?

I suspect they don’t process it on the cloud and indeed do the processing on the particle (at least in part). Processing these type of data on at a central location requires a fairly high speed data connection with high availability. Do some level of edge analytics and that requirement comes waaay down.

Don’t under estimate the processing power of the particle in particular if there is also a bit of analog preparation, there is a bunch you can do before you need to send it of to higher powers…

@joost, with that many sensors I suspect there is a minimum of processing. Possibly basic filtering/averaging but nothing too taxing. If you look at the computer screen, the audio data alone is being spectrally analyzed in real time along with the other sensors. Data might be sent using TDM (time division multiplexing) with some data having priority over other. I wonder if they use the P0 with Particle firmware or they went bare-metal and are doing everything custom.

oh wow, maybe @will or @zach can provide more info?

2 Likes

The study that goes along with the article says this about the processing,

On-Board Featurization
Data from our high-sample-rate sensors are transformed into a spectral representation via a 256-sample sliding window FFT (10% overlapping), ten times per second. Note that phase information is discarded. Our raw 8x8 GridEye matrix is flattened into row and column means (16 features). For our other low-sample-rate sensors, we compute seven statistical features (min, max, range, mean, sum, standard deviation and centroid) on a rolling one-second buffer (at 10Hz). The featurized data for every sensor is concatenated and sent to a server as a single data frame, encrypted with 128-bit AES.

2 Likes

Pity it appears the GridEye sensor is a restricted item - I tried to by online but says it can’t be exported outside USA. I already have a system similar to this with multiple sensors (though every much slower in reporting rate!)

FYI. I took a quick glance at the paper. Raw data are processed on P0 using FFT algorithm.

1 Like

I found GridEye can be purchased on Taobao. Not sure if they will believe to your place. However, I read the paper and realized that GridEye only contributes significantly to Kettle-on event, so I think the ‘super sensor’ will still work well without GridEye.

Thanks for the info - I tried finding it but can’t read Chinese & English page doesn’t seem to have it.

I will ask my friend who can read Chinese to find it for me.

I have all the other sensors on my own board and was looking for something to detect stove on and hopefully fridge on as well.

I’m going to China next week. DM me if you need help.

Looks like the GridEye sensor is this:
http://uk.farnell.com/panasonic-grid-eye-infra-red-sensor