I’m looking for discussions on best practices for managing high volumes of time series sensor data. To be more specific;
- I have one proof-of-concept Photon deployed with a bunch of temperature sensors that report every 3 minutes.
- This generates approx 2.5M data points per year per device.
- I need native resolution data for anomaly detection and alerts. I plan to keep this data ~4 weeks to support post anomaly/event diagnostics.
- I am considering summarizing data hourly for data that is older than 4 weeks. This would include; count, maximum, minimum, sum, and sum of squares.
There are lots of ways to approach this and they all have some trade offs.If anyone has worked through the trade offs associated with these decisions I welcome your input.