Storing high-frequency (10kHz) data

time-series
#1

Hello,

I am evaluating databases to collect, store and analyze high frequency sensor data. The requirement is to store different time series (around 500) each with a sampling rate of 10kHz. Can anyone tell if this is feasible with InfluxDB and what I have to consider? Can InfluxDb handle this amount of data (~500 billion rows / day) and what performance can I expect when I query some timeframe and aggregate the data?

Thanks! BR Hans

#2

Hi Hans,

I think that, depending on your InfluxDB setup, you should be able to handle this load without a problem. What I mean by “depending on your setup” is the number of nodes in your cluster, amount of memory, etc.

Query performance is going to be an “it depends” answer because that will depend on a number of factors including the number of fields and tags in your query, the timeframe being queried, etc.

Hope this helps get you at least started, though if you’d like to post more details of the kind of data, number of measurements, tags, and fields, and the typical types of queries we can probably start getting a bit more precise in the answer.

Best regards,
dg

#3

It seems to me that influxdb can currently only process individual timeseries using a single thread (correct?) per series. For high-frequency data I guess this can pose a problem, since you quickly get long series?

#4

Hey Hans,

was there ever a conclusion? Did you end up using influxdb? If so, how did it work out?

Thanks for sharing your experiences
Philip