Performance & Stability

InfluxDB 2
1

Hi,

I’m working on a small research project that aggregates data from a hand full of sensors.
My requirements are really not to special, I want to stream the data (~120 datapoints) of 1-3 sensor with a frequency of 3Hz. Ideal estimations will bring me to 4 bytes (per value) * 120 values * 3Hz * 3600 * 24 = ~124MB/day/sensor.

I decided to give Influxdb a try, but the experience has been rocky, to put it mildly.
I installed the first v2.0 beta and started to write my sensor data to it, unfortunately the database crashed periodically.
After some testing, it seemed like a memory problem so I moved from a virtualized cloud node, to a dedicated machine (4 cores, 12GB RAM, 128GB SSD, 24TB of network attached storage for the data).
The first problem occured when trying to port the data from the cloud node, to the server, as the datastructure of the v2.0 beta was incompatible to the v.2.0 rc. So months of data down the drain, as there is no reliable way to import the data as the v2.0 beta crashed periodically (that was on me, for using a beta).

Now with a dedicated server and the non-beta v2.0 things look a bit better, but still really grim.

  1. Writes are ok and relatively stable now
  2. Reads are unusable slow over a wider timespan
  3. The server crashes frequently when I try to do reads

When I for example, try to read the min/max/mean of one value, it only works for about 14 days and takes maybe 15-30sec. Reading longer periodes produce a timeout or let the influxdb just plain crash.

I have to admit, that I’m not really an expert with time-series databases - so any input would be greatly appreciated, before I throw in the towel and move back to a relational database.

Is this amount of data already pushing influxdb to it’s knees?
Is the hardware not strong enough? (In theory all the data ever capture would still fit in the RAM of the server, so I’d doubt that)
Is there a way to improve the performance?
Is there any way to buffer the queries to make them faster? (for example to read min/max/mean per day and put it into a new timeseries).

Kind regards
BWA