Hi there!
In order to store time series data on our edge device we are trying to make use on InfluxDb. We simply use the InfluxDb without any other components of the TICK stack. The technical details of the device are as follows:
CPU: Intel Atom (E3805) 1.33GHz (2 cores)
RAM: 2GB
What we try to achieve is to write 5000 points per second to the database (by having a series/measurement cardinality of about 700). Every 60 seconds we read the latest data point for each measurement.
Our retention policy is set to 4 weeks with no special shard duration. All other configuration options are default.
After about one hour we begin to see the first log outputs that look a bit strange: Compaction is aborted because a file already exists and it was not possible to allocate memory, see(1).
After 38 hours the InfluxDb crashes due to an out of memory error, see (2).
Has anyone any idea what we did wrong? Or what we can make better? Are there any configuration settings that we can change.
We tried to set “index-version=tsi1” , cache-max-memory-size = “500m” and max-index-log-file-size = “500k”, but nothing changed.
Thanks in advance for any hints or solutions!
Florian
(1)
ts=2018-11-16T14:55:49.888059Z lvl=info msg=“Aborted compaction” log_id=0Bmu~waW000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0BoGB0Vl000 op_name=tsm1_compact_group error=“compaction in progress: open /home/admin/.influxdb/data/bufferdb/bufferRetentionPolicy/4/000000512-000000005.tsm.tmp: file exists”
ts=2018-11-16T14:55:29.955279Z lvl=info msg=“Error replacing new TSM files” log_id=0Bmu~waW000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0BoF5K8l000 op_name=tsm1_compact_group error=“cannot allocate memory”
[…]
(2)
ts=2018-11-17T03:25:34.946891Z lvl=info msg=“Aborted compaction” log_id=0Bmu~waW000 engine=tsm1 tsm1_level=1 tsm1_strategy=level trace_id=0Bow4gMG000 op_name=tsm1_compact_group error=“compaction in progress: open /home/admin/.influxdb/da
ta/bufferdb/bufferRetentionPolicy/6/000000088-000000002.tsm.tmp: file exists”
ts=2018-11-17T03:25:35.127852Z lvl=info msg=“Error adding new TSM files from snapshot. Removing temp files.” log_id=0Bmu~waW000 engine=tsm1 trace_id=0Bow4V8W000 op_name=tsm1_cache_snapshot error=“cannot allocate memory”
ts=2018-11-17T03:25:35.131367Z lvl=info msg=“Cache snapshot (end)” log_id=0Bmu~waW000 engine=tsm1 trace_id=0Bow4V8W000 op_name=tsm1_cache_snapshot op_event=end op_elapsed=3321.035ms
ts=2018-11-17T03:25:35.131447Z lvl=info msg=“Error writing snapshot” log_id=0Bmu~waW000 engine=tsm1 error=“cannot allocate memory”
ts=2018-11-17T03:25:35.131570Z lvl=info msg=“Cache snapshot (start)” log_id=0Bmu~waW000 engine=tsm1 trace_id=0Bow4h6l000 op_name=tsm1_cache_snapshot op_event=start
runtime: out of memory: cannot allocate 8192-byte block (999686144 in use)
fatal error: out of memory
