I am using InfluxDB 1.8.9 (32bit binary) on a Raspberry Pi 4, 4GB RAM, 64bit kernel (aarch64).
I was bitten by the out of memory errors of the Raspi 32bit kernel and thus enabled the 64bit kernel. But now it happens again, regularly, and the system load skyrockets, POST requests time out and lose data:
raspberrypi influxd-systemd-start.sh: ts=2021-10-24T16:44:15.882924Z lvl=info msg="Error replacing new TSM files" log_id=0XOc6RHG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0XOnoRxW000 op_name=tsm1_compact_group db_shard_id=56 error= cannot allocate memory"
This happens in an endless loop, always for the same shard_ids, and even offline compaction did not help at first (
influxd buildtsi -compact-series-file).
At this time,
influxd is consuming ~600MB of real and ~3700MB of virtual memory and there is some to spare (I had buffers of 2GB), so it doesn’t seem to be a physical memory issue, but maybe one of address space (32bit = 4GB)?
I have some questions.
- Would installing the
armhfarchitecture help? Would this work at all on my Raspi 4 preferably without reinstalling everything?
- Alternatively, would upgrading to Influx 2.0 do any better, is the problem solved there and the memory requirements when compacting do not depend on database size any more?
- What other alternatives do I have?