InfluxDB not starting after power outage

InfluxDB 1
1

Hey there!

My influxdb 1.8.9 isn’t starting after a power outage, what shall I do?

Failed start attempt:

# systemctl start influxdb
Job for influxdb.service failed because the control process exited with error code.
See "systemctl status influxdb.service" and "journalctl -xe" for details.
root@<name>:/var/lib# systemctl status influxdb.service
? influxdb.service - InfluxDB is an open-source, distributed, time series database
   Loaded: loaded (/lib/systemd/system/influxdb.service; enabled; vendor preset: enabled)
   Active: activating (start) since Fri 2021-08-27 21:55:44 CEST; 1s ago
     Docs: https://docs.influxdata.com/influxdb/
Cntrl PID: 19689 (influxd-systemd)
    Tasks: 11 (limit: 4915)
   Memory: 33.3M
      CPU: 834ms
   CGroup: /system.slice/influxdb.service
           +-19689 /bin/bash -e /usr/lib/influxdb/scripts/influxd-systemd-start.sh
           +-19690 /usr/bin/influxd -config /etc/influxdb/influxdb.conf
           +-19727 sleep 1

Aug 27 21:55:45 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:45.872958Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_in
Aug 27 21:55:45 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:45.874132Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_in
Aug 27 21:55:45 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:45.875416Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_in
Aug 27 21:55:45 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:45.876573Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_in
Aug 27 21:55:45 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:45.876799Z lvl=info msg="Reading file" log_id=0WEIlQIl000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_
Aug 27 21:55:45 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:45.930262Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_in
Aug 27 21:55:46 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:46.090121Z lvl=info msg="Opened shard" log_id=0WEIlQIl000 service=store trace_id=0WEIlQnG000 op_name=tsdb_open index_v
Aug 27 21:55:46 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:46.092988Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/ope
Aug 27 21:55:46 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:46.191859Z lvl=info msg="Opened shard" log_id=0WEIlQIl000 service=store trace_id=0WEIlQnG000 op_name=tsdb_open index_v
Aug 27 21:55:46 <name> influxd-systemd-start.sh[19689]: ts=2021-08-27T19:55:46.192857Z lvl=info msg="Opened file" log_id=0WEIlQIl000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/ope
lines 1-23/23 (END)

Logs:


Aug 27 21:56:20 <name> systemd[1]: Starting InfluxDB is an open-source, distributed, time series database...
-- Subject: Unit influxdb.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit influxdb.service has begun starting up.
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.516115Z lvl=info msg="InfluxDB starting" log_id=0WEIn_e0000 version=1.8.9 branch=1.8 commit=d9b56321d579
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: Merging with configuration at: /etc/influxdb/influxdb.conf
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.517952Z lvl=info msg="Go runtime" log_id=0WEIn_e0000 version=go1.13.8 maxprocs=2
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: Merging with configuration at: /etc/influxdb/influxdb.conf
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.642547Z lvl=info msg="Using data dir" log_id=0WEIn_e0000 service=store path=/var/lib/influxdb/data
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.644144Z lvl=info msg="Compaction settings" log_id=0WEIn_e0000 service=store max_concurrent_compactions=1 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.645242Z lvl=info msg="Open store (start)" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open op_event=start
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.675342Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/692/000000029-000000002.tsm id=0 duration=2.113ms
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.675950Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/96/000000001-000000001.tsm id=0 duration=0.126ms
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.863636Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/96 duration=188.557ms
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.868634Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/693/000000029-000000002.tsm id=0 duration=2.348ms
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.886444Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/692 duration=214.216ms
Aug 27 21:56:20 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:20.890652Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/694/000000029-000000002.tsm id=0 duration=2.020ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.126018Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/693 duration=260.825ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.130421Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/695/000000029-000000002.tsm id=0 duration=2.366ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.149072Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/694 duration=261.298ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.171237Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/697/000000029-000000002.tsm id=0 duration=20.714ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.385650Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/695 duration=258.696ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.422121Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/697 duration=272.955ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.423003Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/698/000000029-000000002.tsm id=0 duration=34.938ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.425908Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/699/000000029-000000002.tsm id=0 duration=2.534ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.715455Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/698 duration=328.496ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.719600Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000008-000000002.tsm id=0 duration=1.905ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.719600Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000022-000000001.tsm id=7 duration=1.774ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.721374Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000017-000000001.tsm id=2 duration=1.560ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.721413Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000016-000000002.tsm id=1 duration=1.660ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.723333Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000018-000000001.tsm id=3 duration=1.809ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.727337Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000019-000000001.tsm id=4 duration=5.733ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.728226Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000020-000000001.tsm id=5 duration=4.789ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.728841Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/700/000000021-000000001.tsm id=6 duration=1.358ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.729074Z lvl=info msg="Reading file" log_id=0WEIn_e0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/700/_00111.wal size=10523762
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.738452Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/699 duration=316.189ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.760310Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/104/000000001-000000001.tsm id=0 duration=0.221ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.970330Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/104 duration=231.809ms
Aug 27 21:56:21 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:21.971525Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/112/000000001-000000001.tsm id=0 duration=0.161ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.173070Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/112 duration=202.633ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.174600Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/120/000000001-000000001.tsm id=0 duration=0.216ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.327240Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/120 duration=153.976ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.330492Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/128/000000001-000000001.tsm id=0 duration=0.722ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.511257Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/128 duration=182.950ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.512396Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/136/000000001-000000001.tsm id=0 duration=0.156ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.722425Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/136 duration=211.060ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.723713Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/144/000000001-000000001.tsm id=0 duration=0.168ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.898896Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/144 duration=176.316ms
Aug 27 21:56:22 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:22.903048Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/152/000000001-000000001.tsm id=0 duration=3.245ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.079075Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/152 duration=180.040ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.081342Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/16/000000001-000000001.tsm id=0 duration=0.130ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.276272Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/16 duration=195.894ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.279735Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/160/000000001-000000001.tsm id=0 duration=1.498ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.469802Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/160 duration=192.368ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.472005Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/168/000000001-000000001.tsm id=0 duration=0.173ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.670023Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/168 duration=199.007ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.671203Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/176/000000001-000000001.tsm id=0 duration=0.191ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.874646Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/176 duration=204.490ms
Aug 27 21:56:23 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:23.876984Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/184/000000001-000000001.tsm id=0 duration=0.186ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.117065Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/184 duration=241.034ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.119591Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/192/000000001-000000001.tsm id=0 duration=0.223ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.261219Z lvl=info msg="Reading file" log_id=0WEIn_e0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/700/_00112.wal size=10525498
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.295661Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/192 duration=177.310ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.296575Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/2/000000003-000000002.tsm id=0 duration=0.110ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.515913Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/2 duration=220.161ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.516955Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/200/000000001-000000001.tsm id=0 duration=0.171ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.680070Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/200 duration=164.029ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.720654Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/208/000000001-000000001.tsm id=0 duration=39.603ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.881928Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/208 duration=201.730ms
Aug 27 21:56:24 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:24.921120Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/216/000000001-000000001.tsm id=0 duration=38.182ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.086249Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/216 duration=204.195ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.106222Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/224/000000001-000000001.tsm id=0 duration=17.519ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.303412Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/224 duration=215.959ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.305621Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/232/000000001-000000001.tsm id=0 duration=0.216ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.484041Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/232 duration=179.524ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.485045Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/24/000000001-000000001.tsm id=0 duration=0.173ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.690814Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/24 duration=206.653ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.693024Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/240/000000001-000000002.tsm id=0 duration=0.192ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.897700Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/240 duration=205.688ms
Aug 27 21:56:25 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:25.899058Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/248/000000002-000000002.tsm id=0 duration=0.261ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.077891Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/248 duration=180.036ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.079195Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/256/000000001-000000001.tsm id=0 duration=0.218ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.280303Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/256 duration=202.292ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.281398Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/264/000000001-000000001.tsm id=0 duration=0.207ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.415845Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/264 duration=135.419ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.417963Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/272/000000001-000000001.tsm id=0 duration=0.191ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.606825Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/272 duration=189.856ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.608903Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/280/000000001-000000001.tsm id=0 duration=0.149ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.803226Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/280 duration=195.290ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.805474Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/288/000000004-000000002.tsm id=0 duration=0.134ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.875902Z lvl=info msg="Reading file" log_id=0WEIn_e0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/700/_00113.wal size=10525464
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.945109Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/288 duration=140.710ms
Aug 27 21:56:26 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:26.947375Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/296/000000001-000000001.tsm id=0 duration=0.188ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.049708Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/296 duration=103.425ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.050804Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/304/000000001-000000001.tsm id=0 duration=0.186ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.163160Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/304 duration=113.337ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.164684Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/312/000000001-000000001.tsm id=0 duration=0.165ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.287606Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/312 duration=124.309ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.288644Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/32/000000001-000000001.tsm id=0 duration=0.171ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.389928Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/32 duration=102.203ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.392301Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/320/000000001-000000001.tsm id=0 duration=0.201ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.492098Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/320 duration=100.856ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.494338Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/328/000000001-000000001.tsm id=0 duration=0.188ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.607728Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/328 duration=114.358ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.610607Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/336/000000001-000000001.tsm id=0 duration=0.976ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.732274Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/336 duration=124.431ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.733483Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/344/000000002-000000002.tsm id=0 duration=0.210ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.834314Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/344 duration=101.925ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.835464Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/352/000000001-000000001.tsm id=0 duration=0.210ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.948394Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/352 duration=113.977ms
Aug 27 21:56:27 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:27.949932Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/360/000000001-000000001.tsm id=0 duration=0.256ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.074418Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/360 duration=125.898ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.075777Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/368/000000001-000000001.tsm id=0 duration=0.178ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.187749Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/368 duration=113.200ms
2 
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.190273Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/376/000000001-000000001.tsm id=0 duration=0.271ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.301559Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/376 duration=112.574ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.303841Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/384/000000001-000000001.tsm id=0 duration=0.185ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.416464Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/384 duration=113.720ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.424550Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/392/000000001-000000001.tsm id=0 duration=1.423ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.572877Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/392 duration=152.292ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.640127Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/40/000000001-000000001.tsm id=0 duration=34.729ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.705645Z lvl=info msg="Reading file" log_id=0WEIn_e0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/700/_00114.wal size=1992818
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.819117Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/40 duration=245.047ms
Aug 27 21:56:28 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:28.882386Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/400/000000001-000000001.tsm id=0 duration=62.175ms
Aug 27 21:56:29 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:29.265015Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/400 duration=445.802ms
Aug 27 21:56:29 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:29.486662Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/408/000000001-000000001.tsm id=0 duration=153.285ms
Aug 27 21:56:29 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:29.727900Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/700 duration=8011.644ms
Aug 27 21:56:29 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:29.808209Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/416/000000001-000000001.tsm id=0 duration=78.540ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.126234Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/408 duration=860.234ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.237337Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/424/000000001-000000001.tsm id=0 duration=108.623ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.281753Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/416 duration=552.847ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.380851Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/432/000000001-000000001.tsm id=0 duration=97.587ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.580686Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/424 duration=453.059ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.701953Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/440/000000009-000000002.tsm id=0 duration=120.360ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.703189Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/432 duration=420.703ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.771563Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/448/000000001-000000001.tsm id=0 duration=67.256ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.874063Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/440 duration=293.305ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.913638Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/448 duration=210.391ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.924730Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/456/000000005-000000002.tsm id=0 duration=48.335ms
Aug 27 21:56:30 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:30.952097Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/464/000000001-000000001.tsm id=0 duration=37.623ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.178831Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/456 duration=303.436ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.216046Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/464 duration=302.352ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.264439Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/472/000000001-000000001.tsm id=0 duration=48.595ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.288734Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/48/000000001-000000001.tsm id=0 duration=38.831ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.485633Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/472 duration=305.874ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.526142Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/48 duration=309.281ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.548973Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/480/000000001-000000001.tsm id=0 duration=23.060ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.577761Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/488/000000001-000000001.tsm id=0 duration=32.659ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.834963Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/480 duration=349.230ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.871978Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/488 duration=345.781ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.883624Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/496/000000001-000000001.tsm id=0 duration=46.889ms
Aug 27 21:56:31 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:31.916278Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/504/000000001-000000001.tsm id=0 duration=43.443ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.097095Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/496 duration=261.124ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.120200Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/504 duration=248.164ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.142419Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/512/000000001-000000001.tsm id=0 duration=32.845ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.172434Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/520/000000001-000000001.tsm id=0 duration=34.426ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.368489Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/512 duration=271.287ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.407431Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/520 duration=287.180ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.421392Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/528/000000001-000000001.tsm id=0 duration=14.239ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.453279Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/536/000000001-000000001.tsm id=0 duration=36.481ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.617562Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/528 duration=248.986ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.656292Z lvl=info msg="Opened shard" log_id=0WEIn_e0000 service=store trace_id=0WEIna9G000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/database/autogen/536 duration=248.804ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.678200Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/544/000000001-000000001.tsm id=0 duration=21.674ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: ts=2021-08-27T19:56:32.703107Z lvl=info msg="Opened file" log_id=0WEIn_e0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/database/autogen/552/000000001-000000001.tsm id=0 duration=29.209ms
Aug 27 21:56:32 <name> influxd-systemd-start.sh[19976]: Failed to reach influxdb http endpoint at http://localhost:8086/health
Aug 27 21:56:32 <name> systemd[1]: influxdb.service: Control process exited, code=exited status=1
Aug 27 21:56:32 <name> systemd[1]: Failed to start InfluxDB is an open-source, distributed, time series database.
-- Subject: Unit influxdb.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit influxdb.service has failed.
--
-- The result is failed.
Aug 27 21:56:32 <name> systemd[1]: influxdb.service: Unit entered failed state.
Aug 27 21:56:32 <name> systemd[1]: influxdb.service: Failed with result 'exit-code'.
Aug 27 21:56:33 <name> systemd[1]: influxdb.service: Service hold-off time over, scheduling restart.
Aug 27 21:56:33 <name> systemd[1]: Stopped InfluxDB is an open-source, distributed, time series database.
-- Subject: Unit influxdb.service has finished shutting down
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit influxdb.service has finished shutting down.

Diagnostics:

# influx_inspect verify -dir /var/lib/influxdb
/var/lib/influxdb/data/_internal/monitor/692/000000029-000000002.tsm: healthy
[... healthy for every other one]
Broken Blocks: 0 / 407017, in 9.822059917s

# influx_inspect verify-seriesfile -dir /var/lib/influxdb
2021-08-27T19:45:29.608440Z     error   Series file does not exist      {"log_id": "0WEIAr20000", "path": "/var/lib/influxdb/data/_series"}
2021-08-27T19:45:29.611765Z     error   Series file does not exist      {"log_id": "0WEIAr20000", "path": "/var/lib/influxdb/meta/_series"}
2021-08-27T19:45:29.612444Z     error   Series file does not exist      {"log_id": "0WEIAr20000", "path": "/var/lib/influxdb/wal/_series"}

# influx_inspect report-disk  /var/lib/influxdb

Completed in 66.914133ms
{
  "Summary": {"shards": 95, "tsm_files": 102, "total_tsm_size": 150178679 },
  "Shard": [
    {"db": "_internal", "rp": "monitor", "shard": "692", "tsm_files": 1, "size": 6814468},
[... truncated]

What to do to get InfluxDB running again? I need my data and I am currently loosing new data :frowning:

3

Okay, found it myself - seems as I ran into https://github.com/influxdata/influxdb/issues/21967
Removed Type=forking from /etc/systemd/system/multi-user.target.wants/influxdb.service and finally my database is up again - just lost two hours of data.
(auth-enabled = false in my setup)

Thought quality management of the releases is a bit better :frowning:
Now I’m alerted for next time…

4

@StefanMUC,
Thanks for sharing the solution