Hello,
I have some problem to start my influxdb 1.11.8 running on a Debian VM. It looks like that I have some invalid series index error:
× influxdb.service - InfluxDB is an open-source, distributed, time series database
Loaded: loaded (/lib/systemd/system/influxdb.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since Sun 2025-08-03 17:47:08 CEST; 2min 37s ago
Docs: https://docs.influxdata.com/influxdb/
Process: 656 ExecStart=/usr/share/influxdb/scripts/influxd-systemd-start.sh (code=exited, status=1/FAILURE)
CPU: 55ms
Aug 03 17:47:08 debian systemd[1]: influxdb.service: Control process exited, code=exited, status=1/FAILURE
Aug 03 17:47:08 debian systemd[1]: influxdb.service: Failed with result 'exit-code'.
Aug 03 17:47:08 debian systemd[1]: Failed to start influxdb.service - InfluxDB is an open-source, distributed, time series database.
Aug 03 17:47:08 debian systemd[1]: influxdb.service: Scheduled restart job, restart counter is at 5.
Aug 03 17:47:08 debian systemd[1]: Stopped influxdb.service - InfluxDB is an open-source, distributed, time series database.
Aug 03 17:47:08 debian systemd[1]: influxdb.service: Start request repeated too quickly.
Aug 03 17:47:08 debian systemd[1]: influxdb.service: Failed with result 'exit-code'.
Aug 03 17:47:08 debian systemd[1]: Failed to start influxdb.service - InfluxDB is an open-source, distributed, time series database.
root@debian:~# influx -version
InfluxDB shell version: v1.11.8
root@debian:~# sudo influxd
8888888 .d888 888 8888888b. 888888b.
888 d88P" 888 888 "Y88b 888 "88b
888 888 888 888 888 888 .88P
888 88888b. 888888 888 888 888 888 888 888 888 8888888K.
888 888 "88b 888 888 888 888 Y8bd8P' 888 888 888 "Y88b
888 888 888 888 888 888 888 X88K 888 888 888 888
888 888 888 888 888 Y88b 888 .d8""8b. 888 .d88P 888 d88P
8888888 888 888 888 888 "Y88888 888 888 8888888P" 8888888P"
2025-08-03T13:51:59.255444Z info InfluxDB starting {"log_id": "0y835y5l000", "version": "v1.11.8", "branch": "unknown", "commit": "199a607e1c7a5a687f96a636bb3bab9a61e4ae31"}
2025-08-03T13:51:59.255494Z info Go runtime {"log_id": "0y835y5l000", "version": "go1.22.7", "maxprocs": 2}
2025-08-03T13:51:59.255542Z info configured logger {"log_id": "0y835y5l001", "format": "auto", "level": "info"}
2025-08-03T13:51:59.257862Z info Using data dir {"log_id": "0y835y5l000", "service": "store", "path": "/var/lib/influxdb/data"}
2025-08-03T13:51:59.257925Z info Compaction settings {"log_id": "0y835y5l000", "service": "store", "max_concurrent_compactions": 1, "throughput_bytes_per_second": 50331648, "throughput_bytes_per_second_burst": 50331648}
2025-08-03T13:51:59.257973Z info Open store (start) {"log_id": "0y835y5l000", "service": "store", "trace_id": "0y835y6G000", "op_name": "tsdb_open", "op_event": "start"}
2025-08-03T13:51:59.258105Z error Unable to open series file {"log_id": "0y835y5l000", "path": "/var/lib/influxdb/data/_internal/_series", "partition": 0, "error": "invalid series index"}
2025-08-03T13:51:59.258163Z info Open store (end) {"log_id": "0y835y5l000", "service": "store", "trace_id": "0y835y6G000", "op_name": "tsdb_open", "op_event": "end", "op_elapsed": "0.191ms"}
run: open server: open tsdb store: invalid series index
root@debian:~# influx_inspect verify-seriesfile -dir /var/lib/influxdb/data
2025-08-03T13:56:04.003792Z error Error opening index {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/_internal/_series", "partition": "00", "error": "invalid series index"}
2025-08-03T13:56:04.003900Z error Error opening segment {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/_internal/_series", "partition": "02", "segment": "0000", "error": "invalid series segment"}
2025-08-03T13:56:04.003993Z error Error opening segment {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/_internal/_series", "partition": "03", "segment": "0000", "error": "invalid series segment"}
2025-08-03T13:56:04.004977Z error Panic parsing key {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/_internal/_series", "partition": "04", "segment": "0000", "key": "db01000874736d315f77616c0800086461746162617365000867726170686974650006656e67696e650004747370650005696e6d656d000470617468002a2f7661722f64622f696e666c757864622f646174612f67726170686974652f6175746f67656e2f323532000f726574656e74696f6e506f6c69637900076175746f67656e000777616c5061746800292f7661722f64622f696e666c757864622f77616c2f67726170686974652f6175746f67656e2f32353201000000000000056be101000e74736d315f66696c6573746f7265080008646174616261736500", "offset": 4042, "recovered": "runtime error: slice bounds out of range [256:37]"}
2025-08-03T13:56:04.006609Z error Unable to advance buffer {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/graphite/_series", "partition": "04", "segment": "0000", "offset": 285, "error": "unable to advance 60 bytes: 19 remaining"}
2025-08-03T13:56:04.007005Z error Index inconsistency {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/graphite/_series", "partition": "06", "id": 41, "got_deleted": true, "expected_deleted": false}
2025-08-03T13:56:04.008229Z error Error opening segment {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/telegraf/_series", "partition": "00", "segment": "0000", "error": "invalid series segment"}
2025-08-03T13:56:04.009270Z error Index inconsistency {"log_id": "0y83Ku2l000", "path": "/var/lib/influxdb/data/telegraf/_series", "partition": "06", "id": 47, "got_deleted": true, "expected_deleted": false}
When I do a
root@debian:~# influx_inspect verify -dir /var/lib/influxdb/data
/var/lib/influxdb/data/_internal/monitor/1/000000001-000000001.tsm: healthy
....
/var/lib/influxdb/data/telegraf/autogen/1749/000000002-000000002.tsm: healthy
/var/lib/influxdb/data/telegraf/autogen/1758/000000001-000000001.tsm: healthy
Broken Blocks: 0 / 2490073, in 10.052598455s
it looks ok.
Don’t know how this could happen. How can I fix this? Is there any repair possibility?
