Can't start InfluxDB!

InfluxDB 1
1

Hi,

Had to reboot system for patching. It was a clean shutdown.
Now InfluxDB can’t start properly. Running 1.8.5 on latest CentoOS 7. Upgraded to 1.8.9, same result.

It starts to load, but quickly gets to a point where this starts happening repeatedly:

influxd[28690]: /go/src/github.com/influxdata/influxdb/tsdb/store.go:363 +0x50
influxd[28690]: created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
influxd[28690]: /go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c8a
influxd[28690]: goroutine 8512 [chan send]:
influxd[28690]: github.com/influxdata/influxdb/pkg/limiter.Fixed.Take(...)
influxd[28690]: /go/src/github.com/influxdata/influxdb/pkg/limiter/fixed.go:40
influxd[28690]: github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc000b136e0, 0xc0007b5200, 0xc000ae8a80, 0xc000b13740, 0x1687600, 0xc01f974510, 0xc008d219f0, 0xc001257a75, 0x8, 0xc0620e4bee, ...)

I’ve used influx_inspect verify to check for issues with TSM and TSI files, but zero issues found.
I run another instance in parallel, with the exact same data, hardware, OS, and it rebooted and is operating normally. Can’t figure out what happened on this one.

Any hints for root cause or possible resolution without nuking it?

Thanks

2

It is a known problem starting from version 1.8.7.
The bug is introduced from 1.8.7 until 1.8.9.

The latest working version is 1.8.6. You may refer to this post:

1 Like