InfluxDB Docker stuck in kernel call - Can not be killed anymore, not even with kill -9

1

We have an InfluxDB docker instance running on our redhat host OS.

That instance currently can’t be killed anymore not even with kill -9, so the process must be stuck in a blocking kernel call. (See: process - What if 'kill -9' does not work? - Unix & Linux Stack Exchange)

The block happened at the end of the startup phase. So our last log messages are that a lots of shards were loaded (tsi1 indexed data), but not a single request was logged yet (our metrics should usually write at last every 10 seconds data and writes would be visible in the logs).

As base, we use influxdb:1.7 image.

Any idea on what could have happend? The container is currently still stuck, as the machine is quite important and the reboot is scheduled for somewhere the next week, but I have no idea, if we can debug something in that situation?

Best regards
Theo

2

Can you show me the output of ps aux | grep influxd, please?

Thanks

3

Bild%20eingef%C3%BCgt%20am%202019-11-29%2010-55
Bild%20eingef%C3%BCgt%20am%202019-11-29%2010-55728×85 14.8 KB

Note that, in order to don’t reboot the machine, I restarted the docker service and created a new instance of influx. That’s why there are two processes.

4

Hi,

Unfortunately, that D means there’s very little that can be done.

Do you have the logs from that process and that may shed some light?

My concern is that another process is now working on the data directory, so that may be further blocking any waiting syscalls.