Why am I suddenly unable to connect to InfluxDB?

1

Hi,
I’m using InfluxDB 1.8.4 and Grafana through docker for months without any problem until two days ago. Suddenly I can’t access InfluxDB. The error I get is:

Failed to connect to http://localhost:8086: Get http://localhost:8086/ping: EOF
Please check your connection settings and ensure 'influxd' is running.

Docker is running, it’s checked. Some tests I did were to restart docker, change the port, run InfluxDB without docker, move the data files to another unit, and finally try it without the databases (all empty).

When I executed a verify command I could see that all the files were healthy except one that indicates: \InfluxDB\data_internal\monitor\976\000000030-000000001.tsm: could not get checksum for key block 988 due to error: “tsm file closed”

If I try without old databases it works but then I lose all my data collected for months.

Then I deleted the corrupted file but the EOF error persisted. However, now if I run the verify tool there are no broken blocks.

So I was creating backup copies of each one of the databases separately and I was importing them one by one to see if any of them failed. However, none failed and I was able to connect, but after a few seconds or minutes I can’t connect again.

As I was displaying data with the CLI tool once I can’t connect I got the following error:
ERR: Post http://localhost:8086/query?chunked=true&db=&q=show+databases: EOF

Any idea what might be causing this problem?

Thanks in advance

2

It seems there is no solution for this problem. unfortunately access to InfluxDB keeps failing (EOF error) after a few minutes or a couple of days. Perhaps it is that this tool is not reliable for projects in production and it is time to think about changing the tool?

3

@pacogam

About a year ago, I was using v1.8 on an AWS EC2 instance. One day Grafana could not display the data collected in Influx, and I recall getting the same “Failed to connect” error when using the terminal window to access InfluxDB. I restarted it and it would connect for a while (maybe an hour or so, but later only minutes), then again fail to connect.

The problem in the end was the CPU and/or storage size that I had allocated on the EC2 instance. Enlarging the CPU and/or storage (I cannot remember which…perhaps it was both) made the problem go away.

1 Like
4

I’m writing this down in case it helps anyone else. I had InfluxDB running for months on a HDD, from the time it first failed, it failed even with an empty DBD, so I couldn’t figure out where the problem was. It was fixed when I ran InfluxDB on an SSD, it hasn’t happened since.