Telegraf not sending data to influxdb

1

Hello,
i have installed telegraf on a machine and tried to send data to influxdb which is on other machine
i checked version of influxdb it is 1.2.2
telegraf version is 1.2.1
but am facing the problem with telegraf not sending data to influxdb and shows error message as below

2017-04-17T09:40:35Z E! Database creation failed: Post http://52.170.223.20:8086/query?db=&q=CREATE+DATABASE+"metrics": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

2

@hema This is the error message when telgraf cannot reach InfluxDB. This is most likely due to a firewall at your cloud provider. I would check there first.

3

Hello,
even after checking the firewall, telegraf is not able to reach influxdb and shows the same error

4

@hema This is likely not an InfluxDB or Telegraf issue. Can you curl the InfluxDB instance from the telegraf instance?

5

FWIW, some of my hosts report this sometimes, but the error usually goes away fairly soon afterwards. In my case I think Influx stops responding for a few seconds (maybe system resource problems - Iā€™m looking into it).

Things to check are:

curl -kI https://your_influx_server_name:8086/ping
(you should see something like ā€œHTTP/1.1 204 No Contentā€ in the response)

If thatā€™s working, then you have TCP connectivity and Influx is running. If Telegraf is still not sending, then maybe check things like usernames and passwords and make sure Telegraf can authenticate correctly.

1 Like
6

Some network troubleshooting is needed before digging into the app. If you have access to you influx server do some basic checks. (assuming linux)

Make sure your daemon is up and running on the ports you expect

netstat -ntlp
or
ss -ntlp
You should see some line simialr to
tcp6 0 0 :::8086 :::* LISTEN 6501/influxd

The ss command will provide something similar but more verbose.
If you see that your server should accept connections. Jump over to the client and run

telnet <influx_server> 8086
you should see something like

Trying <influx_server>ā€¦
Connected to <influx_server>.
Escape character is ā€˜^]ā€™.

You can Ctl ] to escape the socket connection or enter ā€œGET /*ā€ (less the ā€œā€) and you will get a HTTP error msg.
What this proves is you can connect to the influxdb web interface. And you can start looking at influx config issues.
If you donā€™t get a connection or it times out. The port is blocked. As mentioned earlier look for a firewall.
If you are sure itā€™s not firewall it could be SELINUX getting in the way. Edit the /etc/selinux/config file (or your distroā€™s equivalent) and disable SELINUX .

SELINUX=disabled

SELINUXTYPE=targeted

Reboot and try to connect again. If that is still not working for you. Then you will need to see if the packets are even getting to the port (tcpdump). Chances are firewall or selinux will do the trick for you.

Cheers

1 Like
7

@coofercat We have this exact problem. Telegraf can successfully authenticate and send metrics, but fail seemingly random. Did you find a cause?

@Karl_Raffelsieper Weā€™re not using SELinux and metrics make it all the way into InfluxDB until they donā€™t. InfluxDB is receiving traffic properly from other devices with Telegraf instances on same config. Iā€™m looking into getting TCP dumps- any suggestions on particulars to dump?

8

Hi,
I am trying to run Jmeter Load test and do the Live monitoring using Grafana_influxDB.
But facing issue while connecting to influxdb getting below error in JMeter .Looks like timeout observed while writing to InfluxDB .Very records get inserted into DB but hen timeouts are observed .Please help to improve Influxdb configuration to avoid these Timeouts.

My Influxdb version - 1.7.5
Error in Jmeter - failed to send data to influxDB server : 3,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]

9

Hi ,
We are also facing the same issue. Tried changing the time out to 5000 or more but still we get this exception randomly.Any thoughts ?