Context Deadline Exceeded

1

I’ve read the other posts on this topic, and so far, no one has had an answer.

2021-07-09T13:49:49Z E! [agent] Error writing to outputs.influxdb_v2: Post "https://davidgs.com:8086/api/v2/write?bucket=telegraf&org=influxdata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

And in indeed, I have no data being written to InfluxDB. Suspecting a token issue, I made a new all-access token, yet the error persists.

Earlier in the log:

2021-07-09T13:52:01Z I! Loaded inputs: cpu disk diskio dovecot http_listener http_listener_v2 (2x) kernel mem mongodb mqtt_consumer mysql net netstat processes swap syslog system webhooks
2021-07-09T13:52:01Z I! Loaded aggregators:
2021-07-09T13:52:01Z I! Loaded processors:
2021-07-09T13:52:01Z I! Loaded outputs: file influxdb_v2
2021-07-09T13:52:01Z I! Tags enabled: host=whm.davidgs.com
2021-07-09T13:52:01Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"whm.davidgs.com", Flush Interval:10s
2021-07-09T13:52:01Z D! [agent] Initializing plugins
2021-07-09T13:52:01Z D! [agent] Connecting outputs
2021-07-09T13:52:01Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2021-07-09T13:52:01Z D! [agent] Successfully connected to outputs.influxdb_v2
2021-07-09T13:52:01Z D! [agent] Attempting connection to [outputs.file]
2021-07-09T13:52:01Z D! [agent] Successfully connected to outputs.file
2021-07-09T13:52:01Z D! [agent] Starting service inputs

So it can connect, it just won’t write to the database.

Other processes can write directly to InfluxDB on that host (I have, as you know, tons of sensors around that are all writing data directly to InfluxDBv2 without Telegraf).

This problem seems unique to telelgraf.

dg

2

I found this explanation of the error coming from Golang, it seems to be intentionally triggered when some part of the code (probably a network call, given the Client.Timeout) is taking too long to complete.

Some StackOverflow answers have suggested that a DNS failure could cause this. Can you verify that you can resolve davidgs.com from your telegraf’s host?

3

The fact that the log shows this:

2021-07-09T13:52:01Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2021-07-09T13:52:01Z D! [agent] Successfully connected to outputs.influxdb_v2

Would indicate that it can. The telegraf host is also davidgs.com so one would think …

dg

4

It’s the SSL configuration. My influxDB instance is running as an SSL-enabled service, using a cert-file and key-file. So I told telegraf to also connect over SSL< using the same cert-file and key-file. And errors ensued. When I switch the hostname to localhost I of course get errors that the cert- and key-files are only valid for the domain davidgs.com, and not localhost.

The fix seems to be to use localhost and then use insecure_skip_verify = true which, though annoying and not entirely secure, at least works.

dg

1 Like