[agent] Error writing to outputs.influxdb: could not write any address

Telegraf
1

I am using telegraf on a raspberry pi to write to an influxdb on another machine (running grafana) and lately I started seeing these messages:

    E! [outputs.influxdb] When writing to [http://192.168.139.20:8086]: failed doing req:
     Post "http://192.168.139.20:8086/write?db=telegraf": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

It doesn’t happen all the time so I assume maybe due to the infludb machine handling a lot of data? Do I need to increase the timeout somewhere is there a bigger issue?

Here’s my telegraf config (the relevant part anyway):

      # Configuration for sending metrics to InfluxDB
      [[outputs.influxdb]]
        ## The full HTTP or UDP URL for your InfluxDB instance.
        ##
        ## Multiple URLs can be specified for a single cluster, only ONE of the
        ## urls will be written to each interval.
        # urls = ["unix:///var/run/influxdb.sock"]
        # urls = ["udp://127.0.0.1:8089"]
         urls = ["http://192.168.139.20:8086"]

        ## The target database for metrics; will be created as needed.
         database = "telegraf"

        ## If true, no CREATE DATABASE queries will be sent.  Set to true when using
        ## Telegraf with a user without permissions to create databases or when the
        ## database already exists.
         skip_database_creation = true

        ## Name of existing retention policy to write to.  Empty string writes to
        ## the default retention policy.  Only takes effect when using HTTP.
        # retention_policy = ""

        ## Write consistency (clusters only), can be: "any", "one", "quorum", "all".
        ## Only takes effect when using HTTP.
        # write_consistency = "any"

        ## Timeout for HTTP messages.
        # timeout = "5s"

        ## HTTP Basic Auth
        # username = ""
        # password = ""

        ## HTTP User-Agent
        # user_agent = "telegraf"

        ## UDP payload size is the maximum packet size to send.
        # udp_payload = 512

        ## Optional TLS Config for use on HTTP connections.
        # tls_ca = "/etc/telegraf/ca.pem"
        # tls_cert = "/etc/telegraf/cert.pem"
        # tls_key = "/etc/telegraf/key.pem"
        ## Use TLS but skip chain & host verification
        # insecure_skip_verify = false

        ## HTTP Proxy override, if unset values the standard proxy environment
        ## variables are consulted to determine which proxy, if any, should be used.
        # http_proxy = "http://corporate.proxy:3128"

        ## Additional HTTP headers
        # http_headers = {"X-Special-Header" = "Special-Value"}

        ## HTTP Content-Encoding for write request body, can be set to "gzip" to
        ## compress body or "identity" to apply no encoding.
        # content_encoding = "identity"

        ## When true, Telegraf will output unsigned integers as unsigned values,
        ## i.e.: "42u".  You will need a version of InfluxDB supporting unsigned
        ## integer values.  Enabling this option will result in field type errors if
        ## existing data has been written.
        # influx_uint_support = false
2

Hi @junior466,
What version of Telegraf and InfluxDB are you using? Is InfuxDB also running on a PI? if not what are the specs of this system. What is your average data throughput do you think (number of metrics, how often)?

Thanks,
Jay

3

I am running Telegraf 1.21.2 (on the Pi) and InfluxDB shell version: 1.8.10 on an Ubuntu 18.04 VM (ESXi) with 1 vCPU and 2GB or RAM. I am not sure on the number of metrics but I am monitoring around 20 VMs through vCenter and a few devices.

Again, it doesn’t happen often.

4

So it could be the case that the system just spikes from a number of tasks going off at the same time. I would monitor your system resources for the InfluxDB host and see if you notice any patterns. To help with the issue what you could try is increasing the timeout setting within the InfluxDB plugin. Try 10s and see if this has any effect.

1 Like