[solved] Telegraf inputs.disk collection empty

Telegraf
1

Just getting an empty collection from the [[inputs.disk]] plugin.
telegraf is running in a docker container.

Telegraf v1.3.5 (git: release-1.3 7192e68b2423997177692834f53cdf171aee1a88)

root@99a3dda91f0e:~# cat /etc/telegraf/telegraf.conf
[global_tags]
  host = "$HOSTNAME"

# Configuration for telegraf agent
[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  debug = false
  quiet = true
  logfile = ""
  hostname = ""
  omit_hostname = false
[[outputs.file]]
  files = ["stdout"]
[[inputs.diskio]]
[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs"]
[[inputs.kernel]]
root@99a3dda91f0e:~# telegraf --config /etc/telegraf/telegraf.conf --input-filter disk --test --debug
* Plugin: inputs.disk, Collection 1
root@99a3dda91f0e:~#

Also have a look at telegraf disk-input does not write to output in phusion/baseimage - Stack Overflow, which is a similar problem on the same infrastructure.

2

Just tested again with a default config of telegraf:

root@99a3dda91f0e:~# echo $HOST_MOUNT_PREFIX 
docker-host-fs
root@99a3dda91f0e:~# mountpoint /docker-host-fs
/docker-host-fs is a mountpoint
root@99a3dda91f0e:~# telegraf config > /etc/telegraf/telegraf.conf 
root@99a3dda91f0e:~# telegraf --config /etc/telegraf/telegraf.conf --input-filter disk --test 
* Plugin: inputs.disk, Collection 1
root@99a3dda91f0e:~#

//edited:
Don’t know what’s going on here, but with this config on some containers (deployed in swarm mode with mode: global) the output is sometimes empty and sometimes it contains collected metrics:

[global_tags]
  host = "$HOSTNAME"
  dockerhost = "$DOCKERHOSTNAME"

# Configuration for telegraf agent
[agent]
  interval = "1s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "100s"
  flush_jitter = "0s"
  precision = ""
  debug = true
  quiet = false
  logfile = ""
  hostname = ""
  omit_hostname = false

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"] # required
  ## The target database for metrics (telegraf will create it if not exists).
  database = "$INFLUX_DATABASE"

  ## Name of existing retention policy to write to.  Empty string writes to
  ## the default retention policy.
  retention_policy = ""
  ## Write consistency (clusters only), can be: "any", "one", "quorum", "all"
  write_consistency = "any"

  ## Write timeout (for the InfluxDB client), formatted as a string.
  ## If not provided, will default to 5s. 0s means no timeout (not recommended).
  timeout = "5s"
  # username = "telegraf"
  # password = "metricsmetricsmetricsmetrics"
  ## Set the user agent for HTTP POSTs (can be useful for log differentiation)
  # user_agent = "telegraf"

[[outputs.file]]
  files = ["stdout"]
[[inputs.disk]]
3

Does this comment on the Telegraf issue tracker,which describes how to setup a docker container to collect disk metrics from the host help?

4

Just found that and commented HOST_MOUNT_PREFIX not working as intended · Issue #2811 · influxdata/telegraf · GitHub last week. With the workaround there and the official telegraf docker image everything works as expected.

5

@daniel There are several solutions and workarounds about that. Are you planing to generalize the things about /hostfs or /rootfs?

It seems that we need this for several inputs, like procs, disk, …
There should be a generic way to mount the host-filesystem and set the env vars.

6

how can I set this to resolved?

7

Right now we are not sure how it is all supposed to work, the workaround might be right or only just a way to get it working. I need to take some time and look through all the environment variables and the docker input plugin to determine how they are meant to be used and provide some documentation. I’ll update that issue on github when I do this.

I don’t think there is a way to mark a question resolved on this site.

1 Like