Hi,
I am trying to run telegraf inside a docker container to monitor the host system metrics and have mounted the hosts file system into the container as suggested in the documentation.
The CPU, memory, disk plugins are correctly collecting metrics for the host system, but the net plugin is collecting metrics about the container not the host. This can be seen by running telegraf on the host system and comparing the metrics.
Is this expected or am I missing some config, there was nothing suggested in the docs for the plugin?
This is my docker compose config:
telegraf-system:
image: docker.io/library/telegraf:1.21.4
container_name: telegraf-system
restart: always
privileged: true
volumes:
- ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
- /var/run/docker.sock:/var/run/docker.sock
- /:/hostfs:ro
- /run/udev:/run/udev:ro
depends_on:
- "influxdb"
environment:
- HOST_MOUNT_PREFIX=/hostfs
- HOST_ETC=/hostfs/etc
- HOST_PROC=/hostfs/proc
- HOST_SYS=/hostfs/sys
- HOST_VAR=/hostfs/var
- HOST_RUN=/hostfs/run
Hello @jamesgibo ,
I found this:
opened 12:25PM - 01 Dec 16 UTC
closed 12:55PM - 01 Dec 16 UTC
## description
Now telegraf is running inside docker.I set input config to coll… ect net, processor measurements, but the net and processors measurements collected is the docker container's measurements which run telegraf. In fact, my mind is to collect that measurements on the host which the telegraf docker is running on.How can I fix it?
## my telegraf input conf
```
# Get the number of processes and group them by status
[[inputs.processes]]
# no configuration
# Read metrics about swap memory usage
[[inputs.swap]]
# no configuration
# Read metrics about system load & uptime
[[inputs.system]]
# no configuration
# Read metrics about network interface usage
[[inputs.net]]
## By default, telegraf gathers stats from any up interface (excluding loopback)
## Setting interfaces will tell it to gather these explicit interfaces,
## regardless of status.
##
interfaces = ["eth0", "eth1"]
```
Specifically,
NOTE: It doesn’t work for netstat. Telegraf’s inputs.netstat
plugin can report values only for the container. It cannot report on the host. It does not use the HOST_PROC
environmnet variable. This is because the plugin uses the gopsutil library which executes the lsof
command:
https://github.com/shirou/gopsutil/blob/a3b23c5ccf4fb7b33d319fcaad53d7777907f4e1/internal/common/common_unix.go#L12
And lsof
always opens /proc
. The string is hard-coded:
https://github.com/lsof-org/lsof/blob/e4bc415f0c4659af44ed4a2ef69f87e88a8c6a0a/dialects/linux/dproc.c#L297
Thanks for the response, that’s a shame. As the network metrics are key for monitoring my application I will run the telegraf app natively.
Might be worth adding a note about that to the inputs.netstat
plugin docs.
The problem is that inputs.net uses /proc/net/dev for data and this is confined to the telegraf process itself when it runs as a container. To get access to the host’s /proc/net, that needs to be specifically mounted inside the container rather than just /proc as /hostfs.
Ah, I just found a solution:
spec:
...
template:
...
spec:
...
volumeMounts:
- name: procnet
mountPath: /hostfs/proc/net
...
volumes:
-name: procnet
hostPath:
path: /proc/net
type: ''