Outputs.file plugin in docker container cannot write to mapped volume on host

I’m two days new to Telegraf and, as a test case, want to write the default CPU metrics to a permanent file on the host defined as files = [“/tmp/metrics.out”] in telegraf.conf. When there is no volume mapping of this file in the docker-compose.yaml, it runs fine inside the container. The telegraf log shows normal startup with no errors, and inside the container I can ‘cat metrics.out’ and watch the data accumulating. ls -la shows the UID:GID as telegraf:root. Curious, that it’s not root:root.

But when I add a volume mapping in the docker-compose file, let’s say ’ - /tmp:/tmp:rw’ I get an error on open metrics.out permission denied. And if the mapping is very specific, ’ - /tmp/metrics.out:/tmp/metrics.out:rw’ I get an error on open metrics.out is a directory. What’s more: This actually creates a new directory on the host with the name metrics.out! But . . . if I create the file on the host ‘touch metrics.out’ and set the permissions wide-open ‘chmod go+w metrics.out’ the error changes back to permission denied. Finally, just for fun I set the volume mapping to point to my telegraf.conf - the one telegraf uses to startup. In this case data is actually appended to that file.

Thinking it might be a UID:GID issue - mismatch between host and container, maybe. But something this trivial should work “right out of the box”’ and it’s weird that it does work if the output file is set equal to the config file but in none of these other attempts. What am I missing?

Hi @phill,
I think you are correct it’s a permissions mapping issue. This could be that docker itself doesn’t have the relevant authority to write data within these directories on the host. Can we try the following instead: Telegraf-Community-Configs/docker/writing-logs-to-host at master · InfluxCommunity/Telegraf-Community-Configs · GitHub

With regards to your system could we try the doomed command chmod 777 container_stats. We can then remove permissions from there.

Thanks @Jay_Clifford. It’s working, and here’s what I learned. There must be a metrics.out file on the host and it must be owned by root (UID:GID = root:root) and the permissions must be wide open (sudo chmod go+w metrics.out). The host file can be empty, but it must exist, else a directory with the same name is created as described above and execution stops with error. Whatever is opening the file must be assuming it already exists. BTW I’m running Debian on a RPi 4.

I conducted a methodical investigation only changing one thing at a time and re-running to see the effect. Maybe the file handler that initiates the connection to the input file needs to check for file existence and have more logic when it doesn’t exist. I run other containers (e.g. home assistant) that create directories, files and files within directories where none existed before, so I don’t think it’s a docker issue.

1 Like

Hi Phill,
Noted, I am glad you got it up and running. I disagree slightly with the statement that the host metrics file must exist. The file output plugin autogenerates the metrics file to be written to if it does not already exist. I agree that the directories must exist first however as the plugin will not do this.

Personally, I would consider handling the permission at the host directory level rather than mounting the metrics file directly. This leaves you room to expand with further file output plugins in the future.

If you would like to contribute to the project and add further logic to the file plugin then we more than welcome the code improvements. :slight_smile:

@Jay_Clifford On the host all containers have their own permanent data directory in /mnt (i.e., /mnt/telegraf, /mnt/home assistant, etc) and those directories are root:root owned with permissions ‘drwxr-xr-x’, so no write permission for group & others.

The docker-compose volume maps the directory, rather than the metrics file. That is,
volumes:

  • /mnt/telegraf:/tmp
    but again, the file metrics.out must pre-exist otherwise metrics.out is created but as directory with owner root:root! And the telegraf log inside the container says ‘error metrics.out is a directory’. I haven’t tried setting the /mnt/telegraf directory permissions to write for group and others and running without a pre-existing file.

Hope that addresses your suggestion.

Hi @phill,
Can you please send along your docker-compose file and the version of Telegraf that you are using? This seems to be an issue on Debian that we need to investigate. As I cannot reproduce this issue on my current system.

The plugin works as intended by creating a metrics.out a file within the mounted host directory. My current OS is MacOS.

@Jay_Clifford Here ya go . . .
Using telegraf v1.23.0

---
version: "3"
services:
  telegraf:
    image: telegraf:latest
    container_name: telegraf
    volumes:
      #  Sync timezone with host
      - /etc/localtime:/etc/localtime:ro
      #  Map Telegraf configuration file
      - /mnt/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
      #  Map /tmp to permanent storage  (this handles file /tmp/metrics.out)
      - /mnt/telegraf:/tmp:rw
    restart: unless-stopped

2 Likes