Running telegraf as a service - Not loading config files in subtree

Telegraf
1

Hi!

I got telegraf running, but as it shows here with only one input:

[root@arys telegraf.d]# systemctl status telegraf
● telegraf.service - Telegraf
   Loaded: loaded (/usr/lib/systemd/system/telegraf.service; enabled; vendor preset: disabled)
   Active: active (running) since lun 2024-07-15 16:22:42 -03; 5s ago
     Docs: https://github.com/influxdata/telegraf
 Main PID: 37606 (telegraf)
   CGroup: /system.slice/telegraf.service
           └─37606 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d/

jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Starting Telegraf 1.32.0-7417f686 brought to you by InfluxData the makers of InfluxDB
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 61 ...et-stores
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Loaded inputs: ping
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Loaded aggregators: derivative
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Loaded processors: converter strings
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Loaded secretstores:
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Loaded outputs: influxdb
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! Tags enabled:
jul 15 16:22:42 arys telegraf[37606]: 2024-07-15T19:22:42Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:1m0s
jul 15 16:22:42 arys systemd[1]: Started Telegraf.

The problem is that is not reading all the conf files that are there.

When I run it from the linux prompt I got this:

[root@arys telegraf.d]# telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d/
WARN[0000]log.go:244 gosnowflake.(*defaultLogger).Warn DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null.
2024-07-15T19:22:15Z I! Loading config: /etc/telegraf/telegraf.conf
2024-07-15T19:22:15Z I! Loading config: /etc/telegraf/telegraf.d/AGREGATOR_DERIVATIVE.conf
...
2024-07-15T19:22:16Z I! Starting Telegraf 1.32.0-7417f686 brought to you by InfluxData the makers of InfluxDB
2024-07-15T19:22:16Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 61 outputs, 6 secret-stores
2024-07-15T19:22:16Z I! Loaded inputs: ping snmp (250x)
2024-07-15T19:22:16Z I! Loaded aggregators: derivative
2024-07-15T19:22:16Z I! Loaded processors: converter strings
2024-07-15T19:22:16Z I! Loaded secretstores:
2024-07-15T19:22:16Z I! Loaded outputs: influxdb
2024-07-15T19:22:16Z I! Tags enabled:
2024-07-15T19:22:16Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:1m0s
^C2024-07-15T19:22:27Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-07-15T19:22:27Z I! [agent] Stopping running outputs

All my 251 inputs.
Should I have to put all my conf files in the same directory?
Why the daemon is not reading the directory tree?
Maybe I am missing a flag in the command line?

This is my service file:

[Unit]
Description=Telegraf
Documentation=https://github.com/influxdata/telegraf
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
NotifyAccess=all
EnvironmentFile=-/etc/default/telegraf
User=telegraf
ImportCredential=telegraf.*
ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d/ $TELEGRAF_OPTS
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartForceExitStatus=SIGPIPE
KillMode=mixed
LimitMEMLOCK=8M:8M
PrivateMounts=true

[Install]
WantedBy=multi-user.target

Regards!

2

Hi!

When you run telegraf directly as you did, it uses the permissions of the user, in this case root.
This might be a permission issue. It´s important to understand if the .conf files tree can be read by the user telegraf.
You can test using:
sudo -u telegraf telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d/

Or, if you want to test the service:
Change User=telegraf to User=root inside the service file
then run systemctl daemon-reload and start the process.
Keep in mind that this is not advised and there are security concerns.

Hope it helps

1 Like
3

I would also add please checkout the full logs via journalctl --no-pager --unit telegraf and see what files it thinks it tries loading.

4

Thanks to both!!
My bad!

I failed to check properly execution permits on subdirectories.

Once corrected it works perfectly.