Agent/Plugin startup clarification

icvdok · October 18, 2021, 7:16am

Dear community, I’m a noob on Telegraf and InfluxDb therefore sorry in advance if my question sounds stupid :-).
I’m installed Influxdb 2.0 on a docker. I’ve then created a telegraf configuration (load data>telegraf>create configuration) where the setup instruction provide two string (certificate and startup string) to activate the agents on a target server.
I’ve installed Telegraf on a target server and started the agents with the provided string where the conf file is centralized. telegraf --config http://192.168.1.105:8086/api/v2/telegrafs/xxxxxxxxxxxx
The data are collected correctly on by my bucket. I would like now to automatically startup the telegraf agent on boot with this parameter (centralized config file) but I didn’t find any documentation or valid solution.
Usually the agent start with a local config file according to the documentation.

Thanks in advance for the support.

Jay_Clifford · October 18, 2021, 9:42am

hi @icvdok,
Not a dumb question at all! Also welcome to the community. I will make sure to get this added to the docs. Have you tried editing the Telegraf service:

sudo nano /lib/systemd/system/telegraf.service

Then modifying the service like so:

[Service]
EnvironmentFile=-/etc/default/telegraf
User=telegraf
ExecStart=/usr/bin/telegraf -config http://192.168.1.105:8086/api/v2/telegrafs/xxxxxxxxxxxx -$TELEGRAF_OPTS
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartForceExitStatus=SIGPIPE
KillMode=control-group

Let me know how you get on

icvdok · October 18, 2021, 11:21am

Hello Jay, thanks for the welcome and the reply.

There is no telegraf.service on the system folder, here the output of the systemctl command.

ubuntu@ubuntu:/$ systemctl status telegraf.service
● telegraf.service - The plugin-driven server agent for reporting metrics into InfluxDB
Loaded: loaded (/lib/systemd/system/telegraf.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-10-18 13:03:22 CEST; 16min ago
Docs: GitHub - influxdata/telegraf: The plugin-driven server agent for collecting & reporting metrics.
Process: 5681 ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d $TELEGRAF_OPTS (code=exited, status=1/FAILURE)
Main PID: 5681 (code=exited, status=1/FAILURE)
.

icvdok · October 18, 2021, 1:53pm

ubuntu@ubuntu:/etc/telegraf$ systemctl status telegraf.service
● telegraf.service - The plugin-driven server agent for reporting metrics into InfluxDB
Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Mon 2021-10-18 15:41:21 CEST; 7min ago
Docs: GitHub - influxdata/telegraf: The plugin-driven server agent for collecting & reporting metrics.
Process: 6946 ExecStart=/usr/bin/telegraf -config http://192.168.1.105:8086/api/v2/telegrafs/084bc78a1facc000 -$TELEGRAF_OPTS (code=exited, status=0/SUCCESS)
Main PID: 6946 (code=exited, status=0/SUCCESS)

Oct 18 15:41:21 ubuntu telegraf[6946]: telegraf --input-filter cpu --output-filter influxdb config
Oct 18 15:41:21 ubuntu telegraf[6946]: # run a single telegraf collection, outputting metrics to stdout
Oct 18 15:41:21 ubuntu telegraf[6946]: telegraf --config telegraf.conf --test
Oct 18 15:41:21 ubuntu telegraf[6946]: # run telegraf with all plugins defined in config file
Oct 18 15:41:21 ubuntu telegraf[6946]: telegraf --config telegraf.conf
Oct 18 15:41:21 ubuntu telegraf[6946]: # run telegraf, enabling the cpu & memory input, and influxdb output plugins
Oct 18 15:41:21 ubuntu telegraf[6946]: telegraf --config telegraf.conf --input-filter cpu:mem --output-filter influxdb
Oct 18 15:41:21 ubuntu telegraf[6946]: # run telegraf with pprof
Oct 18 15:41:21 ubuntu telegraf[6946]: telegraf --config telegraf.conf --pprof-addr localhost:6060
Oct 18 15:41:21 ubuntu systemd[1]: telegraf.service: Succeeded.

Looks good now, but no data on the bucket

icvdok · October 19, 2021, 6:17am

Here some additional info, on my setup, sudo systemctl start telegraf

Add a telegraf.service located in

“/usr/lib/systemd/system/telegraf.service”

with a link to

“/etc/systemd/system/multi-user.target.wants/telegraf.service”

File Telegraf.service

[Unit]
Description=The plugin-driven server agent for reporting metrics into InfluxDB
Documentation=GitHub - influxdata/telegraf: The plugin-driven server agent for collecting & reporting metrics.
After=network.target

[Service]
EnvironmentFile=-/etc/default/telegraf
User=_telegraf
Group=_telegraf
ExecStart=/usr/bin/telegraf --config http://xxx.xxx.xxx.xxx:8086/api/v2/telegrafs/084bc78a1facc000 -$TELEGRAF_OPTS
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartForceExitStatus=SIGPIPE
KillMode=control-group

[Install]
WantedBy=multi-user.target

As in my previous post, it seams to work but no data and no PID found.
Here the journaling output:

Oct 19 07:34:48 HA-homeservices systemd[1]: Started The plugin-driven server agent for reporting metrics into InfluxDB.
Oct 19 07:34:49 HA-homeservices telegraf[2027]: flag provided but not defined: -$TELEGRAF_OPTS
Oct 19 07:34:49 HA-homeservices telegraf[2027]: Telegraf, The plugin-driven server agent for collecting and reporting metrics.
Oct 19 07:34:49 HA-homeservices telegraf[2027]: Usage:
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf [commands|flags]
Oct 19 07:34:49 HA-homeservices telegraf[2027]: The commands & flags are:
Oct 19 07:34:49 HA-homeservices telegraf[2027]: config print out full sample configuration to stdout
Oct 19 07:34:49 HA-homeservices telegraf[2027]: version print the version to stdout
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --aggregator-filter filter the aggregators to enable, separator is :
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --config configuration file to load
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --config-directory directory containing additional *.conf files
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --plugin-directory directory containing *.so files, this directory will be
Oct 19 07:34:49 HA-homeservices telegraf[2027]: searched recursively. Any Plugin found will be loaded
Oct 19 07:34:49 HA-homeservices telegraf[2027]: and namespaced.
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --debug turn on debug logging
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --input-filter filter the inputs to enable, separator is :
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --input-list print available input plugins.
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --output-filter filter the outputs to enable, separator is :
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --output-list print available output plugins.
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --pidfile file to write our pid to
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --pprof-addr pprof address to listen on, don’t activate pprof if empty
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --processor-filter filter the processors to enable, separator is :
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --quiet run in quiet mode
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --section-filter filter config sections to output, separator is :
Oct 19 07:34:49 HA-homeservices telegraf[2027]: Valid values are ‘agent’, ‘global_tags’, ‘outputs’,
Oct 19 07:34:49 HA-homeservices telegraf[2027]: ‘processors’, ‘aggregators’ and ‘inputs’
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --sample-config print out full sample configuration
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --once enable once mode: gather metrics once, write them, and exit
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --test enable test mode: gather metrics once and print them
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --test-wait wait up to this many seconds for service
Oct 19 07:34:49 HA-homeservices telegraf[2027]: inputs to complete in test or once mode
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --usage print usage for a plugin, ie, ‘telegraf --usage mysql’
Oct 19 07:34:49 HA-homeservices telegraf[2027]: --version display the version and exit
Oct 19 07:34:49 HA-homeservices telegraf[2027]: Examples:
Oct 19 07:34:49 HA-homeservices telegraf[2027]: # generate a telegraf config file:
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf config > telegraf.conf
Oct 19 07:34:49 HA-homeservices telegraf[2027]: # generate config with only cpu input & influxdb output plugins defined
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf --input-filter cpu --output-filter influxdb config
Oct 19 07:34:49 HA-homeservices telegraf[2027]: # run a single telegraf collection, outputting metrics to stdout
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf --config telegraf.conf --test
Oct 19 07:34:49 HA-homeservices telegraf[2027]: # run telegraf with all plugins defined in config file
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf --config telegraf.conf
Oct 19 07:34:49 HA-homeservices telegraf[2027]: # run telegraf, enabling the cpu & memory input, and influxdb output plugins
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf --config telegraf.conf --input-filter cpu:mem --output-filter influxdb
Oct 19 07:34:49 HA-homeservices telegraf[2027]: # run telegraf with pprof
Oct 19 07:34:49 HA-homeservices telegraf[2027]: telegraf --config telegraf.conf --pprof-addr localhost:6060
Oct 19 07:34:49 HA-homeservices systemd[1]: telegraf.service: Succeeded.

It seams that something is not working with the -$TELEGRAF_OPTS.

Any idea?
Another question, the PID created, should be listed by the ps command?

Thanks again for the support.

Jay_Clifford · October 22, 2021, 8:40am

Can you try and remove the environment variable at the end of the command? Looka like Telegraf is having trouble starting because this variable does not exist.

ExecStart=/usr/bin/telegraf --config http://xxx.xxx.xxx.xxx:8086/api/v2/telegrafs/084bc78a1facc000 -$TELEGRAF_OPTS
to
ExecStart=/usr/bin/telegraf --config http://xxx.xxx.xxx.xxx:8086/api/v2/telegrafs/084bc78a1facc000

icvdok · October 22, 2021, 10:04am

Hello Jay, the issue has been solved removing the -$TELEGRAF_OPTS as you suggested. In addition, I’ve to add the variable

INFLUX_TOKEN=<my token>

in the file

/etc/default/telegraf

Now the agent start automatically and it works.

I get these error now but i suppose it’s an agent issue.

Oct 20 10:45:10 ha-test telegraf[663]: time=“2021-10-20T10:45:10+02:00” level=error msg="failed to create cache directory. /etc/telegraf/.cache/snowflake, err: mkdir /etc/telegraf/.cache: permis
Oct 20 10:45:10 ha-test telegraf[663]: time=“2021-10-20T10:45:10+02:00” level=error msg="failed to open. Ignored. open /etc/telegraf/.cache/snowflake/ocsp_response_cache.json: no such file or di

Now I can start to do something
Thanks for the support

Topic		Replies	Views
Telegraf Data Collector Service Not Starting Windows Powershell Telegraf windows , cli	2	1556	November 4, 2021
Telegraf failed to start? Telegraf	3	10747	August 23, 2021
Telegraf failed to start after reboot Telegraf	9	8228	April 27, 2022
Problem configuring Telegraf Welcome & Getting Started telegraf	1	2695	April 29, 2020
Running Telegraf as a service Linux Telegraf	5	5939	December 6, 2021

Agent/Plugin startup clarification

Related Topics