It is possible to template certain configuration properties of an input plugin, where i might have multiple instances of this plugin configured to run?
For example i have 16+ of the following input.exec configs, which only really differ in the command and tag:
Telegraf doesn’t have a built in template system, but you can do this externally using whichever templating tools you prefer, I use ansible for this myself. Generally you would generate one file per plugin and place them into the /etc/telegraf/telegraf.d directory with the .conf extension.
Daniel - thanks, i’m currently making use of config files in telegraf.d. Even if telegraf doesn’t support templating, are you able to define / override the defaults for a plugin to achieve the same?
Sure - for my input plugin (exec), are you able to override or set some defaults that are inherited by each instance?
I guess to skin this a different way:
How do multiple commands work in input.exec - are they run in parallel or series, does one ‘failing’ in some way have cascading failure on the other commands defined, such that they won’t be run (compared to individual input.exec instances which are run in parallel and failures are isolated from each other)
Can you add dynamic tags, based upon content output of an exec command?
I’m thinking i could consolidate multiple conf files in telegraf.d to this:
[[inputs.exec]]
commands = [
"/etc/telegraf/telegraf.d/db-metrics.sh table1",
"/etc/telegraf/telegraf.d/db-metrics.sh table2",
"/etc/telegraf/telegraf.d/db-metrics.sh table3"
]
interval = "1h"
timeout = "2m"
name_override = "db-metrics"
data_format = "csv"
csv_header_row_count = 0
csv_column_names = [ "count", "tstamp", "last_updated_secs" ]
csv_timestamp_column = "tstamp"
csv_timestamp_format = "unix"
fielddrop = [ "tstamp" ]
[inputs.exec.tags]
# just need to work this out dynamically?
table = "table1"
The commands are run concurrently, however all commands must finish each interval to complete the plugins gather step. When a plugin doesn’t complete it’s gather step before the next interval then a warning message is logged and the plugin skips the gather step. Failures won’t interfere, you mostly need to worry about the command not completing. It is possible for a single command to block other commands, you will have to have them each in their own plugin definition if you want to avoid this and have full isolation. In general, it works this way across any plugin that allows multiple commands or addresses to be listed.
For dynamic tags it would be up to the parser aka data_format. In the case of csv you could use csv_tag_columns to set tags from the input document. Anything more advanced would be handled by a processor.
Daniel - great, so i can easily consolidate all the conf files I currently have into a single conf file.
The only issue i now have is that of i’d like to still use some jitter vs all the commands running concurrently, but i think i can simulate that with a wrapper script for each command with a random initial sleep period - unless you can configure jitter specifically in the exec plugin for all the commands listed?
