You will want to use the tail plugin, the other two plugins don’t work well for log data because they parse the full file each interval.
When restarting Telegraf you can set the from_beginning
option to true and the file will be reparsed from the start of the file, or if this is false the parsing will start at the end of the file and would potentially miss new lines added during the restart. If you do use from_beginning
, it will rely on the property of InfluxDB to only record a single value per measurement+tags+field+timestamp to avoid duplicates, but it still can be quite expensive if you have a large log file. Instead of restarting, you can reload Telegraf by sending a SIGHUP and it will pick up at the position it left off when from_beginning = false
.
The plugin only follows files that matches the files pattern, if the file is renamed so that it no longer matches the pattern then it will stop reading the file as soon as it notices the rename. This means it is possible for a rotated file to be removed before it is fully read, we have an open issue for this. You might be able to list both the current and last file to avoid this, for example:
files = ["/var/log/syslog", "/var/log/syslog.0"]