Hi,
We are currently trying to use Telegraf tail input plugin to collect Nginx access logs in Linux VM and convert to metrics to be consumed by downstream components based on InfluxDb line protocol. Nginx will put access logs in a specific location, where tail plugin can read logs line by line. However, when service needs to restart maybe due to a deployment, tail plugin will read all logs again which causes huge duplicates. Is there a way that it can keep a persistent record of last read position? Or what is the suggested way to tail logs in this scenario?
Hello. @shenmuxiaosen,
Interesting question, I’m looking into how to make the tail plugin persistent.
Have you taken a look at the logparser plugin?
I was thinking logparser plugin has been deprecated and moved to tail plugin. Basically we are trying to figure out if there is a way to read logs line by line and keep going forward regardless of service restart.
If you are using from_beginning = true you will probably want to disable it, this way Telegraf will pick up from the end of the logfile when restarted. It isn’t currently possible to store the offsets when restarting, though it is something we would like to add, but you can often avoid restarting Telegraf by using SIGHUP to reload which does keep track of the offsets.
SIGHUP will reload config if Telegraf service already running, or start Telegraf with config if not running, correct?
What is the syntax for SIGHUP from a Windows 10 command line? I tried this…
telegraf.exe -conf “config-file-name” sighup
The response looks good, says “Starting Telegraf” but then it does not complete/return to the command line. I have to hit Ctrl-C to get it back. This happens when Telegraf service is already running or stopped, but Telegraf does get started (although the Windows Services list does not refresh its status correctly).
