Telegraf tail Input plugin does not read newly created log file after retention

Telegraf
, ,
1

Hello!
I have the Java Spring application with Logback logging framework which places logs in application.log file. And I’m using telegraf tail input plugin to collect specific events from this file. Logback has SizeAndTimeBasedRollingPolicy, so when file is reached certain size or when time is 00:00 the old application.log is archived (as an app-%d{yyyy-MM-dd}.%i.log.gz) and new one creates instead. The problem is, the plugin doesn’t switch to the newly created file and the data stops coming after 00:00 at all. Please help me find the right approach, there is possibly some configure option which I miss? How to make the plugin switch to parsing a newly created file (without telegraph restart)?
I’m using telegraf v1.19.0 and Influxdb v1.8.7
My telegraf config:

[[inputs.tail]]
  files = ["/opt/app1/logs/application.log"]
  from_beginning = true
  name_override = "app1_events"
  grok_patterns = ['%{TIMESTAMP_ISO8601:timestamp}.*x-artefacts-%{GREEDYDATA:vid_sv}/data.*<ns1:status>%{WORD:status}</ns1:status>']
  data_format = "grok"
  tagexclude = ["path"]
2

From the docs, I’m guessing that the watch_method = "inotify" default is what’s tripping you up – the tail plugin launches a service to watch the file but when the file is renamed (but still exists), the service doesn’t see any new data.

Maybe try watch_method = "poll" ? (docs link)

Not sure what that’ll do to your performance, so keep an eye on things if you do try that.

1 Like
3

Big thanks John! It works for me perfectly. But I don’t clearly understand what’s the difference between “poll” and “inotify” watch_method? I see only this in example config
## Method used to watch for file updates. Can be either "inotify" or "poll". watch_method = "inotify"

4

inotify is a very efficient method of keeping a watch on files or directories
(and sub-directory trees) based on messages from the kernel.

Polling is a much less efficient method of “looking every now and again to see
whether something has happened” to those files or directories.

Polling takes up far more resources, is less timely, and doesn’t provide so
many facilities for what you want to know about the files as inotify does.

If you can use inotify (and any current Linux kernel supports it) you should
do so. There’s a nice command-line utility inotifywatch which enables you to
make use of it in bash scripts as well.

Antony.

2 Likes