Tail plugin Interval parameter not working and High CPU usage over 100% for 10min

Telegraf
1

Dear All,

I got 2 issues with Tail plugin.

  1. Time Interval set at plugin level to override Agent, this does not work.

  2. High CPU usage for 10min while we got a lot of activity in our log due to some applications activities. CPU over 100% over a continuous 10 min period.

  3. I set a parameter to try to tail my logs every 5 minutes to try to avoid High CPU usage which is most likely related to reading our logs.

I input some data at 10:50 for example, 30 sec later I can see those in influxDB, expectation was to get those data 5 min later.

[agent]
interval = “30s”
debug = false
hostname = “ABC”
round_interval = false
flush_interval = “30s”
flush_jitter = “5s”
collection_jitter = “0s”
metric_batch_size = 1000
metric_buffer_limit = 30000
quiet = false

[[inputs.tail]]
interval = “5m”
files = ["/apps/path/log/202*.txt"]
from_beginning = false
pipe = false
data_format = “grok”

  1. Using grok pattern to filter those applications logs and get the ERR and WAR message only. But it seems Telegraf still impacting with High CPU and log data got multiple entries related to other matter which we cannot remove.
    Any idea on how to improve Tail input efficiency, else I will have to remove this input plugin part as too risky for Production rollout.

Telegraf version: 1.17.3