Telegraf - log file watcher


I have a requirement of watching /var/log/messages file and look for a OOM killer message. I want to setup a alert in Grafana based on matching pattern. is it possible with telegraf and yes can u provide an example?

file : /var/log/message
line am interested : Jun 1 12:52:36 kernel: nginx invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=999

Srinivas Kotaru

@Srinivas_Kotaru Have you checked out the logparser plugin for telegraf. There should be some documentation there.

@jackzampolin Yes i saw it. i also saw a tail input plugin.

In either case I don’t understand whether they work for my use case or not as they talking about parsing every line and storing. I don’t want to store every line in /var/log/messages fiel rather just store the matching pattern and use Grafana alert setup.

Would be nice if anyone really used this type of use case and help to write example. I really don’t have too much time to play around and test multiple configurations.

Srinivas Kotaru

@Srinivas_Kotaru Thats exactly what the logparser plugin does. It takes grok patterns and makes metrics out of lines that match. Doing alerting with Grafana once thats setup would be trivial.

@jackzampolin. Thanks. Let me take a look and configure logparser

If you already worked on this input plug-in, can u take a look at sample for my use case

files = ["/var/log/messages"]
from_beginning = false
patterns = [“invoked oom-killer”]

@jackzampolin I’m troubleshooting one more incident on similar pattern . is there anyway we can parse /var/log/messages or dmesg for oom killing pattern and store some data ?