Multiline is not working for tail plugin with regex pattern

Telegraf
1

I’m using below telegraf input tail plugin for multi line log file monitoring however using regex pattern it gives me error:

> “Error running agent: Error parsing /etc/telegraf/telegraf.d/log.conf, line 16: field corresponding to `multiline’ is not defined in tail.Tail”

###############################################################################

SERVICE INPUT PLUGINS
###############################################################################

Parse the new lines appended to a file

[[inputs.tail]]
files = ["/home/use/api.txt"]
from_beginning = true
name_override = “GET_TOKEN_HELLO”
[inputs.tail.multiline]
pattern = “^%{GET_TOKEN_HELLO}”
match_which_line = “previous”
log file pattern is
15:30:54,612 [INFO ] [ABC:70] - [LOGS] JHMRSRP6GH|GenericToken|GET_TOKEN_HELLO|PAY|20210429153054516|20210429153054612|0.093s|SUCCESSFUL

OS is RHEL 7.x

Not sure of what I’m missing here… Please help me fix the issue

2

Hi @kavityp1,

What version of Telegraf are you using? The multiline functionality for the tail input plugin was only added from v1.16.0 onwards.

Let us know if you have any further questions.

3

Thank you, the upgrade worked!! :slight_smile:

Can you please also help to provide below details:

  1. Does input tail plugin only works with grok pattern?
  2. Can we use simple regex parser to capture the error pattern for ex. pattern = “/TEST/”

Basically I want to parse a simple log file using regex.
Is there any guide which can help for input tail with regex…

4

Hey @kavityp1,

a couple things I noticed here.

  1. your log example isn’t multi-line, it’s just one line. if all your log lines are like this you don’t need the multiline configuration
  2. if you do need multiline configuration, the pattern field is supposed to be a regex defining how to identify a line that should be appended or prepended to another line, and not grok format. "%{GET_TOKEN_HELLO}" is almost certainly wrong. A common pattern is "^\s" to identify lines that start with spaces or tabs as part of a multi-line log message.
  3. If you want to use regex to help parse the line, you definitely want to use the grok parser, which is build on regex, and supports regex selectors. Check out the grok documentation for details on that.

Hope that helps :slight_smile: