Telegraf grok and timestamps

Telegraf
1

I am trying to parse a file that contains lines formatted like this

2020-06-02 03:33:27 UTC Local0.Debug 10.10.0.1 Jun 02 2020 03:33:27: messagemessage

With input tail this is the grok pattern that works by parsing the important information I want from the line

grok_patterns = ["%{TIMESTAMP_ISO8601:timestamp} UTC\\t%{WORD:facility:tag}\\.%{WORD:severity:tag}\\t%{NOTSPACE:host:tag}\\t"]

But what I need to do is parse the timestamp so it is the correct timestamp when telegraf writes to the outputs. According to the documentation the timestamp grok should look like this

grok_patterns = ['%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} UTC\\t%{WORD:facility:tag}\\.%{WORD:severity:tag}\\t%{NOTSPACE:host:tag}\\t']

This results in grok not matching anything

2020-06-02T13:03:52Z D! Grok no match found for: "2020-06-02 02:48:18 UTC\tLocal0.Debug\t10.10.0.1\tJun 02 2020 02:48:18: messagemessage"

The whole process is running on windows if that matters.

2

In TOML, when you single quote a string you won’t need to double escape the regex syntax. This is the big reason why the second pattern isn’t working, it is escaped for a double quoted string. One other issue is that you must collect at least one field, this is a requirement of the Telegraf/InfluxDB data model.

Try this pattern out, it just collects everything at the end into a string field:

  grok_patterns = ['%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} UTC\s+%{WORD:facility:tag}\.%{WORD:severity:tag}\s+%{NOTSPACE:host:tag}\s+%{GREEDYDATA:message}']