How to apply grok to logs from syslog in telegraf?

1

Hi! This is my first post and I am a beginner with Telegraf.

In my current setup I am using Telegraf to ingest nginx logs from syslog and spit them out into influxdb. This works great, however the “message” field is a “blob” of text when it arrives to influx. I would like to parse this somehow.

As I understand telegraf has support for grok log parsing. As far as I can tell it can be invoked for [[inputs.file]] and for [[processors.parser]] . So my question is how can I configure telegraf to parse the message part of each incomming log statement from [[inputs.syslog]] with grok?

I found this post that gave me some hope, but there was no definite guide: Grok Parsing on Syslog Input

Left to my own devices I tried the following configuration, but it does not work. I get no errors or messages from telegraf at all and there is no change in the output, the message field still contains the same as before and there are no new fields:

# [...] Boilerplate omitted for brevity

# Accept input from syslog
[[inputs.syslog]]
   server = "tcp://:2010"

# Send metrics to InfluxDB
[[outputs.influxdb]]
  urls = ["http://our_influx.com"]
  username = "admin"
  password = "admin"

# Process logs with grok
[[processors.parser]]
  parse_fields = ["message"]
  merge = "override"
  data_format = "grok"
  grok_patterns = ["%{COMBINED_LOG_FORMAT}"]

Any help with this is welcome! And if you are into that sort of thing, I posted this as a question on stack overflow before I found this community: How to apply grok to logs from syslog in telegraf? - Server Fault

2

If you aren’t getting any errors, I would suspect your pattern isn’t matching your log lines. Run telegraf with debugging enabled and try a more generous pattern such as "%{DATA}"