Line parser with syslog RFC3164

Telegraf
1

I am trying to use the parse the syslog message field using Line Protocol. I am constraint to using RCF3164 and want to only parse the MSG part as explained in the documentation.

<34>Oct 11 22:14:15 mymachine su: 'su root' failed for lonvick on /dev/pts/8

The message section of this syslog line would be 'su root' failed for lonvick on /dev/pts/8.

When I disable the parser and only send a single word as the message, all default syslog fields are set as expected. (first row)
But after enabling [processors.parser] and sending the message in the Line Protocol format all the fields are messed up. (second row) And enabling parse_tags option results in errors printed in the stdout.

image
image2159×118 40.4 KB

The error message:
[processors.parser] could not parse tag host: metric parse error: expected tag at 1:13: "cc83be50ce3b"

Format in the message string

# measurement name
  'nginx-request,'
# tag: the request host
  'host=$http_host,'
# tag: usually http or https
  'scheme=$scheme,'
# tag: http method
  'method=$request_method,'
# tag: request path
#  ','
# tag: protocol, such as HTTP/1.1 or HTTP/2.0
  'protocol=$server_protocol,'
# tag: http response status
  'status=$status '

# field: number of bytes sent to the client
  'bytes_sent=${bytes_sent}u,'
# field: TLS cipher
  'ssl_cipher="$ssl_cipher",'
# field: TLS protocol
  'ssl_protocol="$ssl_protocol",'
# field: request length including headers and body
  'request_length=${request_length}u,'
# field: client IP
  'remote_addr="$remote_addr",'
# field: full path and arguments
  'request_uri="$request_uri",'
# field: http referer
  'http_referer="$http_referer",'
# field: x-forwarded-for header
  'http_x_forwarded_for="$http_x_forwarded_for",'
# field: request processing time in ms
  'request_time=${request_time} '

# end with unix timestamp
  '$epoch_ms';
2

Turned out, the parse_fields in the [[processors.parser]] must be set to “message”. See the appropriate configuration below.

[[processors.parser]]
  ## The name of the fields whose value will be parsed.
  parse_fields = ["message"]
  merge = "override"
  data_format = "influx"

[[inputs.syslog]]
  server = "udp://:6514"
  syslog_standard = "RFC3164"

And for completeness the raw syslog RFC3164 message

<190>Feb  5 08:15:58 8d26e3a0b345 nginxsrv: nginxrequest,hostname=<redacted>,scheme=http,method=GET,protocol=HTTP/1.1,status=200 bytes_sent=1167u,ssl_cipher="-",ssl_protocol="-",request_length=556u,remote_addr="<redacted>",request_uri="/static/media/image.svg",http_referer="<redacted>",http_x_forwarded_for="-",request_time=0.002 1675584958458