Telegraf logparser output prometheus_client help needed

1

Hi, i am trying to count number of lines based on text string in them and send that info to prometheus_client.
My log file looks like this:

2018-05-25T11:09:59.084943+00:00 somehost-a0-2ut appname:  Use of uninitialized value in join or string at /usr/local/lib/...
2018-05-25T11:10:00.345101+00:00 somehost-c4-yac othername:  splice() offset past end of array at /usr/local/lib/...
2018-05-25T11:10:01.148878+00:00 somehost-a3-2ut somthing:  Retrying (403) https://s3-eu-west-1.amazonaws.com/something/...

Could you help me writing correct pattern?

[[inputs.logparser]]
  files = ["/var/log/some-file-*.log"]
  from_beginning = false
  [inputs.logparser.grok]
    pattern: "[.* ]%{C_TEXT:error_count}[ .*]"
    custom_patterns: [ "C_TEXT somehost-.*" ]
    measurement = "errors_count"
    timezone = "UTC"

I can see these lines in outputs.file, but not in prometheus_client…

As suggested i added “modifier”:

pattern: "[.* ]%{C_TEXT:error_count:int}[ .*]"

But that seem to generate E! Error parsing log line: logparser_grok: must have one or more fields

I am trying to grep number or error lines per time period and push that data into prometheus_client - Is that even possible?

2

This sounds like the create static fields parsing log files issue. There is a workaround in the upcoming 1.7 release though by creating a tag and then converting it to a field. If you try this you will need to use the nightly build.

[[inputs.logparser]]
  files = ["/var/log/foo"]
  from_beginning = true
  [inputs.logparser.grok]
    # add hostname as a string field for now to satisfy "at least one field" requirement.
    patterns = ["%{TIMESTAMP_ISO8601} %{DATA:hostname} %{GREEDYDATA}"]
    measurement = "errors_count"
  [inputs.logparser.tags]
    value = "1"

[[processors.converter]]
  namepass = ["errors_count"]
  [processors.converter.tags]
    integer = ["value"]
  [processors.converter.fields]
    tag = ["hostname"]

[[aggregators.basicstats]]
  namepass = ["errors_count"]
  drop_original = true
  period = "10s"
  stats = ["sum"]
3

Works perfectly - thanks :slight_smile: