Grok pattern for telegraf

Telegraf
,
1

I have 2 log entries as below:
2022-02-02 14:09:37 109535 10.0.0.0 TCP_TUNNEL/200 2428 CONNECT abc.ca:443 - HIER_DIRECT/45.60.12.23 -
2022-02-02 14:09:57 4200 10.0.0.1 TCP_TUNNEL/200 1048 CONNECT abc1.com:443 - HIER_DIRECT/45.60.12.23 -

and my grok pattern is:
grok_patterns = [
‘’‘%{TIMESTAMP_ISO8601:time} %{SPACE} %{POSINT:elapsedTime:int} %{IP:clientIp:tag} %{DATA}/%{POSINT:statusCode:tag} %{DATA} %{DATA} %{DATA:address:tag} - %{DATA}/%{IP:ip:tag}’‘’]

The pattern is able to parse the second entry from logs as it has elapsedTime value less than 100001(third coloumn) but for the first entry it is giving me this error

2022-02-02 14:09:37 D! Grok no match found for: “2022-02-02 14:09:37 109535 10.0.0.0 TCP_TUNNEL/200 2428 CONNECT abc.com:443 - HIER_DIRECT/45.60.12.23 -”

2

I really like to use https://grokdebug.herokuapp.com/ to debug grok issues. Start one field at a time and keep adding another field as you go to ensure it keeps parsing.

With your example, I found the %{SPACE} really wasn’t necessary.

Here is what I used:

[[inputs.file]]
  files = ["data"]
  data_format = "grok"
  grok_patterns = [
    '%{TIMESTAMP_ISO8601:time} %{POSINT:elapsedTime:int} %{IP:clientIp:tag} %{DATA}/%{POSINT:statusCode:tag} %{DATA} %{DATA} %{DATA:address:tag} - %{DATA}/%{IP:ip:tag}'
  ]

[[outputs.file]]

Which produced:

file,address=abc.ca:443,clientIp=10.0.0.0,host=ryzen,ip=45.60.12.23,statusCode=200 time="2022-02-02 14:09:37",elapsedTime=109535i 1643910174000000000
file,address=abc1.com:443,clientIp=10.0.0.1,host=ryzen,ip=45.60.12.23,statusCode=200 elapsedTime=4200i,time="2022-02-02 14:09:57" 1643910174000000000
3

Thank you for the suggestion. If I remove the %{SPACE} the other one works fine.

The root cause is, for first log entry there is only one space between second and third coloumn values and for the second log entry there are more than one space

2022-02-02 14:09:37 109535 10.0.0.0 TCP_TUNNEL/200 2428 CONNECT abc.ca:443 - HIER_DIRECT/45.60.12.23 -
2022-02-02 14:09:57   4200 10.0.0.1 TCP_TUNNEL/200 1048 CONNECT abc1.com:443 - HIER_DIRECT/45.60.12.23 -

Any suggestions on how to ignore or eliminate the empty spaces between the coloumns

4

Ah the extra spaces did not come through in your first post!

In that case I would do something like this:

%{TIMESTAMP_ISO8601:time}%{SPACE}%{POSINT:elapsedTime:int}

As the %{SPACE} means 1 or more spaces, if you put spaces around that, you are really expecting 3 spaces.

5

yes it was my bad I just pasted as text in my first post.
But thank you it works now :slight_smile: