Telegraf not parsing even with example grok

Telegraf
1

I verified both my original log file entry and grok filter via the heroku grokdebug and it parsed correctly. So then I tried using an example from https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html:

I have GROK filter as %{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}

When I start the service via systemctl stop/start telegraf, I see nothing to stdout nor the output file I specified. I’ve enabled debug logging in [agent] section, and I don’t see that log file generated either. I vierifed telegraf service is running via ps | ax.

Then I tried manually running service via “telegraf -config /etc/telegraf/telegraf.conf”.

This time I see a metrics.out file, but it’s empty. I also see the telegraf.log file generated, but all It spits out every 10 seconds is: 2017-10-06T06:14:00Z D! Output [file] buffer fullness: 0 / 10000 metrics.

Here is my test.log file:
root@hpe-host-ubuntu16-02 [~]# cat test.log
55.3.244.1 GET /index.html 15824 0.043
55.3.244.2 GET /index.html 15824 0.043
55.3.244.1 PUT /index.html 15824 0.043
55.3.244.1 GET /index.html 15824 0.043
55.3.244.2 GET /index.html 15824 0.043
55.3.244.1 PUT /index.html 15824 0.043
55.3.244.1 GET /index.html 15824 0.043
55.3.244.2 GET /index.html 15824 0.043
55.3.244.1 PUT /index.html 15824 0.043

This is what my inputs and outputs section in /etc/telegraf/telegraf.conf looks like:

Configuration for telegraf agent

[agent]
interval = “10s”
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = “0s”
flush_interval = “10s”
flush_jitter = “0s”
precision = “”
debug = true
quiet = false
logfile = “/root/telegraf.log”

hostname = “”
omit_hostname = false

[[inputs.logparser]]
files = [“/root/test.log”]
from_beginning = false

[inputs.logparser.grok]
patterns = [“%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}”]
measurement = “test_log”

# Send telegraf metrics to file(s)

[[outputs.file]]

## Files to write to, “stdout” is a specially handled file.

files = [“stdout”, “/root/metrics.out”]
data_format = “influx”

Am I missing any configuration? Is my grok pattern incorrect for telegraf? (it does say, “The Telegraf grok parser uses a slightly modified version of logstash “grok” patterns”)

2

Your pattern is works for me with your test file. While testing you may want to set from_beginning = true so that the entire file is parsed, otherwise only new lines will be reported. Also, technically optional, but once you have it working you will probably want to set type conversions on the numeric types:

[[inputs.logparser]]
  files = ["test.log"]
  from_beginning = true

  [inputs.logparser.grok]
    patterns = ["%{IP:client} %{WORD:method:tag} %{URIPATHPARAM:request:tag} %{NUMBER:bytes:int} %{NUMBER:duration:float}"]
    measurement = "test_log"

test_log,method=GET,path=test.log,request=/index.html bytes=15824i,client="55.3.244.1",duration=0.043 1507327764467152650
3

It still does not work, I changed from_beginning = true. Originally I was testing with a log file that I knew was outputting data, and I know my grok pattern was valid as per the heroku grok calculator.

Either way I don’t see it working after making it back to true. I removed the metrics.out and telegraf.log file, WHen I stop and start the service, these files aren’t even recreated. I think this is issue #1. If it’s running as a service, why aren’t even the empty output file or log file created?

root@hpe-host-ubuntu16-02 [~]# cat test.log
55.3.244.1 GET /index.html 15824 0.043
55.3.244.2 GET /index.html 15824 0.043
55.3.244.1 PUT /index.html 15824 0.043
55.3.244.1 GET /index.html 15824 0.043
55.3.244.2 GET /index.html 15824 0.043
55.3.244.1 PUT /index.html 15824 0.043
55.3.244.1 GET /index.html 15824 0.043
55.3.244.2 GET /index.html 15824 0.043
55.3.244.1 PUT /index.html 15824 0.043

root@hpe-host-ubuntu16-02 [~]# cat metrics.out
root@hpe-host-ubuntu16-02 [~]# cat metrics.out
root@hpe-host-ubuntu16-02 [~]#
root@hpe-host-ubuntu16-02 [~]# cat metrics.out
root@hpe-host-ubuntu16-02 [~]# service telegraf stop
root@hpe-host-ubuntu16-02 [~]# rm metrics.out
root@hpe-host-ubuntu16-02 [~]# rm telegraf.log
root@hpe-host-ubuntu16-02 [~]#
root@hpe-host-ubuntu16-02 [~]# vim /etc/telegraf/telegraf.conf
root@hpe-host-ubuntu16-02 [~]# service telegraf start
root@hpe-host-ubuntu16-02 [~]# date
Fri Oct 6 18:47:57 CDT 2017
root@hpe-host-ubuntu16-02 [~]# ls
cloud_pricing index.html index.html.1 telegraf2.log telegraf.conf telegrafdefault.conf test.log
root@hpe-host-ubuntu16-02 [~]# date
Fri Oct 6 18:48:10 CDT 2017
root@hpe-host-ubuntu16-02 [~]# pwd
/root
root@hpe-host-ubuntu16-02 [~]# date
Fri Oct 6 18:48:46 CDT 2017
root@hpe-host-ubuntu16-02 [~]# ls
cloud_pricing index.html index.html.1 telegraf2.log telegraf.conf telegrafdefault.conf test.log
root@hpe-host-ubuntu16-02 [~]#

As you can see, nothing gets generated.

4

My interval and flush interval are both at 10 as you can see

5

Perhaps it is a permission problem? Normally the service runs as the telegraf user and wouldn’t be able to read/write from /root.