# Parse a custom log using telegraf logparser input

**URL:** <https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650>\
**Category:** Telegraf\
**Tags:** telegraf\
**Created:** [July 18, 2017, 3:17am UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650 "2017-07-18T03:17:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Revathy\_K\_T](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@Revathy\_K\_T](https://community.influxdata.com/u/Revathy_K_T)\
**Post date:** [July 18, 2017, 3:17am UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/1 "2017-07-18T03:17:05Z")

</div>

Hi,

I have a custom log file with entries as given below  
How to write a grok pattern to match this and output it to influxdb. I have been trying for a long time with no success.

```
[Tue, 18 Jul 2017 02:39:45 GMT] GET 200 /lectures/questions/150 2032 47.8.10.6 [17.427 ms] phone
[Tue, 18 Jul 2017 02:39:45 GMT] GET 200 /forums/fetch/lectures/31 2032 47.8.10.6 [8.398 ms] phone
[Tue, 18 Jul 2017 02:39:45 GMT] GET 304 /users/phone/2032 2032 47.8.10.6 [6.613 ms] phone
[Tue, 18 Jul 2017 02:40:06 GMT] GET 200 /lectures/summary 2032 47.8.10.6 [12.817 ms] phone
```

---

<div class="post-metadata">

**Author:** ![daniel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/daniel/32/142_2.png) [@daniel](https://community.influxdata.com/u/daniel)\
**Post date:** [July 18, 2017, 5:54pm UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/2 "2017-07-18T17:54:04Z")

</div>

I wrote a few tips in the [logparser readme](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/logparser#tips-for-creating-patterns), have you seen these? I highly suggest working on your pattern one token at a time.

---

<div class="post-metadata">

**Author:** ![Revathy\_K\_T](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@Revathy\_K\_T](https://community.influxdata.com/u/Revathy_K_T)\
**Post date:** [July 20, 2017, 2:40am UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/3 "2017-07-20T02:40:40Z")

</div>

Thanks for the reply. I followed it and was able to find the pattern of all except timestamp.

%{WORD:method} %{NUMBER:resp} %{URIPATHPARAM:request} %{NUMBER:userid} %{IPORHOST:clientip} [%{NUMBER:resptime}%{SPACE}ms] %{WORD:device}

For the time I tried %{DATESTAMP\_RFC2822: timestamp} but it says not matched.  
Can I get some help here

---

<div class="post-metadata">

**Author:** ![Revathy\_K\_T](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@Revathy\_K\_T](https://community.influxdata.com/u/Revathy_K_T)\
**Post date:** [July 20, 2017, 2:58am UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/4 "2017-07-20T02:58:01Z")

</div>

Ok, I got the pattern. Thanks for the help.  
If any body needs it the pattern to match time stamp is

^[(?%{DAY}, %{MONTHDAY} %{MONTH} %{YEAR} %{TIME} GMT)]

---

<div class="post-metadata">

**Author:** ![roey\_collignon](https://avatars.discourse-cdn.com/v4/letter/r/bcef8e/32.png) [@roey\_collignon](https://community.influxdata.com/u/roey_collignon)\
**Post date:** [November 13, 2017, 8:14pm UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/5 "2017-11-13T20:14:26Z")

</div>

Hi,

Here is an example of my log:

[test.co:443](http://test.co:443) 80.81.174.142 [www.test.co](http://www.test.co) [07/Nov/2017:09:53:34 +0200] 10524941 “GET /api/v1/company/jobs HTTP/1.1” 200 34653 “[https://www.test.co/v/hunt](https://www.test.co/v/hunt)” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10\_12\_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36” WgFmfn8AAAEAAF7ZVwUAAAAB ad3doh5o6zpyk5zubtrck8ta2p1tz187

I added this to my telegraf.conf:

[[inputs.logparser]]

## files to tail.

files = [“/var/log/apache2/other\_vhosts\_access.log”]

## Read file from beginning.

from\_beginning = false

## Override the default measurement name, which would be “logparser\_grok”

name\_override = “apache\_access\_log”

## For parsing logstash-style “grok” patterns:

[inputs.logparser.grok]  
patterns = [“%{CUSTOM\_LOG}”]  
custom\_patterns = ‘’’  
CUSTOM\_LOG %{DATA:host} %{IP:client} %{DATA} [%{HTTPDATE:ts:ts-httpd}] %{NUMBER:resp\_time:tag} "(?:%{WORD:verb:tag} %{NOTSPACE:r$  
‘’’

I tried using [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/) to make sure my custom pattern works, and it looks like it does, but no entries were created in influx DB ☹

Tested with other patterns and log examples and it worked, so I guess I’m missing something in the [[inputs.logparser]].

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![eclements](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/eclements/32/5222_2.png) [@eclements](https://community.influxdata.com/u/eclements)\
**Post date:** [June 25, 2020, 8:13pm UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/6 "2020-06-25T20:13:13Z")

</div>

The examples in this post is now slightly outdated. For the new " **grok\_custom\_patterns**" format, see this update I wrote after not getting above 2017 examples to work in 2020 Telegraf: [Custom log parsing with latest Tail Plugin, GROK and InfluxDB configuration](https://community.influxdata.com/t/custom-log-parsing-with-latest-tail-plugin-grok-and-influxdb-configuration-for-grafana/14986)

---

<div class="post-metadata">

**Author:** ![ramyam07](https://avatars.discourse-cdn.com/v4/letter/r/9dc877/32.png) [@ramyam07](https://community.influxdata.com/u/ramyam07)\
**Post date:** [February 18, 2022, 6:04pm UTC](https://community.influxdata.com/t/parse-a-custom-log-using-telegraf-logparser-input/1650/7 "2022-02-18T18:04:37Z")

</div>

Hi I want to filter all the logs where the status code is above 500

Please suggest me either pattern or regex to achieve this
