NGINX Access Log Parser

Telegraf
1

Hi everyone,

Here is what I’ve managed so far

/etc/nginx/nginx.conf

    log_format custom '$remote_addr - $remote_user [$time_local]'
                  '"$request" $status $body_bytes_sent'
                  '"$http_referer" "$http_user_agent"'
                  '$request_time $upstream_connect_time';

/etc/telegraf/telegraf.d/logparser.conf

  [inputs.logparser.grok]
    patterns = ["%{CUSTOM_LOG}"]
    custom_patterns = '''
        CUSTOM_LOG %{CLIENT:client_ip} %{NOTSPACE:ident} %{NOTSPACE:auth} \[%{HTTPDATE:ts:ts-httpd}\]"(?:%{WORD:verb:tag} %{NOTSPACE:request}(?: HTTP/%{NUMBER:http_version:float})?|%{DATA})" %{NUMBER:resp_code:tag} (?:%{NUMBER:resp_bytes:int}|-)%{QS:referrer} %{QS:agent}%{NUMBER:request_time:float} %{NUMBER:upstream_connect_time:float}
    '''

I got data in influxdb as I wanted but sometimes I got this result “Grok no match found for …”

tail -f /var/log/telegraf/telegraf.log

Grok no match found for: "<ip> - - [02/Mar/2020:18:35:55 +0300]\"GET / HTTP/1.1\" 301 169\"-\" \"Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko\"0.000 -"

Do you have any suggestion?