Telegraf is not reading Nginx access log file data

Telegraf
,
1

Telegraf is not reading Nginx access log file data.

Below is the Telegraf plugin configuration.

[[inputs.tail]]
files = ["/var/log/nginx/access.log"]
from_beginning = false
grok_patterns = ["%{IP:client_ip} .* %{CUSTOM_TIMESTAMP:timestamp:ts-httpd} %{NOTSPACE:method:tag} %{NOTSPACE:request_path:tag} .* %{NOTSPACE:country:tag} %{NUMBER:response_code:tag} %{NUMBER:resp_bytes:int} .* %{IP:http_x_forwarded_for} .* rt=%{NUMBER:request_time} .*"]

grok_custom_patterns = ‘’’
CUSTOM_TIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{CUSTOM_TZ}
CUSTOM_TZ [±][0-9]{4}
‘’’
data_format = “grok”

Below is the Nginx access log format.

202.164.53.178 - - [22/Dec/2021:11:34:08 +0530] “GET / HTTP/1.1” 200 15 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36” “-” “3.7.231.59” sn=“10.30.20.131” rt=0.002 ua=“10.30.10.109:8082” us=“200” ut=“0.004” ul=“15” cs=-

Below is the Telegraf debug log

2021-12-22T08:59:48Z D! [agent] Attempting connection to [outputs.influxdb]
2021-12-22T08:59:48Z D! [agent] Successfully connected to outputs.influxdb
2021-12-22T08:59:48Z D! [agent] Starting service inputs
2021-12-22T08:59:48Z D! [inputs.tail] Tail added for “/var/log/nginx/access.log”
2021-12-22T08:59:58Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2021-12-22T09:00:08Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2021-12-22T09:00:18Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

2

Welcome to the community.

Your grok pattern will probably not work.
It is a pain to write and test such complex patterns.:grimacing:

I am not familiar with the Nginx log files.
Is this the default format of Nginx or has it been customized?
If it is the default format of Nginx, you could try the following predefined formats:

grok_patterns = ["%{COMMON_LOG_FORMAT}"]   # (plain apache & nginx access logs)

or

grok_patterns = ["%{COMBINED_LOG_FORMAT}"]   # (access logs + referrer & agent)
3

We have updated the grok pattern and getting output in the Grok debugger but still no measurement found in influxdb

Below is the updated Telegraf plugin configuration

[[inputs.tail]]
files = ["/var/log/nginx/access.log"]
from_beginning = true
grok_patterns = ["%{CUSTOM_LOG]"]
grok_custom_patterns = ‘’’
CUSTOM_TIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR}:%{HOUR}:%{MINUTE}:%{SECOND} %{CUSTOM_TZ}
CUSTOM_TZ [±][0-9]{4}
CUSTOM_LOG %{IP:client_ip} - - [%{CUSTOM_TIMESTAMP:timestamp:ts-httpd}] “%{NOTSPACE:method:tag} %{NOTSPACE:request_path:tag} %{NOTSPACE:html_version:tag}” %{NUMBER:response_code:tag} %{NUMBER:resp_bytes:int} “-” “%{NOTSPACE:client:tag}%{GREEDYDATA:sn=}”%{IP:http_x_forwarded_for}" rt=%{NUMBER:request_time}
‘’’
data_format = “grok”
name_override = “access.log”

[[outputs.file]]
files = [“stdout”]
data_format = “influx”

Below is the Nginx access log

202.164.53.178 - - [23/Dec/2021:14:28:48 +0530] “GET / HTTP/1.1” 304 0 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36” “-” “3.7.237.59” sn=“10.30.20.131” rt=0.002 ua=“10.30.10.110:8082” us=“304” ut=“0.000” ul=“0” cs=-

Below is the telegraf debug log

2021-12-23T09:50:30Z D! Grok no match found for: “202.164.53.178 - - [23/Dec/2021:14:29:02 +0530] “GET / HTTP/1.1” 304 0 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36” “-” “3.7.237.59” sn=“10.30.20.131” rt=0.002 ua=“10.30.10.110:8082” us=“304” ut=“0.000” ul=“0” cs=-”
2021-12-23T09:50:40Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2021-12-23T09:50:40Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2021-12-23T09:50:50Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2021-12-23T09:50:50Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

4

This is a very complicated grok pattern. :grimacing:
As the log says, it cannot find a match.
Have you tried the predefined formats as I suggested?

I would build the pattern step by step starting from the left and keep testing between each step. So first the IP, then the timestamp, then the method, then the agent and so on.

5

Hi,
The standard format is not working as desire.
As I have already mentioned, the pattern is tested successfully with https://grokdebug.herokuapp.com/. You can try from your end, it is producing the output with same pattern.
I have tested step by step approach, but it was throwing error
2021-12-23T09:50:40Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2021-12-23T09:50:40Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

Can you please advise.

6

Have you even tried?
I took your data and it worked, see below.

nginx.input file:

202.164.53.178 - - [23/Dec/2021:14:28:48 +0530] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36" "-" "3.7.237.59" sn="10.30.20.131" rt=0.002 ua="10.30.10.110:8082" us="304" ut="0.000" ul="0" cs=-

Telegraf config nginx.conf file:

[agent]  # Configuration for telegraf agent
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "ns"
  debug = true
  logtarget = "file"
  logfile = "nginx.log"
  hostname = "localhost"
  omit_hostname = true

[[inputs.tail]]  # only for testing
  files = ["nginx.input"]
  from_beginning = true
  data_format = "grok"
  # grok_patterns = ["%{COMMON_LOG_FORMAT}"]   # (plain apache & nginx access logs)
  grok_patterns = ["%{COMBINED_LOG_FORMAT}"]   # (access logs + referrer & agent)
  name_override = "nginx"

[[outputs.file]]  # only for debugging
  namepass = ["nginx"]
  files = ["nginx.out"]
  influx_sort_fields = true

I ran both standard grok_patterns from above with:

telegraf.exe --once --config nginx.conf

Both standard grok_patterns worked, here are the outputs of both in the nginx.out file:

nginx,path=nginx.input,resp_code=304,verb=GET auth="-",client_ip="202.164.53.178",http_version=1.1,ident="-",request="/",resp_bytes=0i 1640249928000000000
nginx,path=nginx.input,resp_code=304,verb=GET agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36",auth="-",client_ip="202.164.53.178",http_version=1.1,ident="-",referrer="-",request="/",resp_bytes=0i 1640249928000000000