Telegraf is not reading Nginx access log file data on nginx 1.20

Telegraf
, ,
1

Hello guys. I’m studying about telegraf passing the data to prometheus and displaying the data in grafana. The nginx log is not passing through telegraf.
My telegraf.conf:

[[inputs.nginx]]
   urls = ["http://localhost/nginx_status"]
   response_timeout = "5s"
[[inputs.tail]]
  name_override = "nginxlog"
  files = ["/var/log/nginx/access.log"]
  from_beginning = true
  pipe = false
  data_format = "grok"
  grok_patterns = ["%{COMBINED_LOG_FORMAT}"]
[[inputs.cpu]]
  percpu = true
[[inputs.disk]]
[[inputs.diskio]]
[[inputs.io]]
[[inputs.net]]
[[inputs.mem]]
[[inputs.system]]

This code is working on nginx(1.18.0 - ubuntu), but on nginx(1.20.1 - centos) it doesn’t. I need you to pass this log to perform the metrics in grafana.

nginxlog_http_version{agent="Go-http-client/1.1",auth="-",client_ip="127.0.0.1",host="ubuntu",ident="-",path="/var/log/nginx/access.log",referrer="-",request="/nginx_status",resp_code="200",verb="GET"} 1.1

nginxlog_resp_bytes{agent="Go-http-client/1.1",auth="-",client_ip="127.0.0.1",host="ubuntu",ident="-",path="/var/log/nginx/access.log",referrer="-",request="/nginx_status",resp_code="200",verb="GET"} 101

Can you help me?

2

This code is working on nginx(1.18.0 - ubuntu), but on nginx(1.20.1 - centos)

What is not working? Are you getting an error message or are metrics just missing?

3

Centos is not working. No error shows in the telegraf log.
Telegraf is not reading Nginx access log file data.

4

Telegraf is not reading Nginx access log file data.

How do you know this? Are there any logs or data you can share with us to help you? Are you not seeing data in your output?

Have you tried using the [[outputs.file]] and only the tail input and see what metrics are coming in?

Can you run Telegraf with --debug to see what metrics it is capturing?

5

If the two log lines are the ones you are trying to parse, I don’t see how they can match with the grok COMBINED_LOG_FORMAT (which is an extension of COMMON_LOG_FORMAT)

for reference, this is what COMMON_LOG_FORMAT looks like

127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326

The two lines you shared look like JSON, if that’s the case use the JSON format… otherwise you will have to write your own grok pattern (which is not easy, but the docs have or link almost all you need)

6

I’m pretty sure the data you want to parse is from some kind of Prometheus exporter tool, which then can be scrapped in intervals using Prometheus. Or it even looks like a Prometheus DB query.

When using Telegraf, you can use the inputs.tail plugin to directly read the Nginx logs files. Nginx (or Apache) log files are for sure either COMMON_LOG_FORMAT or COMBINED_LOG_FORMAT. Both can be parsed using the Grok parser.

In that case your Telegraf config will look like this:

  files = ["/var/log/nginx/access.log"]
  from_beginning = false
  grok_patterns = ["%{COMBINED_LOG_FORMAT}"]
  name_override = "nginx_access_log"
  grok_custom_pattern_files = []
  grok_custom_patterns = '''
  '''
  grok_timezone = "Europe/Amsterdam"
  data_format = "grok"

And make sure you use the correct database queries to retrieve the data from the database (eg. InfluxDB), for example retrieving 200 OKs:

SELECT count("request") FROM "nginx_access_log" WHERE ("host" =~ /^$host$/ AND "resp_code" = '200') AND $timeFilter GROUP BY time($__interval) fill(null)
7

I have the same problem or a similar problem nginx error and performance metric not getting scrapeds from prometheus while others are. Although my grok pattern seems to be good. All fields are recognized well (tested with outputs.file section)