How is my log file malfomed?

Telegraf
1

Hello all, unable to figure out why I can’t get telegraf to tail my nginx logs. I’m sure this likely some type of user error but I’d like some assistance.

This error I receive after telegraf --test:

2020-10-11T18:06:31Z I! Starting Telegraf 1.15.3
2020-10-11T18:06:31Z I! Using config file: /etc/telegraf/telegraf.conf
2020-10-11T18:06:31Z E! [inputs.tail] Malformed log line in “/var/log/nginx/access.log”: [“182.113.54.202 - - [11/Oct/2020:09:54:02 -0400]“POST /GponForm/diag_Form?images/ HTTP/1.1” 401 179”-" “Hello, World”“0.000” “-”“Xuchang” “CN”"]: metric parse error: expected field at 1:17: “182.113.54.202 - - [11/Oct/2020:09:54:02 -0400]“POST /GponForm/diag_Form?images/ HTTP/1.1” 401 179”-" “Hello, World”“0.000” “-”“Xuchang” “CN”"
2020-10-11T18:06:31Z E! [telegraf] Error running agent: input plugins recorded 1 errors

Sample log:
182.113.54.202 - - [11/Oct/2020:09:54:02 -0400]“POST /GponForm/diag_Form?images/ HTTP/1.1” 401 179"-" “Hello, World”“0.000” “-”“Xuchang” “CN”

nginx log config:
log_format custom ‘$remote_addr - $remote_user [$time_local]’
‘"$request" $status $body_bytes_sent’
‘"$http_referer" “$http_user_agent”’
‘"$request_time" “$upstream_connect_time”’
‘"$geoip_city" “$geoip_city_country_code”’;

telegraf config:
[[inputs.tail]]
files = ["/var/log/nginx/access.log"]
from_beginning = true
name_override = “nginx_access_log”
grok_patterns = ["%{LOG_FORMAT_CUSTOM}"]
grok_custom_patterns = ‘’’
LOG_FORMAT_CUSTOM [%{GREEDYDATA}]
‘’’

2

Hello @MervisDiamonds,
Can you please try setting from_beginning = false so you can tail your logs? It also looks like it’s not parsing your nginx log quite right. What do you expect the resulting line protocol to look like?

3

I apologize for the suspense. I corrected the problem by changing the permissions for the access.log file.

1 Like
4

What did you change them to? I suspect I have problem with permissions too…