I’m trying to get telegraf to record the nginx access logs using telegraf’s log parser and send it to influxdb.
Telegraf is definitely recording the nginx stub status and sending it to influxdb, but the access logs for the log parser aren’t being recorded.
The problem is that nginx’s official Docker image redirects /var/log/nginx/access.log to stdout, so if you exec into the container and do cat /var/log/nginx/access.log, you won’t actually see anything. You have to do docker logs <id of container> to get the access logs.
Telegraf needs access to the log files, and they don’t have a Docker log parser, so what I did is created a custom /var/log/nginx/custom/access.log in my nginx container and have the logs written there.
Then I created a shared nginxlog volume so that the telegraf container can read the logs.
I can do cat /var/log/nginx/custom/access.log from within the nginx docker container and see the logs get output, but when I do cat /var/log/nginx/custom/access.log in the telegraf container, it shows an empty file, so the file isn’t being synced somehow.
Keep in mind I needed to create an empty access.log on my host to mount it into the nginx container initially.
Here is my docker-compose.yml file:
version: "3.4"
services:
influxdb:
container_name: influxdb
restart: always
image: influxdb:1.7-alpine
ports:
- "8087:8086" # http
- "8084:8083" # admin
- "8090:8089/udp" # udp
environment:
INFLUXDB_REPORTING_DISABLED: "true"
INFLUXDB_DATA_QUERY_LOG_ENABLED: "true"
INFLUXDB_HTTP_AUTH_ENABLED: "false"
INFLUXDB_ADMIN_USER: "admin"
INFLUXDB_ADMIN_PASSWORD: "admin"
INFLUXDB_UDP_ENABLED: "true"
INFLUXDB_UDP_DATABASE: "metrics"
INFLUXDB_RETENTION_ENABLED: "false"
volumes:
- ./influxdb:/var/lib/influxdb
networks:
log:
ipv4_address: 172.25.0.5
nginx:
image: nginx:1.15-alpine
ports:
- "8008:80"
volumes:
- nginxlog:/var/log/nginx
- ./nginx.conf:/etc/nginx/nginx.conf
- ./index.html:/var/www/index.html
- ./access.log:/var/log/nginx/custom/access.log
- ./error.log:/var/log/nginx/custom/error.log
networks:
log:
ipv4_address: 172.25.0.2
telegraf:
restart: always
image: telegraf:1.9-alpine
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
- nginxlog:/var/log/nginx
networks:
log:
ipv4_address: 172.25.0.4
volumes:
nginxlog:
networks:
log:
driver: bridge
ipam:
config:
- subnet: 172.25.0.0/24
Here is my telegraf.conf file:
[agent]
hostname = "nginx_frontend"
[[outputs.influxdb]]
urls = ["http://172.25.0.5:8086"]
database = "nginx_frontend"
[[inputs.nginx]]
urls = ["http://172.25.0.2:80/nginx_status"]
response_timeout = "5s"
[[inputs.logparser]]
files = ["/var/log/nginx/access.log"]
from_beginning = true
name_override = "nginx_access_log"
[inputs.logparser.grok]
patterns = ["%{COMBINED_LOG_FORMAT}"]
[[inputs.docker]]
endpoint = "unix:///var/run/docker.sock"
[[outputs.file]]
files = ["stdout", "/tmp/nginx.out"]
nginx.conf:
worker_processes auto;
events {
worker_connections 1000;
}
http {
server_tokens off;
server {
listen 80 default_server;
index index.html;
root /var/www/html;
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow 172.25.0.4;
deny all;
}
}
}
As a workaround I ended up mounting the host’s access.log into the telegraf container instead of doing the shared volume.
I don’t like the idea of one giant nginx log file, so is there a better way to log nginx access logs from a Docker container w/o having to resort to creating a physical access log?
