Telegraf and Influx dont get info from GeoIP - nginx

Telegraf
, ,
1

I am setting up nginx metrics and geo maps visualization in Grafana… Nginx metrics part is working, I managed to get information, but for the GeoIP map I am not so lucky…

/etc/nginx/nginx.conf
##
# Logging Settings
##
# Add GEO IP support

        geoip2 /opt/geoip2/GeoLite2-City.mmdb {
        auto_reload 5m;
        $geoip2_data_country_iso_code country iso_code;
        $geoip2_data_city_name city names en;
        }

        log_format custom '$remote_addr - $remote_user [$time_local]'
                   '"$request" $status $body_bytes_sent'
                   '"$http_referer" $host "$http_user_agent"'
                   '"$request_time" "$upstream_connect_time"'
                   '"$geoip2_data_city_name" "$geoip2_data_country_iso_code"';
        access_log /var/log/nginx/access.log custom;

/etc/telegraf/telegraf.conf

   urls = ["http://localhost/nginx_status"]
   response_timeout = "5s"

[[inputs.tail]]
  files = ["/var/log/nginx/access.log"]
  from_beginning = false
  name_override = "nginx_access_log"
  [inputs.logparser.grok]
    patterns = ["%{COMBINED_LOG_FORMAT}"]

[[inputs.tail]]
  files = ["/var/log/nginx/access.log"]
  from_beginning = false
  name_override = "nginx_access_log"
   [[inputs.tail]]
     data_format = "grok"
     grok_patterns = ["%{CUSTOM_LOG_FORMAT}"]
     grok_custom_patterns = '''
     CUSTOM_LOG_FORMAT %{CLIENT:client_ip} %{NOTSPACE:ident} %{NOTSPACE:auth} \[%{HTTPDATE:ts:ts-httpd}\]"(?:%{WORD:verb:tag} %{NOTSPACE:request}(?: HTTP/%{NUMBER:http_version:float})?|%{DATA})" $%{NUMBER:resp_code:tag} (?:%{NUMBER:resp_bytes:int}|-)%{QS:referrer} %{QS:agent}%{QS:request_time} %{QS:upstream_connect_time}%{QS:geoip__city} %{QS:country_code}
'''
2

@Anaisdg I feel like you can help me… :sweat_smile:

1 Like
3

Hello @network ,
I’m not so sure I can but I will get you an answer.
I see that others are using this grok pattern":

   [inputs.logparser.grok]
     patterns = ["%{CUSTOM_LOG_FORMAT}"]
     custom_patterns = '''
        CUSTOM_LOG_FORMAT %{CLIENT:client_ip} %{NOTSPACE:ident} %{NOTSPACE:auth} \[%{HTTPDATE:ts:ts-httpd}\]"(?:%{WORD:verb:tag} %{NOTSPACE:request}(?: HTTP/%{NUMBER:http_version:float})?|%{DATA})" %{NUMBER:resp_code:tag} (?:%{NUMBER:resp_bytes:int}|-)%{QS:referrer} %{QS:agent}%{QS:request_time} %{QS:upstream_connect_time}%{QS:geoip_city} %{QS:country_code}
      '''

From Total Nginx monitoring, with application performance and a bit more, using Telegraf/InfluxDB/Grafana. | by Alexey Nizhegolenko | FAUN Publication

Is that helpful to you? Or have you already seen this?