Hi all,
I am using the following input config to parse out IIS logs in the server:
## logparser for IIS logs
[[inputs.file]]
files = ["E:/inetpub/logs/W3SVC1/u_ex211029_x-Clean.log"]
name_override = "iis_api"
data_format = "grok"
grok_patterns = ["%{TS_IIS:timestamp:tag} %{NOTSPACE} %{IPORHOST} %{WORD:http_request_method:tag} %{NOTSPACE:url_path:tag} %{NOTSPACE} %{NUMBER} %{NOTSPACE} %{IPORHOST} %{NOTSPACE} %{GREEDYDATA} %{GREEDYDATA} %{GREEDYDATA} %{GREEDYDATA} %{NUMBER:http_response_status_code:tag} %{NUMBER} %{NUMBER} %{NUMBER} %{NUMBER} %{NUMBER:time_taken} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE}"]
For testing, I am using the below output plugin to console in prometheus format:
[[outputs.file]]
files = ["stdout"]
prometheus_export_timestamp = false
prometheus_sort_metrics = false
prometheus_string_as_label = true
data_format = "prometheus"
However, in the command line logs, I don’t see any metrics created and I see that the Telegraf has processed the first 3 lines in the logfile but not the rest of the lines. Refer to the part in the red square below.
This is a sample of the IIS log file that I am trying to read.
#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2021-10-29 00:00:09
#Fields: date time s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken TRUE-CLIENT-IP Org-Src-IP CF-Connecting-IP CF-Ray X-EdgeConnect-Session-ID CF-Request-ID X-Forwarded-For
2021-10-29 15:19:14 MYHOSTNAME123 10.211.142.205 GET /MYSITE/ - 443 - 10.211.224.19 HTTP/1.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/94.0.4606.71+Safari/537.36+Edg/94.0.992.38 agent-authn-tx-gRHhrwVyadGmltPynsoSc6OVsxE=eAEVysEKQDAcB+B3+Z0nhkm77cSFlCeQrZbYFn8heXecv+/Gvs6QiDvVx2BYyGpIzmD1ukHeGL0j4yiiK5jv4WEwZ/hKkQmRcs4TBufd+OMQRDPlZciqti4VHXheOkYb1Q==;+ASP.NET_SessionId=gtywb2rdogytx3pakkf2q4dp;+UnicaNIODID=nCkkRWu4Rnb-cxPHYv2;+b2bamep=;+BlockSHPAccess=;+agent-authn-tx-nb9XUObqz7t8nCu_Ii3WCfMTLHg=eAEVyr0KgCAUBtB3+Waj7MfBrV0ocGpNBYe6RioU0rtX8zkF+dwgUc+jrsGwJ28hOYO3Z4QsMIGSo1Sl+3Dfw8PgruMrohuGlnPRM1Ag86NWgvfrkjNNTVDR43kBPCQcMw==;+agent-authn-tx-c54cCbQpMAm95-e5YvoZOnGtuYo=eAEVykELQDAYBuD/8p4nhim7KTk5KE6us5L4xoxI++84P8+Dw86QCJuiDcGwuHGA5AzjYHfIB8qQ0+QCd6/6e/AM+lq/kiVCxJznEQMZUj+aYjtL0VXUz/U0pQv8C0GVHG8=;+agent-authn-tx-9JVnQXR6Oz3b3oNC-3qD0_Cc1Fc=eAEVyjEKgCAYBtC7fLMhmja41RZIBB0hhYYysb8opLtn83sZZ1phwMd24mDYaHEwgmFx6YDJmPdAPlBFT/Tl4WXwdyylqbWWQirNEPYw/zgo0fvNddOlLbU24f0AOyscAg==;+b2bam-agent-jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJka3hrMTA2MTF1IiwiYXVkaXRUcmFja2luZ0lkIjoiM2JiZmI2M2YtOTRmOS00YmQ0LTk4ZDQtZGJiMWI4Zjg0ZjgyLTM0MDA4Mjc0IiwiaXNzIjoiaHR0cHM6Ly9iMmJhbXFhZ2wudmlzYS5jb206ODQ0My9vcGVuYW0vb2F1dGgyL3JlYWxtcy9yb290L3JlYWxtcy92b2wiLCJ0b2tlbk5hbWUiOiJpZF90b2tlbiIsIm5vbmNlIjoiTjQxSWVtZEJTdjVMdEFMciIsImF1ZCI6InByZXBhaWRwYXMtMDAxOTUwMS1BZ2VudCIsImFjciI6IjAiLCJzX2hhc2giOiJzTkdOVFkzcHYzdmlPdVk1dmZpa3BBIiwiYXpwIjoicHJlcGFpZHBhcy0wMDE5NTAxLUFnZW50IiwiYXV0aF90aW1lIjoxNjM1NTIwNjM3LCJmb3JnZXJvY2siOnsic3NvdG9rZW4iOiJJMDhnLXhXaFN3U19YUnZzWVNNdThUaFBHQzQuKkFBSlRTUUFDTVRRQUFsTkxBQnd2UXpBdlJuaE9TM3BoYkRaV1RreFhSalpsUzJSeVZ6aENXWGM5QUFSMGVYQmxBQU5EVkZNQUFsTXhBQUl3TWcuLioiLCJzdWlkIjoiM2JiZmI2M2YtOTRmOS00YmQ0LTk4ZDQtZGJiMWI4Zjg0ZjgyLTM0MDA4MTUxIn0sInJlYWxtIjoiL3ZvbCIsImV4cCI6MTYzNTU2Mzg0NiwidG9rZW5UeXBlIjoiSldUVG9rZW4iLCJpYXQiOjE2MzU1MjA2NDYsImFnZW50X3JlYWxtIjoiL0FnZW50UHJvZmlsZXMifQ.EG-mall2Cjx9PFf08Ofna6QkAVcOhKPtYpQY9x868kEBdxD-Tyq4sRqNhI_xm2tDU_XxB6IvRkB7XAsCCEoGh49A6BHmsvIIDIt6H6cePJire4idzRJ5HWQNkXwSL51TY60drHV93_8_tsZAIMjaA7vH41SY40pXjz_kwITnGEhKt0LzGls7iVIif3UH3_YUi3O7wEA3wu2TtcPI34fen9-aCIJAf0VkGfE9QO3QgnJwbJd2qVMadzAly1pt4vtRCFYOuBPrEaHq8ffG4mkkZRBrHv4ko5Qs8fMPosY730q5C_uLN6Qah9cSPk4bxcWDiqyoq3dVvGmuCrvoOUNo9g https://secure.company.com/ qa.mydomainweb.com 302 0 0 1535 3000 15 - - - - - - 10.55.170.15
2021-10-29 15:19:15 MYHOSTNAME123 10.211.142.205 GET /MYSITE/ - 443 dkxk10611u 10.211.224.19 HTTP/1.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/94.0.4606.71+Safari/537.36+Edg/94.0.992.38 agent-authn-tx-gRHhrwVyadGmltPynsoSc6OVsxE=eAEVysEKQDAcB+B3+Z0nhkm77cSFlCeQrZbYFn8heXecv+/Gvs6QiDvVx2BYyGpIzmD1ukHeGL0j4yiiK5jv4WEwZ/hKkQmRcs4TBufd+OMQRDPlZciqti4VHXheOkYb1Q==;+ASP.NET_SessionId=gtywb2rdogytx3pakkf2q4dp;+UnicaNIODID=nCkkRWu4Rnb-cxPHYv2;+b2bamep=;+BlockSHPAccess=;+agent-authn-tx-nb9XUObqz7t8nCu_Ii3WCfMTLHg=eAEVyr0KgCAUBtB3+Waj7MfBrV0ocGpNBYe6RioU0rtX8zkF+dwgUc+jrsGwJ28hOYO3Z4QsMIGSo1Sl+3Dfw8PgruMrohuGlnPRM1Ag86NWgvfrkjNNTVDR43kBPCQcMw==;+agent-authn-tx-c54cCbQpMAm95-e5YvoZOnGtuYo=eAEVykELQDAYBuD/8p4nhim7KTk5KE6us5L4xoxI++84P8+Dw86QCJuiDcGwuHGA5AzjYHfIB8qQ0+QCd6/6e/AM+lq/kiVCxJznEQMZUj+aYjtL0VXUz/U0pQv8C0GVHG8=;+agent-authn-tx-9JVnQXR6Oz3b3oNC-3qD0_Cc1Fc=eAEVyjEKgCAYBtC7fLMhmja41RZIBB0hhYYysb8opLtn83sZZ1phwMd24mDYaHEwgmFx6YDJmPdAPlBFT/Tl4WXwdyylqbWWQirNEPYw/zgo0fvNddOlLbU24f0AOyscAg==;+agent-authn-tx-EBigRw98LsYctZXjufNy30Z84SQ=eAEVyr0OQDAUBtB3+eaK+Kmhm8FmEBZ7VTrUpXUlpPHuaj4n4goOCvnQTjkENrYLVCFgl3BCReid2BBn/BwmPbwC5j5SaSopy6KStQDtpH/snevIzd7rjWs7rng/Rskc5w==;+b2bam-agent-jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJka3hrMTA2MTF1IiwiYXVkaXRUcmFja2luZ0lkIjoiM2JiZmI2M2YtOTRmOS00YmQ0LTk4ZDQtZGJiMWI4Zjg0ZjgyLTM0MDA5NTYxIiwiaXNzIjoiaHR0cHM6Ly9iMmJhbXFhZ2wudmlzYS5jb206ODQ0My9vcGVuYW0vb2F1dGgyL3JlYWxtcy9yb290L3JlYWxtcy92b2wiLCJ0b2tlbk5hbWUiOiJpZF90b2tlbiIsIm5vbmNlIjoiTGxsRW5sWHFxY210NGhSZiIsImF1ZCI6InByZXBhaWRwYXMtMDAxOTUwMS1BZ2VudCIsImFjciI6IjAiLCJzX2hhc2giOiIyNW1oYTZacUFxRkYyUUpsdklHdEhBIiwiYXpwIjoicHJlcGFpZHBhcy0wMDE5NTAxLUFnZW50IiwiYXV0aF90aW1lIjoxNjM1NTIwNzQzLCJmb3JnZXJvY2siOnsic3NvdG9rZW4iOiJCNmFzbkVqTVFmc1R2SW45cmJtZ2RrOS1hNGsuKkFBSlRTUUFDTVRRQUFsTkxBQnhIWW0xaU9WQkRTVW9yVDFwa05GTnVkRWhtVlRRNVNVbDZVVVU5QUFSMGVYQmxBQU5EVkZNQUFsTXhBQUl3TWcuLioiLCJzdWlkIjoiM2JiZmI2M2YtOTRmOS00YmQ0LTk4ZDQtZGJiMWI4Zjg0ZjgyLTM0MDA5NDAxIn0sInJlYWxtIjoiL3ZvbCIsImV4cCI6MTYzNTU2Mzk1NCwidG9rZW5UeXBlIjoiSldUVG9rZW4iLCJpYXQiOjE2MzU1MjA3NTQsImFnZW50X3JlYWxtIjoiL0FnZW50UHJvZmlsZXMifQ.nT1gtY1gu2UbvmZZMsGRB_4mRkQQBtybfbeFPvCZLP_2IGV1FICXokXeqQDOoiGQ2cws6Nk8aK_QDoK1kkyiiuKXk6VkhFHXeDjUQZTCs-fh59hsihKmuXKGB4UUAY8AGz6iLlNnonL1e-9kOiTrRMhbBHhAzRgF7v-EbjwiGT4QKiTmwUR0fY7fmCDCg4GUPpuq5HuERMwzL23fItutqlROmDbd425OKXEh6ZNhrFDL1hdwEcB7SvoFCkAm5lqP6dzLGlmeQLkv7Zp7_O9jPrdhX7FifThoovWMUQ5I-Toa-tUy-CZGZMxbiCJ1dA-sBfPM8-7mvyJ0lNv7H8E4QQ https://aic.1qa.company.com/ qa.mydomainweb.com 302 0 0 736 3180 218 - - - - - - 10.55.170.15
2021-10-29 16:36:48 MYHOSTNAME123 10.211.142.205 POST /MYSITE/SubUrl/webpage.aspx nav=1 443 username123 10.211.224.18 HTTP/1.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/94.0.4606.81+Safari/537.36 agent-authn-tx-JWx3aGxS13Bj-sD0Ky0e-DqH_Tg=eAGrViotylGyUtIPcAzWV9JRyi3JSFGyMtRRykgpKlayqlZKzs8rSc0r0S2pLEgFqlOq1VFKrSgAKjEzNjU1MrU0B6rNy89LBkm6uUZZGlSlOAWZFUWlmke4KtUCADt7G9Y=;+ASP.NET_SessionId=zaw5vdlnyfjrxzxqylzbrlvw;+__RequestVerificationToken_L0lkZW50aXR50=craq6w6ASKN_gDbzEIDIn5u6Aro-ruNxTaxfHo9VfrsBr8_DzvrmAs7Pg3DNYADb9qel3LWKlGHCiWURpewNDvYM6d01;+amlbcookie=qh1;+b2bamtokenq=HAeef94-MLr3VvmVYjx1PAdheqc.*AAJTSQACMTQAAlNLABw3Zm05bStXSHBTWXlxQXdPekV6dWJRT0JNWEU9AAR0eXBlAANDVFMAAlMxAAIxMg..*;+gRedirect=N;+b2bam-agent-jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJndHFsOTY5OTl1IiwiYXVkaXRUcmFja2luZ0lkIjoiM2JiZmI2M2YtOTRmOS00YmQ0LTk4ZDQtZGJiMWI4Zjg0ZjgyLTM0MDU4OTQyIiwiaXNzIjoiaHR0cHM6Ly9iMmJhbXFhLnZpc2EuY29tOjg0NDMvb3BlbmFtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvdm9sIiwidG9rZW5OYW1lIjoiaWRfdG9rZW4iLCJub25jZSI6ImZ3NWZ3dExJSzZ3T1RBS2YiLCJhdWQiOiJwcmVwYWlkcGFzLTAwMTk1MDEtQWdlbnQiLCJhY3IiOiIwIiwiYXpwIjoicHJlcGFpZHBhcy0wMDE5NTAxLUFnZW50IiwiYXV0aF90aW1lIjoxNjM1NTI1Mzg1LCJmb3JnZXJvY2siOnsic3NvdG9rZW4iOiJIQWVlZjk0LU1McjNWdm1WWWp4MVBBZGhlcWMuKkFBSlRTUUFDTVRRQUFsTkxBQnczWm0wNWJTdFhTSEJUV1hseFFYZFBla1Y2ZFdKUlQwSk5XRVU5QUFSMGVYQmxBQU5EVkZNQUFsTXhBQUl4TWcuLioiLCJzdWlkIjoiYjk1Yzk1ZGYtZDczZC00OWU2LWE4ODktMjk4YzBkOGVlNDEzLTM0NDMxNzA1In0sInJlYWxtIjoiL3ZvbCIsImV4cCI6MTYzNTU2ODU4NSwidG9rZW5UeXBlIjoiSldUVG9rZW4iLCJpYXQiOjE2MzU1MjUzODUsImFnZW50X3JlYWxtIjoiL0FnZW50UHJvZmlsZXMifQ.WknhdKJErbtZvmW1jBoPYCkKUE1PmY8sQQ3Wcy-QPtonPjoaRO8HWkxwlutSXFZBzUbPX6NNQoGm0w8pmwf9zOihBEPsF22mTeHmEdH4E-8MVBigpbfo3DbpJ99U6odg7PTzuu5QW7qdYfV8TJhqLxHv_d4d0KwnWSQYATdJNDrDu3LCcFQV0G0NeES2X7O9DXYy7BdElK5qxPXZKLyzmfZMbRG-hfj0pkfgbe-qwYC7YoPt_A1_QWePfgQFTv59DfTGw06yGTSVV4cIcy9oAmnIGvJP88M3q013Kc1fHCJPDSwVsZvyZvaZo-zchYp8dMCQ2J_SxXVRznLeR9Lwew;+UnicaNIODID=Ouhgga6CjCS-cxPb0hO;+b2bamep=;+BlockSHPAccess=true https://qa.mydomainweb.com/MYSITE/SubUrl/webpage.aspx?nav=1 qa.mydomainweb.com 200 0 0 34031 5291 489 - - - - - - 10.55.216.58
To narrow down the issue, I’ve tried to remove some the data in the logs and test whether any metrics will be produced, but still it seems there are no metrics created. I’ve tested the pattern in the grok debugger tool and it was successful, but still I don’t see any lines getting processed in Telegraf. Note that the second line should fail and give an error since it does not match the pattern I’ve provided but even that part I don’t see in the console.
2021-10-29 15:19:14 MYHOSTNAME123 10.211.142.205 GET /MYSITE/ - 443 - 10.211.224.19 HTTP/1.1 https://referrersite.com/ randomsiteurl.com 302 0 0 1535 3000 15
2021-10-29 15:19:14 MYHOSTNAME123 10.211.142.205 GET /MYSITE/postendpoint.aspx - 443 - 10.211.224.19 HTTP/1.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/94.0.4606.71+Safari/537.36+Edg/94.0.992.38 agent-authn-tx-9JVnQXR6Oz3b3oNC-3qD0_Cc1Fc=eAEVyjEKgCAYBtC7fLMhmja41RZIBB0hhYYysb8opLtn83sZZ1phwMd24mDYaHEwgmFx6YDJmPdAPlBFT/Tl4WXwdyylqbWWQirNEPYw/zgo0fvNddOlLbU24f0AOyscAg==;+agent-authn-tx-EBigRw98LsYctZXjufNy30Z84SQ=eAEVyr0OQDAUBtB3+eaK+Kmhm8FmEBZ7VTrUpXUlpPHuaj4n4goOCvnQTjkENrYLVCFgl3BCReid2BBn/BwmPbwC5j5SaSopy6KStQDtpH/snevIzd7rjWs7rng/Rskc5w== https://referrer.com/ randomsiteurl.com 302 0 0 1836 2550 15 - - - - - - 10.55.170.15
New pattern used (should match 1st line but not 2nd line):
%{YEAR}[/-]%{MONTHNUM}[/-]%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{NOTSPACE} %{IPORHOST} %{WORD:http_request_method:tag} %{NOTSPACE:url_path:tag} %{NOTSPACE} %{NUMBER} %{NOTSPACE} %{IPORHOST} %{NOTSPACE} %{GREEDYDATA} %{GREEDYDATA} %{NUMBER:http_response_status_code:tag} %{NUMBER} %{NUMBER} %{NUMBER} %{NUMBER} %{NUMBER:time_taken}
My understanding is that I should at least see the error log “Grok no match found for:” in the second line but I don’t seem to know what is happening and if Telegraf is even parsing the line.
Can anyone help on checking what the issue is? Thank you!!
