Telegraf log parser load initial data only

,
1

HI ,

Im tying to load some IIS logs into influx but im having a weird issue telegraf is loading all the logs and the stop loading new entries…

this is the config

[inputs.logparser.grok]
## This is a list of patterns to check the given log file(s) for.
## Note that adding patterns here increases processing time. The most
## efficient configuration is to have one pattern per logparser.
## Other common built-in patterns are:
## %{COMMON_LOG_FORMAT} (plain apache & nginx access logs)
## %{COMBINED_LOG_FORMAT} (access logs + referrer & agent)
patterns = [“%{TIMESTAMP_ISO8601:timestamp} %{IP:sourceip} %{WORD:method} %{URIPATH:uristem} (?:-|%{NOTSPACE:uriquery}) %{POSINT:port} (?:-|%{NOTSPACE:username}) %{IP:clientip} %{NOTSPACE:useragent} %{NUMBER:status} %{NUMBER:substatus} %{NUMBER:win32status} %{NUMBER:timetaken}”]

## Name of the outputted measurement name.
measurement = "bzppr_iis"

## Full path(s) to custom pattern files.
custom_pattern_files = []

## Custom patterns can also be defined here. Put one pattern per line.
custom_patterns = '''
    timezone = "Local"
'''

## Timezone allows you to provide an override for timestamps that 
## don't already include an offset
## e.g. 04/06/2016 12:41:45 data one two 5.43µs
##
## Default: "" which renders UTC
## Options are as follows:
##   1. Local             -- interpret based on machine localtime
##   2. "Canada/Eastern"  -- Unix TZ values like those found in https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
##   3. UTC               -- or blank/unspecified, will return timestamp in UTC

this is how its look on the graph
SELECT count(“status”) AS “count_status” FROM “telegraf”.“autogen”.“bzppr_iis” WHERE time > :dashboardTime: GROUP BY :interval:

2

I don’t understand what the expected output is, is the problem that the graph line should continue until the end of the selected time range?

3

@daniel i expect output is that new enteritis in the log file will be written to influx

4

Is there any event occurring when the log entries end, in particular does it stop when the logfile is rotated?

5

i didn’t understand the question

This is the scenario lets say i configure the log parser with the from beginning true

2017/08/15 10:47:26 Seeked D:\InetLogs\W3SVC1\u_ex170815.log - &{Offset:0 Whence:0}
the logs are written to file , but lines that appear in the file after 2017/08/15 10:47:26 wont written to influx .

if i set the from begging to false noting is written to influx.

6

I think you are running into issue #2981, it seems there are some issues with this plugin in Windows.

1 Like
7

@daniel this is it ?
do you think it will be merged soon ?

8

I think there is a good chance this would fix it but my comments on the pull request need to be addressed before it can be merged.