Telegraf inputs.win_eventlogs: question arround "from_beginning"

Telegraf
1

Bonjour/Hello

i’m trying to understand the correct span of the parameter “from_beginning” that is usable in inputs.win_eventlogs.

this is what we found in the documentation

  ## When true, event logs are read from the beginning; otherwise only future
  ## events will be logged.
  # from_beginning = false

it’s clear the search for event will be done from beginning of the logs.

just to be sure, now is it doing it only once when starting Telegraf and the next event search will be done only on the new one?

say, i’m looking for app crash, here the xml filter:

<QueryList>
  <Query Id="0" Path="Application">
    <Select Path="Application">*[System[(Level=2) and (EventID=1000)]]</Select>
  </Query>
</QueryList>

i put from_beginning = true in the config file.

and telegraf will execute the search every 2 min

so do Telegraf will execute a first search on all the logs, and the next look it will do it on new events only, or look from beginning?

i want to optimize my search on telegraf agent to set the less cpu/memory usage, while increasing the number of itm to search.

thanks for your lights!

2

Correct!

so do Telegraf will execute a first search on all the logs, and the next look it will do it on new events only, or look from beginning?
so my understanding is that if set to true it will search at all the logs each time. and false execute first search on all and then only new events.

@srebhan can confirm if I have that wrong.

3

Hello @Anaisdg

thanks for your fast response

In the same time i did more tests

if from_beginning = false, only new events (as expected)

if at True, it log all (as expected) and then wait for new events.

that push me on one question:

if we collect from_beginning it can collect some data very old:

  • influxdb bucket with max data of 30 days, will generate logs/ignore if events sent are older than 30 days

maybe i can ask an improvement of from_begining concept with something “older_than” ,

  • older_than = 0 → mean from begining
  • older_than = 30 ->look only for the last 30 days

do you know a way to avoid sending old data? (when you have 1000 devices sending old data that will be ignored, it’s not optimised :slight_smile:

regards

4

@tonic8 you can drop older messages already using the generally available metricpass option (see documentation). Something like

metricpass = "time > now() - duration('30d')"
5

hello Srebhan

so i stripped the config to the minimal just to validate the parsing (without the metricpass it take all errors)

when “metricpass = “time >= now() - duration(‘2d’)”” is disabled the logs are collected

when enabled there is a fail at startup with error parsing win_eventlog, type conversion error from ‘string’ to ‘google.protobuf.Duration’

  • if i put the metricpass in the input the message is error parsing win_eventlog
  • if i put the metricpass in the output the message is error parsing file array
C:\Workspace\Telegraf\Telegraf>telegraf_test_win_event.cmd
2024-12-18T17:50:12Z I! Loading config: C:\Workspace\Telegraf\sandbox\telegraf_test_win_event.conf
2024-12-18T17:50:12Z E! loading config file C:\Workspace\Telegraf\sandbox\telegraf_test_win_event.conf failed: error parsing win_eventlog, type conversion error from 'string' to 'google.protobuf.Duration'

Any suggestion? :sweat_smile:

here the config file with the input

[agent]
  ## Log at debug level.
  debug = true
  ## Log only error level messages.
  quiet = false

  logformat = "text"
  logfile = "C:/Workspace/Telegraf/sandbox/telegraf_test_win_event.log"


# # Input plugin to collect Windows Event Log messages
[[inputs.win_eventlog]]
  metricpass = "time >= now() - duration('2d')"
  
  xpath_query = '''
  	<QueryList>
  	  <Query Id="0" Path="Application">
  		<Select Path="Application">*[System[(Level=2)]]</Select>
  	  </Query>
  	</QueryList>
      '''

  from_beginning = true
  timestamp_from_event = true

[[outputs.file]]

  files = ['C:/Workspace/Telegraf/sandbox/telegraf_test_win_event.metrics']
    data_format = "influx"

# example data collected	
# win_eventlog,Channel=Application,Computer=Krokmou,EventID=24,Keywords=0x8000000000000000,Level=2,LevelText=Erreur,Source=Microsoft-Windows-WMI,host=Krokmou UserName="AUTORITE NT\\Système",EventRecordID=31i,ActivityID="{4d0758f0-4951-0003-315c-074d5149db01}",UserID="S-1-5-18",data_0x8000003F_Class="TbtTopologyChange",data_0x8000003F_Namespace="//./ROOT/Intel/Thunderbolt",TimeCreated="2024-12-08T09:12:56.6825676Z",Version=2i,TaskText="",data_0x8000003F_Query="SELECT * FROM TbtTopologyChange",Message="Le fournisseur d’événements  a tenté d’inscrire dans l’espace de noms //./ROOT/Intel/Thunderbolt la requête « SELECT * FROM TbtTopologyChange » dont la classe cible « TbtTopologyChange » n’existe pas. La requête sera ignorée.",OpcodeText="Informations" 1733649177000000000
6

Hello Srebhan hope you pass good end of the year. i did more tests but stay stuck on

error parsing file array, type conversion error from 'string' to 'google.protobuf.Duration'

my only guess is that “time is sent” as string, where it expect something else.

hope you can help me on this.

:slight_smile: happy end of year (i will say new year only next year !)