Telegraf logparser plugin is not working

Hi,
i’m trying to collect data from a csv file,
and output it to stdout, for some reason the process starts but nothing happens.

this is my telegraf config file:

# Telegraf configuration

# Use 'telegraf -config telegraf.conf -test' to see what metrics a config


# Global tags can be specified here in key="value" format.
[global_tags]
  # dc = "us-east-1" # will tag all metrics with dc=us-east-1
  # rack = "1a"

# Configuration for telegraf agent
[agent]
  interval = "10s"
  round_interval = true

  metric_buffer_limit = 1000
  flush_buffer_when_full = true

  collection_jitter = "0s"

  flush_interval = "10s"
  flush_jitter = "0s"

  ## Logging configuration:
  debug = false
  quiet = false
  logfile = "/Program Files/Telegraf/telegraf.log"

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""


###############################################################################
#                                  OUTPUTS                                    #
###############################################################################

# # Configuration for influxdb server to send metrics to
# [[outputs.influxdb]]
#   urls = ["http://192.168.21.128:8086"] # required
#   # The target database for metrics (telegraf will create it if not exists)
#   database = "telegraf" # required
#   precision = "s"
# 
#   timeout = "5s"
#   # username = "telegraf"
#   # password = "metricsmetricsmetricsmetrics"
#   # Set the user agent for HTTP POSTs (can be useful for log differentiation)
#   # user_agent = "telegraf"
#   # Set UDP payload size, defaults to InfluxDB UDP Client default (512 bytes)
#   # udp_payload = 512
# 
[[outputs.file]]
  ## Files to write to, "stdout" is a specially handled file.
  files = ["stdout"]

  ## Data format to output.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
  data_format = "influx"

  ## Additional configuration options go here

###############################################################################
#                                  INPUTS                                     #
###############################################################################
[[inputs.logparser]]
  ## Log files to parse.
  files = ["C:/SafesList_AMAN_IBV.csv"]

  from_beginning = false

  ## Method used to watch for file updates.  Can be either "inotify" or "poll".
  # watch_method = "inotify"

  ## Parse logstash-style "grok" patterns:
  ##   Telegraf built-in parsing patterns: https://goo.gl/dkay10
  [inputs.logparser.grok]
    ## This is a list of patterns to check the given log file(s) for.
    ## Note that adding patterns here increases processing time. The most
    ## efficient configuration is to have one pattern per logparser.
    ## Other common built-in patterns are:
    ##   %{COMMON_LOG_FORMAT}   (plain apache & nginx access logs)
    ##   %{COMBINED_LOG_FORMAT} (access logs + referrer & agent)
    patterns = ["%{GREEDYDATA:batata}"]

    ## Name of the outputted measurement name.
    measurement = "apache_access_log"

    ## Full path(s) to custom pattern files.
    custom_pattern_files = []

    ## Custom patterns can also be defined here. Put one pattern per line.
    custom_patterns = '''
    '''

    ## Timezone allows you to provide an override for timestamps that
    ## don't already include an offset
    ## e.g. 04/06/2016 12:41:45 data one two 5.43µs
    ##
    ## Default: "" which renders UTC
    ## Options are as follows:
    ##   1. Local             -- interpret based on machine localtime
    ##   2. "Canada/Eastern"  -- Unix TZ values like those found in https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
    ##   3. UTC               -- or blank/unspe	cified, will return timestamp in UTC
    timezone = "Local"

this is what the log file is showing:

2018-04-12T08:08:29Z I! Starting Telegraf v1.5.3
2018-04-12T08:08:29Z I! Loaded outputs: file
2018-04-12T08:08:29Z I! Loaded inputs: inputs.logparser
2018-04-12T08:08:29Z I! Tags enabled: host=LAP-DAVIDG
2018-04-12T08:08:29Z I! Agent Config: Interval:10s, Quiet:false, Hostname:"LAP-DAVIDG", Flush Interval:10s 

and this is a sample from the csv file:

21,"BLMS",0,"\",651,50,1,"08/04/2018 22:00:18","NO","NO",7,56,0,0,24,0,0,0,90,0,30,5,1,0,7,"YES","NO","Administrator","NO","NO","NO","YES","25/03/2010 15:47:56","Administrator","-1"
24,"YoavM",0,"\",182318,500,35,"09/04/2018 11:10:20","NO","NO",7,56,0,0,24,0,0,0,90,7,30,5,1,0,7,"YES","NO","Administrator","YES","NO","NO","NO","13/04/2010 14:42:32","Administrator","0"
29,"Kostya",0,"\",0,5000,0,"08/04/2018 22:00:18","NO","NO",7,56,0,0,24,0,0,0,90,7,30,5,1,0,7,"YES","NO","Administrator","YES","NO","NO","YES","02/08/2010 14:38:51","Administrator","-1"
68,"Meitav",0,"\",0,50,0,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,90,0,30,0,1,0,7,"YES","NO","Administrator","YES","NO","NO","YES","16/08/2010 15:55:42","Administrator","-1"
86,"VaultInternal",0,"\",12,50,0,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,90,90,30,5,0,0,7,"NO","NO","Administrator","YES","NO","NO","YES","15/09/2010 16:00:24","Administrator","-1"
125,"Vault Explorer",0,"\",32939,50,64,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,90,3,30,0,1,0,7,"YES","NO","Master","YES","NO","NO","YES","29/11/2010 09:23:56","yoav_admin","-1"
129,"Clalit",0,"\",0,50,0,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,90,7,30,0,1,0,7,"YES","NO","Administrator","YES","NO","NO","NO","09/12/2010 12:05:11","Administrator","0"
131,"Harel",0,"\",0,150,0,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,90,30,30,5,1,2,7,"YES","NO","Administrator","YES","YES","NO","NO","21/12/2010 13:35:56","Administrator","-1"
251,"SFE_conf",0,"\",169,50,0,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,14,5,30,5,0,0,7,"NO","NO","Administrator","YES","NO","NO","NO","17/07/2011 14:12:51","Administrator","-1"
252,"SFE_userpref",0,"\",382,50,0,"08/04/2018 22:00:19","NO","NO",7,56,0,0,24,0,0,0,14,5,30,5,0,0,7,"NO","NO","Administrator","NO","NO","NO","NO","17/07/2011 14:12:52","Administrator","-1"

thanks,
david.

If this is set to false only new data appended to the file after the process is started in parsed.

2 Likes

didn’t see that :slight_smile:
thanks alot.
SOLVED