Telegraf send only last line of CSV to influxDB

Telegraf
1

Hi ,
I am using telegraf to take data from CSV. But it only picks the last line from CSV multiple times.
Below mentioned is the telegraf conf and csv file used.

telegraf.conf
[[inputs.file]]
files = [“/home/telegraf/result.csv”]
data_format = “csv”
csv_header_row_count = 1
csv_skip_rows = 0
csv_skip_columns = 0
csv_comment = “#”
csv_measurement_column = “csv”

result.csv
DirectoryName,Filename,Size,CreationTime
external-configs,myapp-configmap.yaml,4,2022-09-02–10:50:30
external-configs,myapp-deployment-with-secrets-cm.yaml,4,2022-09-02–12:20:44
external-configs,myapp-secret.yaml,4,2022-09-02–11:18:13

Please let me know the further configuration to make this complete csv data available on influxdb.

Thanks

2

That config should read all the lines,can you share to output of telegraf? (run telegraf --config _file_.conf --test)

But I honestly think the problem is the destination… if you are writing to InfluxDB you should know difference between tags and fields, in your config I don’t see csv_tag_columns, which is very much needed in this sample case, as without it only “time” will be the key all the rest is going to be a field… multiple points with the same key will override/update the field values, that’s why you see only the last data point.

  ## Columns listed here will be added as tags. Any other columns
  ## will be added as fields.
  csv_tag_columns = []

note that csv_measurement_column = “csv” doesn’t seem right as “csv” is not an existing column

  ## The column to extract the name of the metric from. Will not be
  ## included as field in metric.
  csv_measurement_column = ""
3

Hey,

Thanks for the suggestion, it worked after adding the tag value. Please suggest how I can avoid the duplicate entries of the old data against the new timestamp.

4

since you have no tags you can’t filter out those old points… also the measurement structure is already polluted with “non-unique” keys meaning that Filename is both a tag and a field… which is an issue when querying the data.

I suggest you drop the whole measurement in order to start with a clean structure

1 Like