Parse csv with telegraf and write into influx

Telegraf
,
1

I have files which I need to parse once in 1h, file contains around 9k lines separated with comma

Telegrad config
[[inputs.file]]
files = [“/tmp/*.log”]
name_override = “test”

data_format = “csv”
csv_header_row_count = 0
csv_column_names = [“time”,“hostname”,“host”,“field1”,“field2”,“field3”,“value”]
csv_delimiter = “,”

Sample data

2024-03-03 13:00:00,hostname,MANAGER,Total_Disk_Usage_of_/opt/log/var,8
2024-03-03 13:00:00,hostname,SERVER@test-033-occ1,1_19089 CXC_1730371_01,DDS_CPU_Usage_of_OCC_Node,0

For some reason, when I write to influx, I got only one line in influx
If I do outputs.file I got all measurements in file
[[outputs.file]]
files = [“stdout”, “/tmp/metrics.out”]
data_format = “influx”

What could be the issue when writing into influxdb?

2

This is usually the common issue that your data is not unique and is treated as duplicate time-series data points. InfluxDB uses line protocol. A unique point in line protocol is a field value with a unique metric name, timestamp, tag set. If all three of those match then the data are considered identical. The following are all identical and only the last line would get written:

foo,drive=bar value=42
foo,drive=bar value=43

To include both of these values you would need to either a unique timestamp on each or add a tag to differentiate them like:

foo,host=a,drive=bar value=42
foo,host=b,drive=bar value=43