Telegraf different measurements input from csv howto

Telegraf
1

I’m having problems with CSV input to influxdb using telegraf plugins. Seems that file plugin uses csv_header_row_count only from the first file read. But i have multiple csv files with different sets of metrics.
Telegraf when started reads header row from the first file of “files = [”/data/siem/input/*.csv"]" and uses it for the next file which has its own different header row.
Is there a way to change this behavior ?

2

Fixed that in source. Just removed the check if columns are already named. Below are diffs for
~/go/src/github.com/influxdata/telegraf/plugins/parsers/csv/parser.go

> 64c64
> <       if len(p.ColumnNames) == 0 {
> ---
> > //    if len(p.ColumnNames) == 0 {
> 84,89c84,89
> <       } else {
> <               // if columns are named, just skip header rows
> <               for i := 0; i < p.HeaderRowCount; i++ {
> <                       csvReader.Read()
> <               }
> <       }
> ---
> > //    } else {
> > //            // if columns are named, just skip header rows
> > //            for i := 0; i < p.HeaderRowCount; i++ {
> > //                    csvReader.Read()
> > //            }
> > //    }
3

This is a bug, would you be able to open a new issue on the Telegraf GitHub page?

4

Yup, did that

1 Like
5

Were you able to find a workaround for this problem? I tried creating an input section for each of the files I am reading. That works on the first time through, but subsequent intervals fail.

6

Unfortunately not. I preferred to write my own processing script using python & influxdb module. The source files structure appeared too complicated to fight against it with telegraf. Or more likely i have not enough experience with telegraf.

7

I was planning to create multiple input sections, which still didn’t work with the previous version. After updating to the current version, it appears to be working just fine now. I am able to read multiple files, all with headers, without issue. Thanks for the response.