with telegraf config for tail input plugin (output is influxdb):
[[inputs.tail]]
files = ["/data/siem/input/*.csv"]
data_format = "csv"
from_beginning = true
watch_method = "poll"
csv_header_row_count = 1
csv_measurement_column = "series"
csv_tag_columns = ["host","vpnname","vpnid","servname","servid","mcc","mnc","lac","rac","card","port","maxrate","vlrname"]
csv_column_types = ["float"]
csv_timestamp_column = "epochtime"
csv_timestamp_format = "unix"
fielddrop = ["localdate","localtime","uptime"]
i’m getting error on the input metric:
2019-09-04T12:05:15Z E! [inputs.tail] Error in plugin: malformed log line in /data/siem/input/host1-cardSch1.csv: [host1,cardSch1,
1567596901,20190904,143501,2549116,12,0.92,99.08,275,16137,98304,1,0.92,99.08,275,16137,98304,cpu0,0.54,0.36,0.00,0.02,99.08,0.00,0.00,0], E
rror: column type: column count exceeded
but if i remove csv_column_types = [“float”] from this config, everything works fine. The problem is that to be compatible with existing database i need all fields to be float except tags and measurement name column.
There are 27 fields and 27 fields in the header row. Header row is:
host,series,epochtime,localdate,localtime,uptime,card,cpubusy,cpuidle,numproc,memused,memtotal,numcpu,cpu0-cpubusy,cpu0-cpuidle,cpu0-numproc
,cpu0-memused,cpu0-memtotal,cpu0-name,cpu0-cpuused-user,cpu0-cpuused-sys,cpu0-cpuused-io,cpu0-cpuused-irq,cpu0-cpuused-idle,cpu1-cpubusy,cpu
1-cpuidle,cpu1-numproc