Telegraf CSV Plugin Problems

Telegraf
,
1

Hi,

this is my first topic in this forum, so be kind with me as im not a very experienced user of influxdb nor telegraf.
I have two problems, the first one is the most important one two me, the second one i can work with but it would be nice to have a solution for this too. But lets start with the first one.

Problem 1:
I have multiple csv files with each of them containing 7300 lines of data. Each dataset looks like this:

  mean_0.25,1630485598417,0,0,0,-5.18,-107.24,105.26,-3.08,51.08,5.03

I want to write them to influxdb (Version 1.8.5.1) with telegraf (Version 1.19.2). im using the csv plugin i found here CSV input data format | Telegraf 1.19 Documentation. After starting the influx server i always check first if its is running correctly with the influx cli client. when everything is ok i start telegraf. at first it writes around 5600 of the 7300 as expected, but then it somehow stops. atleast it seems like this. when i abort the process, it finishes writing all the rest and all 7300 data sets are written. when not aborting the process telegraf just keeps doing its stuff with occasionally writing some datasets. when i abort it, all the data is written in less than 5s. when i dont abort it it will just keep running for ever…
Why is this and how can i prevent it? Will telegraf automatically stop after writing all the data?

Problem 2:
As mentioned at the beginning i have multiple csv files. each of them needs to be written in its own measurement. is there a possibility to do this automatically like “…\file_*.csv” or do i have to change it by hand for each file?

GitHub link where all the screenshots, an example csv and the config files are stored: https://github.com/uweking/influx_telegraf_conf

Thx for your efforts in advance and hopefully we can solve the problem together :slight_smile:

Greetings Uwe

2

About problem 1, telegraf will keep running and do periodical checks until stopped. If you need one time process of telegraf, then use the --once command argument. If you use the file plugin with continuous running, it will read the complete file each interval, you might want to switch to the tail plugin instead or use the once option.

Also, the buffers need to be increased, as the logs show that it is dropping metrics due to buffer overflow.

1 Like
3

Thx alot, im using --once now :slight_smile:

Solution for problem 2 is as i thought. “…\file_*.csv” the star will do its job :slight_smile: