HDFS to influxdb via telegraf

Hi, I’m looking for a way to inject data (tsv files) from hdfs to influxdb via telegraf. I didn’t find a telegraf plugin for hdfs, can you suggest me a way to make this possible ?

Hi @ddebeul ,

is this what you are looking for ?

https://docs.wavefront.com/hadoop-hdfs.html#step-3-configure-telegraf-jolokia-input-plugin

I found the link in

Hi MarcV,

No, as mentionned jolokia is an agent that collect metrics from hdfs/hadoop cluster. In my case, every hours, I recieve from devices some TS information in a file(s) (tsv format), pushed on hdfs, I just want to know if there is a way with telegraf to push these files in influxdb.

If I understand correctly you want to ingest a tab separated file using telegraf …
You can do that by specifying the csv_delimiter “\t” in the [inputs.file] …

$ cat tsv
5       6       7
8       9       10

$ cat tele1.conf
[agent]
  omit_hostname = true
[[inputs.file]]
  files = ["tsv"]
  data_format = "csv"
  csv_delimiter = "\t"
  csv_column_names=["field1","field2","field3"]

$ telegraf --config tele1.conf --test
2019-04-16T09:17:09Z I! Starting Telegraf 1.10.1
> file field1=5i,field2=6i,field3=7i 1555406230000000000
> file field1=8i,field2=9i,field3=10i 1555406230000000000

If I misunderstood , these are the telegraf input formats : there is no hdfs …

Thanks MarcV, the example that you showed is correct but my concern is more that these files are on hdfs, probably there are no telegraf plugins for hdfs. What I can do is create a script in bash or python that will use the hdfs commands and put these scripts in the commands array as showed in the figure. Can you confirm this ?

I think that is a good solution …