Meta data information additon

Telegraf
1

Hi, I am having getting live data from Kafka which I am sending to InfluxDB using telegraf. My current requirement now, is to add some meta-data information based on some key tags. The information is available to me in key value format. For eg -

1032 - [SetupA,abc,xyz]
1033 - [SetupB,def,uvw]

In the data I am getting the keys in form of tag, say ID. So my requirement is to add 3 new tags based on ID’s values. I tried using starlark processor for this, however not able to achieve this as it does not support reading from files. Can anybody suggest a good solution for this.

Thanks

2

reading from files

Right, if you need this level of logic in terms of modifying data coming into InfluxDB through the use of an external file, then I suggest using an external script with Telegraf + the exec plugin or not using Telegraf and moving to use the Client Libraries.

3 
Telegraf + the exec

So, currently I am using Kafka plugin to fetch data, then using starlark processor for some pre processing and then sending it to influxdb. Can you help me, in introducing the exec plugin in this flow. The flow I am thinking is like this:

Kafka input plugin -> exec plugin -> starlark -> output

However the issue is basically combining the metrics from both input plugins. Can it be done?

4

You can use the execd processor, I left the d off in my comment above.

Your flow would be:

  1. use kafka input as you are now
  2. create execd processor plugin that calls external pugin to read these files you need and do the manipulation you need
  3. call the starlark processor (or fold this into your execd plugin script as well
  4. output