Telegraf Kafka Consumer: Choose Database

We are testing using the TICK-Stack in my Team.
I have run into the following Issue:
We have a Software which is generating lots of Data-points and sending it to Kafka.

I have tried Importing this JSON-Encoded Stream directly, it needs reformatting though.

So I have a script which consumes the JSON Topic, rewrites the Data and sends it back into another Topic as Influx Line Protocol.

The Issue I’m facing is: How do I choose which Database the Telegraf Kafka Consumer uses for Importing the Influx Line Protocol Lines it receives over Kafka?

Currently it’s just always writing to the DB “kafka_consumer”.

Awesome would be if I could choose the Database based on the Topic.

Telegraf is configured like described here: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/kafka_consumer

@RobertKaelin You can accomplish that only by running multiple instances of telegraf. Each instance can be set to output to a different database and read off of an individual topic.

@jackzampolin Thank you, could you please tell me the config-key I need to use to accomplish that the kafka_consumer Plugin doesn’t write to its default DB?

I have not found anything yet to specifically configure this, it always writes to the “kafka_consumer” Database.

@RobertKaelin in your config file it will be in the outputs section [[outputs.influxdb]] database = "mydb"

2 Likes