We are testing using the TICK-Stack in my Team.
I have run into the following Issue:
We have a Software which is generating lots of Data-points and sending it to Kafka.
I have tried Importing this JSON-Encoded Stream directly, it needs reformatting though.
So I have a script which consumes the JSON Topic, rewrites the Data and sends it back into another Topic as Influx Line Protocol.
The Issue I’m facing is: How do I choose which Database the Telegraf Kafka Consumer uses for Importing the Influx Line Protocol Lines it receives over Kafka?
Currently it’s just always writing to the DB “kafka_consumer”.
Awesome would be if I could choose the Database based on the Topic.
@RobertKaelin You can accomplish that only by running multiple instances of telegraf. Each instance can be set to output to a different database and read off of an individual topic.
@jackzampolin Thank you, could you please tell me the config-key I need to use to accomplish that the kafka_consumer Plugin doesn’t write to its default DB?
I have not found anything yet to specifically configure this, it always writes to the “kafka_consumer” Database.
@jackzampolin Thank you for your messages! I understand that we need to spin up another instance of telegraf to have another kafka consumer (I tried having two kafka consumers in my telegraf.conf file and that did not work …). However, you also say that we need to write to a different db. Can we not write to the same db but with a different measurement name, something like kafka_consumer1, kafka_consumer2?
