Kafka consumer topic name with regular expression

Hi all. I am using kafka consumer input plugin, so my requirement is - i wanted to have wildcard for topicname. Since topic name are coming dynamically with same prefix and different suffix. Eg - “Data_stream*” for “Data_stream_1” and “Data_stream_2”. Is there anyway to do that or any other through which it can be achieved?


Have you tried wildcard in the topic? Maybe it already works?

topics = ["Data_stream*"]

No its not working. Any other way to do that?

I suspect that won’t work that way. I am not familiar with Kafka, but if the broker does not support wildcards, then it looks rather bad? The Telegraf plugin only polls the broker for one or more specific topics.

After a very very short research, I think there should also be a Kafka REST API endpoint to query for the existing topics of a broker. But this is not supported by the Telegraf plugin.