I try to use flux engine in influxdb2.0. The dataflow in my case is kafka → telegraf → influxdb2.0 → flux second process and store in influxdb and flux sink to kafka.
But I found flux engine can not sink to kafka now(The
old capacitor engine can do this).
I don’t want use kapacitor anymore and I think telegraf maybe another method for this purpose. So the question is :
Can I use telegraf for the dataflow as follow?
Kafka → telegraf(1) → influxdb2.0 → flux second process and store in influxdb → telegraf(2) → kafka
telegraf(1) convert data in kafka topic to influxdb’s table.
telegraf(2) convert data in influxdb’s table to kafka another topic.
Hi @tfnick ,
So we have recently added the experimental mqtt.to() function in Flux. We plan to add more IoT based protocols in the future. I will make sure to flag Kafka with product management. In the meantime what you could use to achieve your ends is http.post(). You could then use the http_v2 input plugin via Telegraf. If this sounds good we can work on an example together.
@Jay_Clifford
Thanks u very much.
In addition, I found kafka.to() method in flux’s source code.
I try to use this method in influxdb2.0’s task. At present , it is running well.
But the method kafka.to() is not available in the flux document, I don’t know why.
@tfnick ,
Let me check with our product management team. Sometimes we preload experimental packages for testing before their official release. Looks like you got lucky and found the crystal skull before the map was made .
.
