# \#kafka **URL:** https://community.influxdata.com/tag/kafka/80.md [Latest](https://community.influxdata.com/latest.md) · [Categories](https://community.influxdata.com/categories.md) · [Tags](https://community.influxdata.com/tags.md) --- ## [Kapacitor .id() field limitation in alert node partitioning issues with Kafka](https://community.influxdata.com/t/kapacitor-id-field-limitation-in-alert-node-partitioning-issues-with-kafka/57561)
**Author:** [@Husnain](https://community.influxdata.com/u/Husnain)\ **Replies:** 0\ **Last updated:** [July 7, 2025, 5:43pm UTC](https://community.influxdata.com/t/kapacitor-id-field-limitation-in-alert-node-partitioning-issues-with-kafka/57561 "2025-07-07T17:43:53Z")
:pushpin: Environment Kapacitor Version: 1.7.7 Deployment: Docker container Output Handler: Kafka (.topic() used to send alerts) Alert Streams: Multiple alert branches (flat\_threshold, analytic, etc.) Kafka Setup: 4… --- ## [Telegraf Dynamic Tag Parsing - Batch Style](https://community.influxdata.com/t/telegraf-dynamic-tag-parsing-batch-style/57186)
**Author:** [@Michael\_Rarela](https://community.influxdata.com/u/Michael_Rarela)\ **Replies:** 2\ **Last updated:** [May 10, 2025, 12:39am UTC](https://community.influxdata.com/t/telegraf-dynamic-tag-parsing-batch-style/57186 "2025-05-10T00:39:05Z")
Via Kafka, I have this incoming JSON payload, that looks very similar to this: { "measurement": "weather", "batches": \[ { "time": 1735854321, "tags": { "tag1": "tagValueA", "tag2": "t… --- ## [Disk Based Buffering](https://community.influxdata.com/t/disk-based-buffering/56379)
**Author:** [@bmagistro](https://community.influxdata.com/u/bmagistro)\ **Replies:** 1\ **Last updated:** [February 19, 2025, 10:41pm UTC](https://community.influxdata.com/t/disk-based-buffering/56379 "2025-02-19T22:41:20Z")
Starting a thread here as well, we’ve opened Disk based buffer causes metric collection/publishing issues · Issue #16500 · influxdata/telegraf · GitHub to try and identify why using a disk buffer strategy with a kafka ou… --- ## [Best deployment strategy for production, 1x Telegraf per NE or per NE hardware type?](https://community.influxdata.com/t/best-deployment-strategy-for-production-1x-telegraf-per-ne-or-per-ne-hardware-type/39183)
**Author:** [@pelas](https://community.influxdata.com/u/pelas)\ **Replies:** 5\ **Last updated:** [December 18, 2024, 1:14pm UTC](https://community.influxdata.com/t/best-deployment-strategy-for-production-1x-telegraf-per-ne-or-per-ne-hardware-type/39183 "2024-12-18T13:14:03Z")
We are working on deploying Telegraf as SNMP & gNMI collectors, but we’ve seen that running 1x Telegraf process per Network Element (NE) uses a lot of threads, on average 11 threads per NE. The reason why we did this was… --- ## [Output to Kafka](https://community.influxdata.com/t/output-to-kafka/35536)
**Author:** [@grades212](https://community.influxdata.com/u/grades212)\ **Replies:** 6\ **Last updated:** [September 5, 2024, 7:22pm UTC](https://community.influxdata.com/t/output-to-kafka/35536 "2024-09-05T19:22:11Z")
Hello, we are trying to configure Telegraf to connect to Confluent Cloud using OAUTHBEARER. I see this is supported in the plugins/output/kafka readme but have been unsuccessful in this. I see these configs here but why… --- ## [Kafka\_consumer does not read timestamp from Kafka events](https://community.influxdata.com/t/kafka-consumer-does-not-read-timestamp-from-kafka-events/35442)
**Author:** [@brennerm](https://community.influxdata.com/u/brennerm)\ **Replies:** 2\ **Last updated:** [August 27, 2024, 2:07pm UTC](https://community.influxdata.com/t/kafka-consumer-does-not-read-timestamp-from-kafka-events/35442 "2024-08-27T14:07:48Z")
Hey folks, my goal is to pull metric events from a Kafka cluster and ingest them into an InfluxDB bucket. The timestamp of each metric point should be taken from the Kafka event and is not part of the event payload. Is … --- ## [Write to Influx via telegraf is showing fewer data points than inserted](https://community.influxdata.com/t/write-to-influx-via-telegraf-is-showing-fewer-data-points-than-inserted/35411)
**Author:** [@Mayur\_Rotti](https://community.influxdata.com/u/Mayur_Rotti)\ **Replies:** 4\ **Last updated:** [August 27, 2024, 6:35am UTC](https://community.influxdata.com/t/write-to-influx-via-telegraf-is-showing-fewer-data-points-than-inserted/35411 "2024-08-27T06:35:16Z")
im using kafka to publish 1000 messages to telegraf which in turn writes to influx. Influx measurement count shows only 529 data points in cli for query (select count(\*) from \ ) telegraf config is this : \[… --- ## [Old conf not reading the new](https://community.influxdata.com/t/old-conf-not-reading-the-new/34218)
**Author:** [@Artax1442](https://community.influxdata.com/u/Artax1442)\ **Replies:** 4\ **Last updated:** [May 14, 2024, 3:45pm UTC](https://community.influxdata.com/t/old-conf-not-reading-the-new/34218 "2024-05-14T15:45:34Z")
Hello I have a telégraf running in a Windows server 2022 I had a 1.23 versión And in my conf had a kafka output with a broker dns Server1292 but It was an error connect to dns and i need to use ip broker instead I d… --- ## [Unable to View Twin Id and Other Details from Kafka in InfluxDB](https://community.influxdata.com/t/unable-to-view-twin-id-and-other-details-from-kafka-in-influxdb/31400)
**Author:** [@karthick\_G](https://community.influxdata.com/u/karthick_G)\ **Replies:** 3\ **Last updated:** [January 23, 2024, 12:07pm UTC](https://community.influxdata.com/t/unable-to-view-twin-id-and-other-details-from-kafka-in-influxdb/31400 "2024-01-23T12:07:31Z")
I’m facing an issue while using InfluxDB to store data from IoT devices (Eclipse Ditto) through Kafka using Telegraf. The data I’m receiving is in JSON format, containing information about various attributes and features… --- ## [Map support for kafka\_consumer input plugin with Avro format](https://community.influxdata.com/t/map-support-for-kafka-consumer-input-plugin-with-avro-format/32164)
**Author:** [@straf](https://community.influxdata.com/u/straf)\ **Replies:** 2\ **Last updated:** [November 17, 2023, 10:07am UTC](https://community.influxdata.com/t/map-support-for-kafka-consumer-input-plugin-with-avro-format/32164 "2023-11-17T10:07:04Z")
it’s not clear from the documentation if maps are supported and what is the output derived from them. would it be possible to have an example? --- ## [Telegraf MSK kafka input plugin](https://community.influxdata.com/t/telegraf-msk-kafka-input-plugin/31872)
**Author:** [@Ajinkya\_patil](https://community.influxdata.com/u/Ajinkya_patil)\ **Replies:** 1\ **Last updated:** [October 20, 2023, 9:45pm UTC](https://community.influxdata.com/t/telegraf-msk-kafka-input-plugin/31872 "2023-10-20T21:45:42Z")
When I try to connect to msk cluster on aws I get an error saying This is my telegraf.conf file \[agent\] debug = true \[\[outputs.influxdb\]\] urls = \["influxdb url"\] database = "test\_data" timeout = "5s" username =… --- ## [\[telegraf\] Error running agent: starting input inputs.kafka\_consumer: create consumer: kafka: client has run out of available brokers to talk to: dial tcp: address 10.166.139.204:9092,10.166.139.205:9092,10.166.139.206:9092: too many colons in address](https://community.influxdata.com/t/telegraf-error-running-agent-starting-input-inputs-kafka-consumer-create-consumer-kafka-client-has-run-out-of-available-brokers-to-talk-to-dial-tcp-address-10-166-139-204-9092-10-166-139-205-9092-10-166-139-206-too-many-colons-in-address/30935)
**Author:** [@kevin-du](https://community.influxdata.com/u/kevin-du)\ **Replies:** 1\ **Last updated:** [August 4, 2023, 12:51am UTC](https://community.influxdata.com/t/telegraf-error-running-agent-starting-input-inputs-kafka-consumer-create-consumer-kafka-client-has-run-out-of-available-brokers-to-talk-to-dial-tcp-address-10-166-139-204-9092-10-166-139-205-9092-10-166-139-206-too-many-colons-in-address/30935 "2023-08-04T00:51:44Z")
this is my brokers,why is this so? --- ## [\[\[inputs.kafka\_consumer\]\] problem "decoding" avro-formated data](https://community.influxdata.com/t/inputs-kafka-consumer-problem-decoding-avro-formated-data/29675)
**Author:** [@dan-m-joh](https://community.influxdata.com/u/dan-m-joh)\ **Replies:** 2\ **Last updated:** [April 27, 2023, 11:58am UTC](https://community.influxdata.com/t/inputs-kafka-consumer-problem-decoding-avro-formated-data/29675 "2023-04-27T11:58:05Z")
Hello, I am trying to collect data from Kafka (from a SAN-Switch) using telegrafs \[\[inputs.kafka\_consumer\]\]. The Kafka-producer uses the avro-format when putting the data into Kafka (which I can not change). Here is t… --- ## [Kafka.to() function](https://community.influxdata.com/t/kafka-to-function/25826)
**Author:** [@aceankit99](https://community.influxdata.com/u/aceankit99)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 4:14pm UTC](https://community.influxdata.com/t/kafka-to-function/25826 "2022-07-21T16:14:48Z")
Hello , can I send query data to the azure event hub using Kafka.to using influx task? or Is there any other way available? --- ## [\[inputs.kafka\_consumer\] Error in plugin: the path id doesn't exist](https://community.influxdata.com/t/inputs-kafka-consumer-error-in-plugin-the-path-id-doesnt-exist/25398)
**Author:** [@Selim\_Skhiri](https://community.influxdata.com/u/Selim_Skhiri)\ **Replies:** 1\ **Last updated:** [June 13, 2022, 6:27pm UTC](https://community.influxdata.com/t/inputs-kafka-consumer-error-in-plugin-the-path-id-doesnt-exist/25398 "2022-06-13T18:27:53Z")
I have set up a Kafka Broker and a producer correctly, I can send and receive info from the topic I have created. I’ve serialized my JSON data correctly as well. I’m trying to set up InfluxDB as a Kafka consumer via Tele… --- ## [Telegraf Configuration Values](https://community.influxdata.com/t/telegraf-configuration-values/22631)
**Author:** [@khushipurvesh.jain](https://community.influxdata.com/u/khushipurvesh.jain)\ **Replies:** 3\ **Last updated:** [November 24, 2021, 2:29pm UTC](https://community.influxdata.com/t/telegraf-configuration-values/22631 "2021-11-24T14:29:35Z")
Hey! I need help configuring my telegraf values.I have provided my telegraf files below. Can I reduce my max\_undelivered messages in this case. From my Internal Stats, I am getting metrics gathered = 950 metrics/min. an… --- ## [Kafka consumer topic name with regular expression](https://community.influxdata.com/t/kafka-consumer-topic-name-with-regular-expression/22493)
**Author:** [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\ **Replies:** 3\ **Last updated:** [November 12, 2021, 9:30am UTC](https://community.influxdata.com/t/kafka-consumer-topic-name-with-regular-expression/22493 "2021-11-12T09:30:13Z")
Hi all. I am using kafka consumer input plugin, so my requirement is - i wanted to have wildcard for topicname. Since topic name are coming dynamically with same prefix and different suffix. Eg - “Data\_stream\*” for “Data… --- ## [How to sink to kafka in flux engine?](https://community.influxdata.com/t/how-to-sink-to-kafka-in-flux-engine/22440)
**Author:** [@tfnick](https://community.influxdata.com/u/tfnick)\ **Replies:** 4\ **Last updated:** [November 10, 2021, 10:57am UTC](https://community.influxdata.com/t/how-to-sink-to-kafka-in-flux-engine/22440 "2021-11-10T10:57:04Z")
I try to use flux engine in influxdb2.0. The dataflow in my case is kafka → telegraf → influxdb2.0 → flux second process and store in influxdb and flux sink to kafka. But I found flux engine can not sink to kafka now(Th… --- ## [Influxdb Kafka monitoring dashboard](https://community.influxdata.com/t/influxdb-kafka-monitoring-dashboard/20933)
**Author:** [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 5:56pm UTC](https://community.influxdata.com/t/influxdb-kafka-monitoring-dashboard/20933 "2021-07-30T17:56:04Z")
Hi everyone. I am using a kafka broker as an input plugin…i wanted to monitor the kafka broker, if data doesnot comes in past 15m i want to setup an alert. For monitoring i found this template - Kafka Monitoring Dashboar… --- ## [Influxdb2.0 connection with Kafka Consumer Legacy](https://community.influxdata.com/t/influxdb2-0-connection-with-kafka-consumer-legacy/20934)
**Author:** [@Haque](https://community.influxdata.com/u/Haque)\ **Replies:** 1\ **Last updated:** [July 30, 2021, 5:51pm UTC](https://community.influxdata.com/t/influxdb2-0-connection-with-kafka-consumer-legacy/20934 "2021-07-30T17:51:45Z")
I am working with Influxdb version 2.0 and wanted to connect kafka consumer leagcy plugin with it.I wanted to know that how should I configure this configuration file of kafka consumer legacy and which path to place it o… --- ## [Inputs.kafka\_consumer not working](https://community.influxdata.com/t/inputs-kafka-consumer-not-working/19648)
**Author:** [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\ **Replies:** 1\ **Last updated:** [April 30, 2021, 2:46pm UTC](https://community.influxdata.com/t/inputs-kafka-consumer-not-working/19648 "2021-04-30T14:46:39Z")
I wanted to use apache kafka as input for telegraf. I am running docker containers. Docker image is from - GitHub - wurstmeister/kafka-docker: Dockerfile for Apache Kafka This is my docker-compose file: version: ‘3’ … --- ## [Telegraf Kafka producer plugin cannot establish session with Kafka server](https://community.influxdata.com/t/telegraf-kafka-producer-plugin-cannot-establish-session-with-kafka-server/19555)
**Author:** [@Marios\_Karatisoglou](https://community.influxdata.com/u/Marios_Karatisoglou)\ **Replies:** 0\ **Last updated:** [April 23, 2021, 1:58pm UTC](https://community.influxdata.com/t/telegraf-kafka-producer-plugin-cannot-establish-session-with-kafka-server/19555 "2021-04-23T13:58:45Z")
I have an issue when trying to write data to Kafka broker with outputs.kafka plugin. The Telegraf logs are the following (Kafka server: 192.168.55.10, Telegraf kafka producer: 192.168.200.45): Apr 23 15:17:47 rancid tel… --- ## [Configure Telegraf with output Kafka](https://community.influxdata.com/t/configure-telegraf-with-output-kafka/19221)
**Author:** [@scottbusi](https://community.influxdata.com/u/scottbusi)\ **Replies:** 5\ **Last updated:** [April 5, 2021, 7:05pm UTC](https://community.influxdata.com/t/configure-telegraf-with-output-kafka/19221 "2021-04-05T19:05:42Z")
Hopefully someone can help as I am not getting how to configure this very simple configuration I want to configure a redfish input to monitor 4 URI’s with an output of kafka, can not get it configured , is there some b… --- ## [Capacitor Alerts to Kafka Broker](https://community.influxdata.com/t/capacitor-alerts-to-kafka-broker/19059)
**Author:** [@fe11x](https://community.influxdata.com/u/fe11x)\ **Replies:** 0\ **Last updated:** [March 24, 2021, 7:30am UTC](https://community.influxdata.com/t/capacitor-alerts-to-kafka-broker/19059 "2021-03-24T07:30:23Z")
Hi Guys, i have a question regarding the Kafka Handler. I have an alert pipeline and I want to send all the alerts to a Kafka topic on another server. This works fine from the technical perspective. What i have not fi… --- ## [Error from broker when executing telegraf --config /etc/telegraf/telegraf.conf](https://community.influxdata.com/t/error-from-broker-when-executing-telegraf-config-etc-telegraf-telegraf-conf/18426)
**Author:** [@scottbusi](https://community.influxdata.com/u/scottbusi)\ **Replies:** 1\ **Last updated:** [February 19, 2021, 10:49pm UTC](https://community.influxdata.com/t/error-from-broker-when-executing-telegraf-config-etc-telegraf-telegraf-conf/18426 "2021-02-19T22:49:32Z")
Error from broker when executing \[root@scottsys ~\]# telegraf --config /etc/telegraf/telegraf.conf Telegraf 1.17.1 Kafka 2.13-2.7.0 telegraf.conf \[root@scottsys ~\]# cat /etc/telegraf/telegraf.conf \[tags\] Configurati… --- ## [Telegraf -could not initialize output kafka: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)](https://community.influxdata.com/t/telegraf-could-not-initialize-output-kafka-kafka-client-has-run-out-of-available-brokers-to-talk-to-is-your-cluster-reachable/18398)
**Author:** [@scottbusi](https://community.influxdata.com/u/scottbusi)\ **Replies:** 1\ **Last updated:** [February 19, 2021, 9:41pm UTC](https://community.influxdata.com/t/telegraf-could-not-initialize-output-kafka-kafka-client-has-run-out-of-available-brokers-to-talk-to-is-your-cluster-reachable/18398 "2021-02-19T21:41:05Z")
Getting the follow error when testing telegraf \[root@scottsys ~\]# telegraf --config /etc/telegraf/telegraf.conf --test 2021-02-16T22:47:14Z I! Starting Telegraf 1.17.2 2021-02-16T22:47:14Z D! \[agent\] Initializing plug…