Telegraf installed with helm chart and configured with kafka consumer input with below configurations
However the pod failed to start with kafka authentication errors as below. It would seems Telegraf does not respect the SASL_mechanism configured “PLAIN”, instead if uses “sarama” which may causing the handshake failure.
Relevant telegraf.conf:
- kafka_consumer:
brokers:
- “(kakfa-broker).azure.confluent.cloud:9092”
topics:
- “telegraf”
version: “2.0.0”
sasl_username: “(username)”
sasl_password: “(password)”
sasl_mechanism: “PLAIN”
consumer_group: “telegraf”
offset: “newest”
data_format: “influx”
[[inputs.kafka_consumer]]
brokers = [“(kafka-server).confluent.cloud:9092”]
topics = [“telegraf”]
version = “2.0.0”
sasl_username = “(username)”
sasl_password = “(password)”
consumer_group = “test”
offset = “oldest”
max_message_len = 1000000
data_format = “influx”
insecure_skip_verify = true
Expected behavior:
Actual behavior:
2021-03-26T16:47:46Z I! Starting Telegraf 1.17.3
2021-03-26T16:47:46Z I! Using config file: /etc/telegraf/telegraf.conf
2021-03-26T16:47:46Z I! Loaded inputs: internal kafka_consumer statsd
2021-03-26T16:47:46Z I! Loaded aggregators:
2021-03-26T16:47:46Z I! Loaded processors: enum
2021-03-26T16:47:46Z I! Loaded outputs: influxdb_v2
2021-03-26T16:47:46Z I! Tags enabled: host=telegraf-polling-service
2021-03-26T16:47:46Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:“telegraf-polling-service”, Flush Interval:10s
2021-03-26T16:47:46Z D! [agent] Initializing plugins
2021-03-26T16:47:46Z D! [agent] Connecting outputs
2021-03-26T16:47:46Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2021-03-26T16:47:46Z D! [agent] Successfully connected to outputs.influxdb_v2
2021-03-26T16:47:46Z D! [agent] Starting service inputs
2021-03-26T16:47:46Z I! [inputs.statsd] UDP listening on “[::]:8125”
2021-03-26T16:47:46Z I! [inputs.statsd] Started the statsd service on “:8125”
2021-03-26T16:47:46Z D! [sarama] Initializing new client
2021-03-26T16:47:46Z D! [sarama] client/metadata fetching metadata for all topics from broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:46Z D! [sarama] Failed to read SASL handshake header : unexpected EOF
2021-03-26T16:47:46Z D! [sarama] Closed connection to broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:46Z D! [sarama] client/metadata got error from broker -1 while fetching metadata: unexpected EOF
2021-03-26T16:47:46Z D! [sarama] client/metadata no available broker to send metadata request to
2021-03-26T16:47:46Z D! [sarama] client/brokers resurrecting 1 dead seed brokers
2021-03-26T16:47:46Z D! [sarama] client/metadata retrying after 250ms… (3 attempts remaining)
2021-03-26T16:47:46Z D! [sarama] client/metadata fetching metadata for all topics from broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:46Z D! [sarama] Failed to read SASL handshake header : unexpected EOF
2021-03-26T16:47:46Z D! [sarama] Closed connection to broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:46Z D! [sarama] client/metadata got error from broker -1 while fetching metadata: unexpected EOF
2021-03-26T16:47:46Z D! [sarama] client/metadata no available broker to send metadata request to
2021-03-26T16:47:46Z D! [sarama] client/brokers resurrecting 1 dead seed brokers
2021-03-26T16:47:46Z D! [sarama] client/metadata retrying after 250ms… (2 attempts remaining)
2021-03-26T16:47:46Z D! [sarama] client/metadata fetching metadata for all topics from broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:46Z D! [sarama] Failed to read SASL handshake header : unexpected EOF
2021-03-26T16:47:46Z D! [sarama] Closed connection to broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:46Z D! [sarama] client/metadata got error from broker -1 while fetching metadata: unexpected EOF
2021-03-26T16:47:46Z D! [sarama] client/metadata no available broker to send metadata request to
2021-03-26T16:47:46Z D! [sarama] client/brokers resurrecting 1 dead seed brokers
2021-03-26T16:47:46Z D! [sarama] client/metadata retrying after 250ms… (1 attempts remaining)
2021-03-26T16:47:47Z D! [sarama] client/metadata fetching metadata for all topics from broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:47Z D! [sarama] Failed to read SASL handshake header : unexpected EOF
2021-03-26T16:47:47Z D! [sarama] Closed connection to broker pkc-epwny.eastus.azure.confluent.cloud:9092
2021-03-26T16:47:47Z D! [sarama] client/metadata got error from broker -1 while fetching metadata: unexpected EOF
2021-03-26T16:47:47Z D! [sarama] client/metadata no available broker to send metadata request to
2021-03-26T16:47:47Z D! [sarama] client/brokers resurrecting 1 dead seed brokers
2021-03-26T16:47:47Z D! [sarama] Closing Client
2021-03-26T16:47:47Z I! [inputs.statsd] Stopping the statsd service
2021-03-26T16:47:47Z I! [inputs.statsd] Stopped listener service on “:8125”
2021-03-26T16:47:47Z E! [telegraf] Error running agent: starting input inputs.kafka_consumer: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
