Persisting Problem - ["outputs.kafka"] did not complete within its flush interval

Hello Team,
Good Day!

In the current setup of using telegraf and Kafka with docker container.
There is a consisting and a persisting warning -
[“outputs.kafka”] did not complete within its flush interval.

docker setup - 4.7.1 (77678)
Telegraf version - 1.23.4
Kafka version - 6.2.2 (Also tried with 7.2.0,7.2.1 and 6.2.6) docker images

Tried configurations.

Fixed telegraf configuration values

‘‘‘
metric_buffer_limit = 20000
collection_jitter = “0s”
flush_jitter = “2s”
‘‘‘

Tried changing configurations.

interval: “1s”
metric_batch_size = 50
metric_buffer_limit = 20000
flush_interval = “2s”

interval: “1s”
metric_batch_size = 100
metric_buffer_limit = 20000
flush_interval = “2s”

interval: “1s”
metric_batch_size = 50
metric_buffer_limit = 20000
flush_interval = “3s”

interval: “1s”
metric_batch_size = 100
metric_buffer_limit = 20000
flush_interval = “3s”

interval: “1s”
metric_batch_size = 50
metric_buffer_limit = 20000
flush_interval = “10s”

interval: “1s”
metric_batch_size = 100
metric_buffer_limit = 20000
flush_interval = “10s”

interval: “1s”
metric_batch_size = 250
metric_buffer_limit = 20000
flush_interval = “10s”

interval: “1s”
metric_batch_size = 250
metric_buffer_limit = 20000
flush_interval = “15s”

interval = “10s”
metric_batch_size = 100
metric_buffer_limit = 20000
flush_interval = “60s”

Please recommend or provide feedback, What could be changed in the telegraf configuration
to help overcome the “did not complete within its flush interval” warning?

Taking reference from other related tickets for telegraf I have updated my version of telegraf and used the latest 1.23.4 version but no luck on it as well.

Thank you,
Best Regards!

I think this is something network related, can you actually reach Kafka from the container/VM?

Hello @Giovanni_Luisotto,

Yes, I can reach Kafka from the container/VM.

Thank you!

P.S - Using the telegraf service with docker, kafka.
The data events from telegraf are published without noticeable data loss.