Relevant telegraf.conf:
[[inputs.kafka_consumer]]
brokers = ["${IIT_BROKER_ONE}","${IIT_BROKER_TWO}"]
topics = ["${BROKER_TOPIC}"]
offset = "newest"
max_message_len = 1000000
max_undelivered_messages = 1000
## Data format to consume.
data_format = "json"
json_time_key = "beginTime"
json_time_format = "unix"
tag_keys = [
"A",
"B"
]
json_string_fields = ["counters_*"]
name_override = "${Measurement}"
[inputs.kafka_consumer.tags]
setup = "${SETUP_TAG_ONE}"
[[inputs.kafka_consumer]]
brokers = ["${SVT_BROKER_ONE}","${SVT_BROKER_TWO}"]
topics = ["Onecell_PM_Data_Stream"]
offset = "newest"
max_message_len = 1000000
max_undelivered_messages = 1000
## Data format to consume.
data_format = "json"
json_time_key = "beginTime"
json_time_format = "unix"
tag_keys = [
"A",
"B"
]
json_string_fields = ["counters_*"]
name_override = "${Measurement}"
[inputs.kafka_consumer.tags]
setup = "${SETUP_TAG_TWO}"
I am using above telegraf configuration. So the data is written properly into influxdb throughout the day; But in night time data was not writing into influxdb. When i checked logs i found this -
Abandoned subscription to TOPIC_name because consuming was taking soo long.
So after this message i don’t see any data, but in morning when i restart the docker container. Data starts coming. Also i tried with offset = “oldest” in order to backfill the missing data that is also not working. In kafkdrop when i check i see offset lag for both sets of kafka brokers. Also i am using all default configuration in agent part.
My questions -
What is solution for this problem?
Is taking offset = “oldest” everytime will allow me to backfill past data, whenever this problem comes.
Telegraf version - 1.20