Hi everyone!
My issue is similar to Metric_buffer_limit can't exceed 1000? but I the resolution posted there didn’t work in my case. I’m pretty new in this area and I’m out of ideas, honestly. I’d be really thankful if you gave me any tips, what could I’ve been doing wrong.
Long story short - I’m trying to configure metric_buffer_limit to 10000 on the Telegraf agent, but as I’m testing it (by disabling connection between Telegraf and Influx), the actual buffer never exceeds 20 (despite MQTT messages still incoming).
On the InfluxDB side, I don’t see any of the metrics collected during the test time, even those 20 that should’ve been delivered. I only get the newest values for each of the subscribed topics (which there are 5 in total), but no “historical” data.
I’m running Telegraf 1.32.2 on Raspbian GNU/Linux 11 connected to InfluxDB v2.7.10 on Ubuntu 22.04.3.
Here’s my config file:
[agent]
interval = "5s"
round_interval = true
flush_interval = "5s"
flush_jitter = "0s"
debug = true
quiet = false
logfile = ""
hostname = ""
metric_batch_size = 1000
metric_buffer_limit = 10000
#####################################################################
# INPUT #
#####################################################################
[[inputs.mqtt_consumer]]
servers = ["tcp://localhost:1883"]
topics = [
"#"
]
qos = 1
# I've added this after consulting mentioned topic, but nothing changed
max_undelivered_messages = 100000
connection_timeout = "30s"
client_id = "telegraf_mqtt_consumer"
data_format = "value"
data_type = "float"
persistent_session = true
topic_tag = "topic"
[[inputs.mqtt_consumer.topic_parsing]]
topic = "home/+/+"
measurement = "_/_/pozycja"
tags = "location/device/_"
####################################################################
# OUTPUT #
####################################################################
[[outputs.influxdb_v2]]
urls = ["http://10.0.0.1:8086"]
token = "<TOKEN>"
organization = "dzialka"
bucket = "dzialka_monitoring_new"
timeout = "10s"
My logs for telegraf:
Nov 16 22:35:08 raspberrypi systemd[1]: Starting Telegraf...
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Loading config: /etc/telegraf/telegraf.conf
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Starting Telegraf 1.32.2 brought to you by InfluxData the makers of InfluxDB
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Available plugins: 235 inputs, 9 aggregators, 32 processors, 26 parsers, 62 outputs, 6 secret-stores
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Loaded inputs: mqtt_consumer
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Loaded aggregators:
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Loaded processors:
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Loaded secretstores:
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Loaded outputs: influxdb_v2
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! Tags enabled: host=raspberrypi
Nov 16 22:35:10 raspberrypi systemd[1]: Started Telegraf.
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z I! [agent] Config: Interval:5s, Quiet:false, Hostname:"raspberrypi", Flush Interval:5s
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z D! [agent] Initializing plugins
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z D! [agent] Connecting outputs
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z D! [agent] Attempting connection to [outputs.influxdb_v2]
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z D! [agent] Successfully connected to outputs.influxdb_v2
Nov 16 22:35:10 raspberrypi telegraf[1945]: 2024-11-16T21:35:10Z D! [agent] Starting service inputs
Nov 16 22:35:11 raspberrypi telegraf[1945]: 2024-11-16T21:35:11Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
Nov 16 22:35:11 raspberrypi telegraf[1945]: 2024-11-16T21:35:11Z D! [inputs.mqtt_consumer] Session found [tcp://localhost:1883]
Nov 16 22:35:16 raspberrypi telegraf[1945]: 2024-11-16T21:35:16Z D! [outputs.influxdb_v2] Wrote batch of 6 metrics in 78.513069ms
Nov 16 22:35:16 raspberrypi telegraf[1945]: 2024-11-16T21:35:16Z D! [outputs.influxdb_v2] Buffer fullness: 1 / 10000 metrics
Nov 16 22:35:21 raspberrypi telegraf[1945]: 2024-11-16T21:35:21Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 52.808375ms
Nov 16 22:35:21 raspberrypi telegraf[1945]: 2024-11-16T21:35:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Nov 16 22:35:26 raspberrypi telegraf[1945]: 2024-11-16T21:35:26Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 38.762542ms
Nov 16 22:35:26 raspberrypi telegraf[1945]: 2024-11-16T21:35:26Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Nov 16 22:35:31 raspberrypi telegraf[1945]: 2024-11-16T21:35:31Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 55.832341ms
Nov 16 22:35:31 raspberrypi telegraf[1945]: 2024-11-16T21:35:31Z D! [outputs.influxdb_v2] Buffer fullness: 1 / 10000 metrics
Nov 16 22:35:36 raspberrypi telegraf[1945]: 2024-11-16T21:35:36Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 47.874435ms
Nov 16 22:35:36 raspberrypi telegraf[1945]: 2024-11-16T21:35:36Z D! [outputs.influxdb_v2] Buffer fullness: 1 / 10000 metrics
Nov 16 22:35:41 raspberrypi telegraf[1945]: 2024-11-16T21:35:41Z D! [outputs.influxdb_v2] Wrote batch of 6 metrics in 47.957436ms
Nov 16 22:35:41 raspberrypi telegraf[1945]: 2024-11-16T21:35:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Nov 16 22:35:46 raspberrypi telegraf[1945]: 2024-11-16T21:35:46Z D! [outputs.influxdb_v2] Wrote batch of 8 metrics in 43.179492ms
Nov 16 22:35:46 raspberrypi telegraf[1945]: 2024-11-16T21:35:46Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Nov 16 22:35:51 raspberrypi telegraf[1945]: 2024-11-16T21:35:51Z D! [outputs.influxdb_v2] Wrote batch of 6 metrics in 41.540512ms
Nov 16 22:35:51 raspberrypi telegraf[1945]: 2024-11-16T21:35:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
#At this moment, I've simulated loosing connection to InfluxDB
Nov 16 22:35:56 raspberrypi telegraf[1945]: 2024-11-16T21:35:56Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:35:56 raspberrypi telegraf[1945]: 2024-11-16T21:35:56Z D! [outputs.influxdb_v2] Buffer fullness: 6 / 10000 metrics
Nov 16 22:35:56 raspberrypi telegraf[1945]: 2024-11-16T21:35:56Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:01 raspberrypi telegraf[1945]: 2024-11-16T21:36:01Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:01 raspberrypi telegraf[1945]: 2024-11-16T21:36:01Z D! [outputs.influxdb_v2] Buffer fullness: 10 / 10000 metrics
Nov 16 22:36:01 raspberrypi telegraf[1945]: 2024-11-16T21:36:01Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:06 raspberrypi telegraf[1945]: 2024-11-16T21:36:06Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:06 raspberrypi telegraf[1945]: 2024-11-16T21:36:06Z D! [outputs.influxdb_v2] Buffer fullness: 15 / 10000 metrics
Nov 16 22:36:06 raspberrypi telegraf[1945]: 2024-11-16T21:36:06Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:11 raspberrypi telegraf[1945]: 2024-11-16T21:36:11Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:11 raspberrypi telegraf[1945]: 2024-11-16T21:36:11Z D! [outputs.influxdb_v2] Buffer fullness: 19 / 10000 metrics
Nov 16 22:36:11 raspberrypi telegraf[1945]: 2024-11-16T21:36:11Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:16 raspberrypi telegraf[1945]: 2024-11-16T21:36:16Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:16 raspberrypi telegraf[1945]: 2024-11-16T21:36:16Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:16 raspberrypi telegraf[1945]: 2024-11-16T21:36:16Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:21 raspberrypi telegraf[1945]: 2024-11-16T21:36:21Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:21 raspberrypi telegraf[1945]: 2024-11-16T21:36:21Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:21 raspberrypi telegraf[1945]: 2024-11-16T21:36:21Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:26 raspberrypi telegraf[1945]: 2024-11-16T21:36:26Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:26 raspberrypi telegraf[1945]: 2024-11-16T21:36:26Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:26 raspberrypi telegraf[1945]: 2024-11-16T21:36:26Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:32 raspberrypi telegraf[1945]: 2024-11-16T21:36:32Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:32 raspberrypi telegraf[1945]: 2024-11-16T21:36:32Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:32 raspberrypi telegraf[1945]: 2024-11-16T21:36:32Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:36 raspberrypi telegraf[1945]: 2024-11-16T21:36:36Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:36 raspberrypi telegraf[1945]: 2024-11-16T21:36:36Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:36 raspberrypi telegraf[1945]: 2024-11-16T21:36:36Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:41 raspberrypi telegraf[1945]: 2024-11-16T21:36:41Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:41 raspberrypi telegraf[1945]: 2024-11-16T21:36:41Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:41 raspberrypi telegraf[1945]: 2024-11-16T21:36:41Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:46 raspberrypi telegraf[1945]: 2024-11-16T21:36:46Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:46 raspberrypi telegraf[1945]: 2024-11-16T21:36:46Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:46 raspberrypi telegraf[1945]: 2024-11-16T21:36:46Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
Nov 16 22:36:51 raspberrypi telegraf[1945]: 2024-11-16T21:36:51Z E! [outputs.influxdb_v2] When writing to [http://10.0.0.1:8086/api/v2/write]: Post "http://10.0.0.1:8086/api/v2/write?bucket=dzialka_monitoring_new&org=dzialka": dial tcp 10.0.0.1:8086: connect: connection refused
Nov 16 22:36:51 raspberrypi telegraf[1945]: 2024-11-16T21:36:51Z D! [outputs.influxdb_v2] Buffer fullness: 20 / 10000 metrics
Nov 16 22:36:51 raspberrypi telegraf[1945]: 2024-11-16T21:36:51Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
#Resumed normal network connectivity
Nov 16 22:36:56 raspberrypi telegraf[1945]: 2024-11-16T21:36:56Z D! [outputs.influxdb_v2] Wrote batch of 20 metrics in 73.432153ms
Nov 16 22:36:56 raspberrypi telegraf[1945]: 2024-11-16T21:36:56Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Nov 16 22:37:01 raspberrypi telegraf[1945]: 2024-11-16T21:37:01Z D! [outputs.influxdb_v2] Wrote batch of 41 metrics in 58.559326ms
Nov 16 22:37:01 raspberrypi telegraf[1945]: 2024-11-16T21:37:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Nov 16 22:37:06 raspberrypi telegraf[1945]: 2024-11-16T21:37:06Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 34.574602ms
Nov 16 22:37:06 raspberrypi telegraf[1945]: 2024-11-16T21:37:06Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
Thank you for your help
