I’ve been toying around with mqtt input and lookup processor for some time. The below shown config works fine, but only for some time (around 7 minutes, equal to around 1.000 ingested and processed mqtt topics). Then the mqtt input stops - or more precisely, no error message and no output from mqtt anymore (other inputs seems to continue, however, so telegraf itself seems not to be affected).
When running without the lookup processor, all is fine (tested over more than 2 hrs).
Restarting the telegraf process (v 1.26.2 on docker) … all fine.
telegraf.conf
[global_tags]
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = "0s"
debug = true
hostname = ""
omit_hostname = true
[[outputs.file]]
files = ["stdout"]
#####################
# SIMPLE mqtt message
# i.e.
# mtr/hzg/temp_VL 25.8
#####################
[[inputs.mqtt_consumer]]
alias = "mqtt_consumer_value_1"
client_id = "telegraf_value_1"
servers = ["tcp://127.0.0.1:1883"]
username = "uuuuuuuu"
password = "pppppppp"
name_override = "logdata_test" # measurement
data_format = "value"
data_type = "string"
topic_tag = "addr" # tag, instead of "topic"
topics = [
"knx/2/2/2",
"knx/2/0/3",
"knx/4/4/10",
"knx/4/4/11",
"knx/4/4/12",
"knx/4/4/13",
"knx/4/4/30",
"knx/4/0/30",
"knx/4/0/31",
"knx/4/0/33",
"knx/4/0/34",
"knx/4/0/35",
"knx/4/0/36",
"knx/4/5/1",
"knx/4/5/2",
"knx/4/5/3",
"knx/4/5/4",
"knx/4/5/5",
"knx/4/5/6",
"mtr/wasser",
"mtr/hzg/temp_VL",
"mtr/hzg/temp_RL",
"mtr/hzg/temp_RLWW",
"mtr/hzg/temp_VLWW",
"mtr/hzg/temp_WW",
"mtr/hzg/temp_abgas",
"shellies/hzg/relay/0/power",
"shellies/hzg/relay/0/energy",
]
[[processors.lookup]]
files = [ "/etc/telegraf/ufn.csv" ] # Path as mapped into docker
format = "csv_key_values"
key = '{{.Tag "addr"}}' # Find "key=addr" in ufn.csv and add "ufn"
ufn.csv (processors.lookup file)
ignored, ufn
knx/2/2/2, BadOG HzStrahler (min)
knx/4/4/30, BadOG Helligkeit (lm)
knx/4/4/10, WetterStat Helligkeit (lm)
knx/4/4/11, Wetterstat Temp (°C)
knx/4/4/12, Wetterstat Wind (m/s)
knx/4/4/13, Wetterstat Regen (ja nein)
knx/4/0/30, Gas (l)
knx/2/0/3, Wasser warm Zirk (min)
knx/4/0/33, Wasser warm (l)
knx/4/0/34, Wasser Haus (l)
knx/4/0/35, Wasser Garten (l)
knx/4/0/36, Wasser Toiletten (l)
mtr/wasser, Wasser Hauptzaehler (l)
mtr/hzg/temp_VL, Hzg Temp VorLauf (°C)
mtr/hzg/temp_RL, Hzg Temp RueckLauf (°C)
mtr/hzg/temp_VLWW, Hzg Temp VorLauf WarmWasser (°C)
mtr/hzg/temp_RLWW, Hzg Temp RueckLauf WarmWasser (°C)
mtr/hzg/temp_WW, Hzg Temp Auslauf WarmWasser (°C)
mtr/hzg/temp_abgas, Hzg Temp Rauchgas (°C)
shellies/hzg/relay/0/power, Hzg Strom Power (W)
shellies/hzg/relay/0/energy, Hzg Strom Energy (Wmin)
knx/4/5/1, PV Autarkie (%)
knx/4/5/2, PV Eigenverbrauch (%)
knx/4/5/3, PV Batterie SOC (%)
knx/4/5/4, PV Leistung Solar (Wh)
knx/4/5/5, PV Leistung Batterie (Wh)
knx/4/5/6, PV Verbrauch Haus (Wh)
knx/4/5/7, PV Bezug Netz (Wh)
mtr/pwr/evu/SENSOR, Strom (Wh)
