Hey fellows!
I am having a hard time understanding what is going on inside of Telegraf.
I am using Telegraf 1.35.3 with Influx 2.7.11.
I have multiple Sensors, that send out messages in jsonv2 format which come in like this:
{
"msg_id" : 3068,
"device_info" : {
"device_id" : "sensorid",
"mac" : "XXXXXXXXXXXX"
},
"data" : {
"timestamp" : "2025-08-12T16:03:28+00:00",
"voltage" : 2300,
"current" : 47,
"power" : 7,
"frequency" : 5006,
"power_factor" : 6
}
}
{
"msg_id" : 3070,
"device_info" : {
"device_id" : "sensorid",
"mac" : "XXXXXXXXXXXX"
},
"data" : {
"timestamp" : "2025-08-12T16:03:00+00:00",
"num" : 30,
"energy" : [ 20, 38, 33, 38, 43, 34, 35, 38, 28, 38, 43, 32, 32, 38, 31, 39, 43, 32, 40, 30, 42, 44, 41, 34, 39, 42, 38, 33, 37, 37 ]
}
}
They send out other/more messages as well, but these two are the most important ones and the handling and format is the same with other messages. (I am aware, that Telegraf cant handle the array that way, but thats a different issue)
My Telegraf Config looks like this (in total I have 13 messages defined):
##(3068)
[[inputs.mqtt_consumer.json_v2]]
name_override = "package_data_3068"
[[inputs.mqtt_consumer.json_v2.tag]]
path = "msg_id"
data_type = "int"
[[inputs.mqtt_consumer.json_v2.tag]]
rename = "device_id"
path = "device_info.device_id"
[[inputs.mqtt_consumer.json_v2.tag]]
rename = "mac"
path = "device_info.mac"
[[inputs.mqtt_consumer.json_v2.field]]
rename = "voltage"
path = "data.voltage"
data_type = "int"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "current"
path = "data.current"
data_type = "int"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "power"
path = "data.power"
data_type = "int"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "frequency"
path = "data.frequency"
data_type = "int"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "power_factor"
path = "data.power_factor"
data_type = "int"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "time_plug"
path = "data.timestamp"
data_type = "string"
##(3070)
[[inputs.mqtt_consumer.json_v2]]
name_override = "package_data_3070"
[[inputs.mqtt_consumer.json_v2.tag]]
path = "msg_id"
data_type = "int"
[[inputs.mqtt_consumer.json_v2.tag]]
rename = "device_id"
path = "device_info.device_id"
[[inputs.mqtt_consumer.json_v2.tag]]
rename = "mac"
path = "device_info.mac"
[[inputs.mqtt_consumer.json_v2.field]]
rename = "num"
path = "data.num"
data_type = "int"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "energy_list_30d"
path = "data.energy"
data_type = "string"
optional = true
[[inputs.mqtt_consumer.json_v2.field]]
rename = "time_plug"
path = "data.timestamp"
data_type = "string"
I have a script, that puts out the raw message handling of Telegraf, which is this:
import sys
def __main__():
with open("/etc/telegraf/telegraf_raw.data", mode="a", encoding="utf-8") as fifo:
for line in sys.stdin:
fifo.write(f"{line.rstrip()}\n")
print(line.rstrip(), flush=True, file=sys.stdout)
if __name__ == "__main__":
__main__()
This is the output, how Telegraf handles the messages:
3068
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring voltage=2300,current=47,power=7,frequency=5006,power_factor=6,time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3068,plugs=Topicstring time_plug="2025-08-12T16:03:28+00:00" 1755014609359087560
3070
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=20,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=38,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=33,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=38,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=43,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=34,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=35,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=38,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=28,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=38,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring
num=30,energy_list_30d=43,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=32,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=32,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=38,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=31,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=39,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=43,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=32,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=40,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=30,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=42,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=44,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=41,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=34,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=39,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=42,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=38,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=33,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring
num=30,energy_list_30d=37,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring num=30,energy_list_30d=37,time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
mqtt_consumer,device_id=sensorid,host=Fred,mac=XXXXXXXXXXXX,msg_id=3070,plugs=Topicstring time_plug="2025-08-12T16:03:00+00:00" 1755014581517836310
My questions are:
Why does telegraf put out the “time_plug” up to 12 times?(Not consistent, sometimes 6 lines, sometimes 10)
If a message matches multiple message-patterns, will it be parsed multiple times?
How does the parsing work in particular and is there a way to match messages and parsing structure? e.g.:
##(3070)
[[inputs.mqtt_consumer.json_v2]]
name_override = "package_data_3070"
match_path = msg_id
match_name = "3070"
Something like that?
