Parsing data from MQTT to telegraf

Telegraf
1

Hi. I’m trying to get the data from Shelly devices to Influx using the telegraf and mosquitto MQTT broker, but still without success. The problem is, that in the data, there are subtopics where are different value types(float, int, string) as can be seen on pic from MQTT Explorer. Thanks for any help.
image

here’s the telegraf.conf

servers = ["tcp://127.0.0.1:1883"]

    username = "********"
    password = "********"
    
   ## Topics that will be subscribed to.
    topics = [
	"shellies/shellyplug-s-7AEC17/#",
	]

    [[inputs.mqtt_consumer.topic_parsing]]
	topic = "shellies/shellyplug-s-7AEC17/relay/0"
	measurement = "measurement/_/_/_"
	fields = "_/_/_/value"
	[inputs.mqtt_consumer.topic_parsing.types]
	value = "string"

    data_format = "value"
    data_type = "float"

and error I get

2023-10-11T20:57:56Z I! Loading config: /etc/telegraf/telegraf.conf
2023-10-11T20:57:56Z I! Starting Telegraf 1.28.2 brought to you by InfluxData the makers of InfluxDB
2023-10-11T20:57:56Z I! Available plugins: 240 inputs, 9 aggregators, 29 processors, 24 parsers, 59 outputs, 5 secret-stores
2023-10-11T20:57:56Z I! Loaded inputs: mqtt_consumer
2023-10-11T20:57:56Z I! Loaded aggregators: 
2023-10-11T20:57:56Z I! Loaded processors: 
2023-10-11T20:57:56Z I! Loaded secretstores: 
2023-10-11T20:57:56Z W! Outputs are not used in testing mode!
2023-10-11T20:57:56Z I! Tags enabled: host=Monitoring
2023-10-11T20:57:56Z I! [inputs.mqtt_consumer] Connected [tcp://127.0.0.1:1883]
2023-10-11T20:58:03Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "74316"
2023-10-11T20:58:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "24.41"
2023-10-11T20:58:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "74316"
2023-10-11T20:58:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:3: "on"
2023-10-11T20:58:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "29.18"
2023-10-11T20:58:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "84.52"
2023-10-11T20:58:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:2: "0"
2

@lubhat you cannot define parsing per topic in Telegraf. Furthermore, you have to be careful on where you place options in TOML as in your configuration both data_format and data_type as options of the [inputs.mqtt_consumer.topic_parsing.types] section. You should use the following with data_type = "auto_float" which falls back to a string if the value at hand is not a float (see [documentation]( telegraf/plugins/parsers/value at master · influxdata/telegraf (github.com)))…

 [[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  username = "********"
  password = "********"
    
   ## Topics that will be subscribed to.
  topics = ["shellies/shellyplug-s-7AEC17/#"]

  data_format = "value"
  data_type = "auto_float"
  
  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "shellies/shellyplug-s-7AEC17/relay/+/+"
    measurement = "measurement/_/_/_/_"
    fields = "_/_/_/value/_"
    tags = "_/_/_/_/field"

    [inputs.mqtt_consumer.topic_parsing.types]
	value = "string"

  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "shellies/shellyplug-s-7AEC17/+/+"
    measurement = "measurement/_/_/_"
    tags = "_/_/_/field"
1 Like
3

Thank you, I will test it :+1:

4

@srebhan thank you for sharing :slight_smile: