Parsing mqtt data in telegraf

Telegraf
1

Hi

I’ve setup mosquitto, influxdb2, telegraf. mqtt data is published by tasmota devices as well as shelly devices.

I already fixed and understand some things, but I need some help understand some other things :slight_smile:

I specified a a topic in my telegraf.conf. Only these values are read. But when I explore Data in the influxdb webinterface, I still see some (not all) other topics which are present in the mqtt broker, but without fields or measurements. I dont know if I have a wrong understanding of mqtt, but I thought, only the explicitly subscriped topics should show up?

I have the following json published to mosquitto:

Summary

{"Time":"2022-08-05T10:39:43","ENERGY":{"TotalStartTime":"2020-11-02T11:03:53","Total":54.687,"Yesterday":0.000,"Today":0.000,"Period": 0,"Power": 0,"ApparentPower": 0,"ReactivePower": 0,"Factor":0.00,"Voltage":221,"Current":0.000}}

The first “Time” does not show up as measurements, only the data in the ENERGY array.

In this array Telegraf did complain about the TotalStartTime with

Error in plugin: metric parse error: expected tag at 1:79:

I added

json_string_fields = [ "TotalStartTime" ]

to my telegraf.conf to cure this error.

My question is now: how can I get telegraf to read the first “Time”. It seems that it is completely ignored, because I assume otherwise telegraf would also have complained about the format? I suspect I have to do some conversion, as this is interpreted as a string?

What is happening with TotalStartTime after I added is to json_string_fields? Is it compeltely ignored now (which is fine, I just want to understand)?

My relevant telegraf.conf:

Summary

`[[inputs.mqtt_consumer]]

servers = [“tcp://mqqtbroker:1883”]

topics = [

“tele/rooms/+/SENSOR/#”

]

data_format = “json”

json_string_fields = [ “TotalStartTime” ]

client_id = “telegrafmqtt”

I want to read the temperature. But If add the the topic to telegraf.conf I only get a lot of the following errors about almost all data. I also dont see data in mqtt.out (which I added to debug it).

Error in plugin: metric parse error: expected tag at 1:4:

Also no data is written to influxdb. I suspect because this is no valid JSON structure, that everything is interpreted as a string?

How can I read the values?

Thank you very much in advance, any help is much appreciated!

2

Hello @alllak,
I’m not sure I’ve asked the telegraf team to help you out. I appreciate your patience.

3

hi @alllak,

I am sorry to hear you are having some issues.

I would recommend looking into the json_v2 parser or the xpath parser. Both are able to get the Time and set it as the timestamp if that is what you want.

The section of this blog labeled “Payload Parsing” should be useful!

Please let me know if you need more help :slight_smile:

4

Hi,
thank you for your answers! I managed to get it working.
The shelly mqtt output could be parsed with

  data_format="value"
  data_type="float"

Now I just have to figure out what to to with the string values which also appear (e. g. termperature_status in the screenshot above). But I think I have to just use a specific topic and set it to

  data_format="value"
  data_type="string"

I will look into json_v2.

Thanks again!

5

Hi,

setting specific topics to parse the string values didn’t work out. I could parse the string topics but when I added the root topic to the value/float input, I still get some erros:
"strconv.ParseFloat: parsing "stop": invalid syntax"

Is it possible to somehow to somehow set the data_type to string for some subtopics?

level1/# : most sub values are float with data_format="value"
level1/level2/level3 : this is a string with data_format="value"

Is it possible to subscribe to level1/# and still parse level1/level2/level3 correctly?