Correct way to configure json_v2 with mqtt_consumer

Telegraf
, ,
1

I’ve spent some time trying to understand how to use json_v2 with my mqtt_consumer flow, without any luck. :frowning:

My json payload looks like this:

{
    "model": "Bresser-3CH",
    "id": 130,
    "channel": 1,
    "battery_ok": 1,
    "temperature_C": 12.66667,
    "humidity": 19,
    "mic": "CHECKSUM",
    "protocol": "Bresser Thermo-/Hygro-Sensor 3CH",
    "rssi": -61,
    "duration": 1753996
}

Of these, the following are tags:

  • channel
  • id
  • model

The following are the fields I’m interested in:

  • humidity
  • temperature_C
  • rssi
  • battery_ok

The rest (protocol, mic, duration) are not interesting and if possible I’d like to ignore them.

I have this setup using the json data format, and it works OK:

[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  topics = [
    "home/OpenMQTTGateway_lilygo_rtl_433_ESP/RTL_433toMQTT/#"
  ]

  topic_tag = "topic"
  qos = 0
  data_format = "json"

  username = "telegraf"
  password = "notmyrealpassword"

  tag_keys = [ "model", "id", "channel" ]

  json_string_fields = []

  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "+/+/+/+/+/+"
    tags = "site/_/_/model/channel/device_id"

I’m not even sure the topic parsing is relevant with this setup?

I had tried with something like this, but getting an error message that my JSON was invalid, which surprised me! What is wrong with below?

[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  topics = [
    # Real topic structure
    # "home/OpenMQTTGateway_lilygo_rtl_433_ESP/RTL_433toMQTT/Bresser-3CH/1/130",
    "home/OpenMQTTGateway_lilygo_rtl_433_ESP/RTL_433toMQTT/#"
  ]

  topic_tag = "topic"
  qos = 0
  data_format = "json_v2"

  username = "telegraf"
  password = "notmyrealpassword"
  
  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "+/+/+/+/+/+"
    tags = "site/_/_/model/channel/device_id"

    [[inputs.mqtt_consumer.json_v2]]
      
      [[inputs.mqtt_consumer.json_v2.field]]
      path = "humidity"
      type = "int"
        
      [[inputs.mqtt_consumer.json_v2.field]]
      path = "battery_ok"
      type = "int"
        
      [[inputs.mqtt_consumer.json_v2.field]]
      path = "temperature_C"
      type = "float"
        
      [[inputs.mqtt_consumer.json_v2.field]]
      path = "rssi"
      type = "int"
    
      [[inputs.mqtt_consumer.json_v2.tag]]
      path = "model"
    
      [[inputs.mqtt_consumer.json_v2.tag]]
      path = "id"
      type = "int"
    
      [[inputs.mqtt_consumer.json_v2.tag]]
      path = "channel"
      type = "int"

And oh, in case it’s interesting, here’s my output:

[[outputs.influxdb_v2]]
  urls = ["http://localhost:8086"]
  token = "$INFLUX_TOKEN"
  organization = "foobar"
  bucket = "metrics"
2

I’m not sure why you need to change your setup if it works, but I would switch to the xpath_json parser instead of using the more complex json_v2 one:

[[inputs.mqtt_consumer]]
  servers = ["tcp://127.0.0.1:1883"]
  topics = [
    # Real topic structure
    # "home/OpenMQTTGateway_lilygo_rtl_433_ESP/RTL_433toMQTT/Bresser-3CH/1/130",
    "home/OpenMQTTGateway_lilygo_rtl_433_ESP/RTL_433toMQTT/#"
  ]

  topic_tag = "topic"
  qos = 0

  username = "telegraf"
  password = "notmyrealpassword"

  data_format = "xpath_json"
  xpath_native_types = true

  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "+/+/+/+/+/+"
    tags = "site/_/_/model/channel/device_id"

  [[inputs.mqtt_consumer.xpath]]
    metric_name = "'mymetric'"

    [inputs.mqtt_consumer.xpath.fields]
      battery_ok = "/battery_ok = 1"
      temperature = "/temperature_C"
      humidity = "/humidity"
      rssi = "/rssi"

    [inputs.mqtt_consumer.xpath.tags]
      channel = "/channel"
      id = "/id"
      model = "/model"

which produces

> mymetric,channel=1,host=Hugin,id=130,model=Bresser-3CH battery_ok=true,humidity=19,rssi=-61,temperature=12.66667 1728307076000000000

for your example…

Regarding topic-parsing: You can leave this out if you don’t need the tags defined there…

3

I’m not sure why you need to change your setup if it works

I just want to understand how it works if I have to work with a more complex data structure in the future, as I didn’t understand why I couldn’t make it work with json_v2.

Also, thank you very much!

4

Yeah for more complex things I find the xpath parser much more convenient compared to json_v2…