Json_v2 how to have all elements in "tags" object added as tags?

Telegraf
,
1

Hello,

I need to parse the following message, which is the json format generated by output.mqtt plugin when using “json” as format.

eg.

 {
      "fields": {
        "temperature": 15.47
      },
      "name": "temperature",
      "tags": {
        "area": "INT",
        "device": "input",
        "equipment": "EQP1",
        "site": "DEV_INTEGRATION",
        "source": "zigbee",
        "topic": "zigbee2mqtt/EQP1/temperature/input"
      },
      "timestamp": 1683297978470
    }

Right now I’m using json_v2 to parse it with the code below

  [[inputs.mqtt_consumer.json_v2]]
      measurement_name_path = "name" 
      timestamp_path = "timestamp"
      timestamp_format = "unix_ms"
      [[inputs.mqtt_consumer.json_v2.object]]
          path = "@this"
          tags = ["tags_area", "tags_host", "tags_site", "tags_source", "tags_topic", "tags_equipment", "tags_device", "name"]
          disable_prepend_keys = true
          excluded_keys = ["timestamp"]

I’ve got 2 questions

  1. Is there already a json or json_v2 to parse this template ?
  2. If not, using json_v2, I want the the json parse to use treat all keys contained in “tags” as a tag, and not only the list I’ve predefined. Is it possible to do so ?
2

The query syntax supports gjson.
Here’s an example of handling specific values:

But also defining tags is optional by default theyll be fields

  • tags (OPTIONAL): You can define json keys to be set as tags instead of fields, if you define a key that is an array or object then all nested values will become a tag.

For example:
Example JSON:


{
    "book": {
        "title": "The Lord Of The Rings",
        "chapters": [
            "A Long-expected Party",
            "The Shadow of the Past"
        ],
        "author": "Tolkien",
        "characters": [
            {
                "name": "Bilbo",
                "species": "hobbit"
            },
            {
                "name": "Frodo",
                "species": "hobbit"
            }
        ],
        "random": [
            1,
            2
        ]
    }
}

Example configuration:

[[inputs.file]]
    files = ["./testdata/multiple_arrays_in_object/input.json"]
    data_format = "json_v2"
    [[inputs.file.json_v2]]
        [[inputs.file.json_v2.object]]
            path = "book"
            tags = ["title"]
            disable_prepend_keys = true

Expected line protocol:

file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",chapters="A Long-expected Party"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",chapters="The Shadow of the Past"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",name="Bilbo",species="hobbit"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",name="Frodo",species="hobbit"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",random=1
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",random=2

So if you were to make characters a tag it would list out the characters as you expect with the field structure here but just as a tag instead.

1 Like
3

@cyril.jean,

Here are a collection of examples that can be helpful too:

4

@cyril.jean for completeness, here the way to parse the input using the xpath parser

[[inputs.mqtt_consumer]]
  ...
  data_format = "xpath_json"

  xpath_native_types = true

  [[inputs.mqtt_consumer.xpath]]
    metric_name = "/name"
    timestamp = "/timestamp"
    timestamp_format = "unix_ms"
    field_selection = "/fields/*"
    tag_selection = "/tags/*"
1 Like