Hello,
I need to parse the following message, which is the json format generated by output.mqtt plugin when using “json” as format.
eg.
{
"fields": {
"temperature": 15.47
},
"name": "temperature",
"tags": {
"area": "INT",
"device": "input",
"equipment": "EQP1",
"site": "DEV_INTEGRATION",
"source": "zigbee",
"topic": "zigbee2mqtt/EQP1/temperature/input"
},
"timestamp": 1683297978470
}
Right now I’m using json_v2 to parse it with the code below
[[inputs.mqtt_consumer.json_v2]]
measurement_name_path = "name"
timestamp_path = "timestamp"
timestamp_format = "unix_ms"
[[inputs.mqtt_consumer.json_v2.object]]
path = "@this"
tags = ["tags_area", "tags_host", "tags_site", "tags_source", "tags_topic", "tags_equipment", "tags_device", "name"]
disable_prepend_keys = true
excluded_keys = ["timestamp"]
I’ve got 2 questions
- Is there already a json or json_v2 to parse this template ?
- If not, using json_v2, I want the the json parse to use treat all keys contained in “tags” as a tag, and not only the list I’ve predefined. Is it possible to do so ?
The query syntax supports gjson.
Here’s an example of handling specific values:
But also defining tags is optional by default theyll be fields
-
tags (OPTIONAL): You can define json keys to be set as tags instead of fields, if you define a key that is an array or object then all nested values will become a tag.
For example:
Example JSON:
{
"book": {
"title": "The Lord Of The Rings",
"chapters": [
"A Long-expected Party",
"The Shadow of the Past"
],
"author": "Tolkien",
"characters": [
{
"name": "Bilbo",
"species": "hobbit"
},
{
"name": "Frodo",
"species": "hobbit"
}
],
"random": [
1,
2
]
}
}
Example configuration:
[[inputs.file]]
files = ["./testdata/multiple_arrays_in_object/input.json"]
data_format = "json_v2"
[[inputs.file.json_v2]]
[[inputs.file.json_v2.object]]
path = "book"
tags = ["title"]
disable_prepend_keys = true
Expected line protocol:
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",chapters="A Long-expected Party"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",chapters="The Shadow of the Past"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",name="Bilbo",species="hobbit"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",name="Frodo",species="hobbit"
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",random=1
file,title=The\ Lord\ Of\ The\ Rings author="Tolkien",random=2
So if you were to make characters a tag it would list out the characters as you expect with the field structure here but just as a tag instead.
1 Like
@cyril.jean,
Here are a collection of examples that can be helpful too:
@cyril.jean for completeness, here the way to parse the input using the xpath parser
[[inputs.mqtt_consumer]]
...
data_format = "xpath_json"
xpath_native_types = true
[[inputs.mqtt_consumer.xpath]]
metric_name = "/name"
timestamp = "/timestamp"
timestamp_format = "unix_ms"
field_selection = "/fields/*"
tag_selection = "/tags/*"
1 Like