Json_v2 array parsing

Hi,

I’m trying to parse an array with several measurements but I’m unable to solve it. I tried to find a solution but it seems that json_v2 is unable to do what I want. I’m here to ask if I oversaw a feature or if I have to find a completely different solution to fetch my data.

My JSON:

{
   "applicationID":"1",
   "applicationName":"Test",
   "deviceName":"TcsSensorNode",
   "deviceProfileName":"TcsSensor",
   "deviceProfileID":"100ca98d-e075-4b1b-8cd3-41edbab355f5",
   "devEUI":"27c0817d2aba3052",
   "rxInfo":[
      {
         "gatewayID":"b827ebfffeaa4582",
         "uplinkID":"659cbfab-3216-42fd-9f71-f2c470b7f9da",
         "name":"local",
         "rssi":-60,
         "loRaSNR":7.5
      },
      {
         "gatewayID":"ca925641ce08b33e",
         "uplinkID":"15ca3b44-17a0-4662-82c9-aa23b40e16eb",
         "name":"local",
         "rssi":-98,
         "loRaSNR":4.2
      }
   ],
   "txInfo":{
      "frequency":868500000,
      "dr":5
   },
   "adr":true,
   "fCnt":71,
   "fPort":2,
   "data":"AGhCAGcA0g==",
   "object":{
      "humiditySensor":{
         "0":33,
         "1":25.5
      },
      "temperatureSensor":{
         "0":21,
         "1":19.3
      }
   }
}

The problem is that that in the rxInfo array only the second values are used. Solutions I found through googling was that the array might need to be flattened first. But I can’t change the format of this. If I write my own application to receive the values I don’t need Telegraf anymore I can directly push to the database.

The current config:

  [[inputs.mqtt_consumer.json_v2]]
    measurement_name = "sensors_test2"

    [[inputs.mqtt_consumer.json_v2.object]]
      path = "rxInfo"
      included_keys = ["rssi", "loRaSNR"]
      [inputs.mqtt_consumer.json_v2.object.fields]
        rssi = "float"
        loRaSNR = "float"
    [[inputs.mqtt_consumer.json_v2.object]]
      path = "txInfo"

Hope somebody knows a solution. Thank you :slight_smile:

This is most likely because you have no tags that differentiate the first and second array items. Internally your current config should create metrics like:

metric rssi=-6,loRaSNR=7.5
metric rssi=-98,IoRaSNR=4.2

There is nothing in there to differentiate between the two and so the last one will win.

I would have expected that it would create something like 0_rssi and 1_rssi. I’m not sure if tags are a good solution.

Anyway I would need a tag like rxInfo with the value 0 and 1. So the iterator should become the value. I also don’t know how many items are in the array. It depends on how many gateways received the message. I can’t find info on how I can use the array iterator in any way in the documentation.

That is my main issue. The documentation just does not apply to the usecase here. Arrays are apparently not thought about.

Hi,
thanks for your suggestion. Really appreciated.
I tied to put the iterator of the array into a tag, but I can’t find any info on how to do that. As you suggested that, do you know the approach to do so?

I do not, @srebhan are you aware of any way to go through a JSON array and create key’s with the index as a part of it?

1 Like

@shilga while playing with your example I found a panic in the xpath parser, so thanks for bringing up this challenge! :wink: Anyway, you can use the binary from fix(parsers.xpath): Fix panic for JSON name expansion by srebhan · Pull Request #12724 · influxdata/telegraf · GitHub with

[[inputs.file]]
  files = ["test_configs/jsontest_forum_shilga.json"]
  data_format = "xpath_json"
  xpath_native_types = true

  [[inputs.file.xpath]]
    metric_name = "'foo'"
    field_name_expansion = true
    field_selection = "descendant::*"

you will get

> foo adr=true,applicationID="1",applicationName="Test",data="AGhCAGcA0g==",devEUI="27c0817d2aba3052",deviceName="TcsSensorNode",deviceProfileID="100ca98d-e075-4b1b-8cd3-41edbab355f5",deviceProfileName="TcsSensor",fCnt=71,fPort=2,object_humiditySensor_0=33,object_humiditySensor_1=25.5,object_temperatureSensor_0=21,object_temperatureSensor_1=19.3,rxInfo_0_gatewayID="b827ebfffeaa4582",rxInfo_0_loRaSNR=7.5,rxInfo_0_name="local",rxInfo_0_rssi=-60,rxInfo_0_uplinkID="659cbfab-3216-42fd-9f71-f2c470b7f9da",rxInfo_1_gatewayID="ca925641ce08b33e",rxInfo_1_loRaSNR=4.2,rxInfo_1_name="local",rxInfo_1_rssi=-98,rxInfo_1_uplinkID="15ca3b44-17a0-4662-82c9-aa23b40e16eb",txInfo_dr=5,txInfo_frequency=868500000 1677100755000000000

Is this what you expect?