A little help parsing json with telegraf

Telegraf
1

I have an mqtt input consumer that parses almost perfectly but a bit of data is missing. I am having issues getting one field from an array into influx.

By default almost everything makes it into influxdb but the string inside of gtw_id field is excluded. I have no way of changing the data format; it’s out of my control. I was able to achieve partially desired results with the following config

  data_format = "json"
  json_query = "metadata.gateways"
  json_string_fields = ["gtw_id"]

This give me the gtw_id field but the only problem with this config is that it excludes almost all of the other keys and fields. What can I do to get all of the json parsed an into influx?

Here is my json:

{
  "app_id": "app0",
  "dev_id": "dev0",
  "hardware_serial": "AAAAAAAA",
  "port": 99,
  "counter": 685,
  "payload_raw": "AHMnlQFnARsCaHcDAGQEAQA=",
  "payload_fields": {
    "barometric_pressure_0": 1013.3,
    "digital_in_3": 100,
    "digital_out_4": 0,
    "relative_humidity_2": 59.5,
    "temperature_1": 28.3
  },
  "metadata": {
    "time": "2019-03-26T21:55:45.207597292Z",
    "frequency": 903.9,
    "modulation": "LORA",
    "data_rate": "SF7BW125",
    "airtime": 71936000,
    "coding_rate": "4/5",
    "gateways": [
      {
        "gtw_id": "gateway0",
        "gtw_trusted": true,
        "timestamp": 3231141555,
        "time": "2019-03-26T21:55:45.163165111Z",
        "channel": 0,
        "rssi": -38,
        "snr": 9.5,
        "rf_chain": 0,
        "latitude": 00.399174,
        "longitude": 00.25094
      }
         ],
    "latitude": 00.399174,
    "longitude": -00.25094,
    "location_source": "registry"
  }
}

Thanks!

2

What other fields are you expecting, can you post some expected output?

With your json_query, the only json data available is that within the gateways array.

Edit:
You may just want to remove the query, but without seeing what data you are expecting, it’s impossible to know for sure.

  data_format = "json"
  json_string_fields = ["metadata_gateways_*_gtw_id"]
3

Hello,

I’m expecting to see something like : metadata_gateways_0_gtw_id, metadata_gateways_1_gtw_id

Without any query, out of the box I get the following fields in influxdb

It goes in the array but it excludes the strings, namely gtw_id and time, both of which are in the metadata.gateways array.

I’d like all of the fields that are in the output including the strings. If I can’t get everything I’d at least like to get gtw_id

This didn’t work. It doesn’t like the wildcard for some reason. I’ve been trying these combinations and getting no where.

If I explicitly put this I get the fields:
json_string_fields = ["metadata_gateways_0_gtw_id","metadata_gateways_1_gtw_id"]

I’m using Telegraf 1.8.3 (git: HEAD f2979106)

4

Can you try using the 1.10 release? IIRC, there were some changes to the json parser.

5 
telegraf --version
Telegraf 1.10.1 (git: HEAD a6778f46)

Still doesn’t work :expressionless: