A little help parsing json with telegraf

I have an mqtt input consumer that parses almost perfectly but a bit of data is missing. I am having issues getting one field from an array into influx.

By default almost everything makes it into influxdb but the string inside of gtw_id field is excluded. I have no way of changing the data format; it’s out of my control. I was able to achieve partially desired results with the following config

  data_format = "json"
  json_query = "metadata.gateways"
  json_string_fields = ["gtw_id"]

This give me the gtw_id field but the only problem with this config is that it excludes almost all of the other keys and fields. What can I do to get all of the json parsed an into influx?

Here is my json:

  "app_id": "app0",
  "dev_id": "dev0",
  "hardware_serial": "AAAAAAAA",
  "port": 99,
  "counter": 685,
  "payload_raw": "AHMnlQFnARsCaHcDAGQEAQA=",
  "payload_fields": {
    "barometric_pressure_0": 1013.3,
    "digital_in_3": 100,
    "digital_out_4": 0,
    "relative_humidity_2": 59.5,
    "temperature_1": 28.3
  "metadata": {
    "time": "2019-03-26T21:55:45.207597292Z",
    "frequency": 903.9,
    "modulation": "LORA",
    "data_rate": "SF7BW125",
    "airtime": 71936000,
    "coding_rate": "4/5",
    "gateways": [
        "gtw_id": "gateway0",
        "gtw_trusted": true,
        "timestamp": 3231141555,
        "time": "2019-03-26T21:55:45.163165111Z",
        "channel": 0,
        "rssi": -38,
        "snr": 9.5,
        "rf_chain": 0,
        "latitude": 00.399174,
        "longitude": 00.25094
    "latitude": 00.399174,
    "longitude": -00.25094,
    "location_source": "registry"


What other fields are you expecting, can you post some expected output?

With your json_query, the only json data available is that within the gateways array.

You may just want to remove the query, but without seeing what data you are expecting, it’s impossible to know for sure.

  data_format = "json"
  json_string_fields = ["metadata_gateways_*_gtw_id"]


I’m expecting to see something like : metadata_gateways_0_gtw_id, metadata_gateways_1_gtw_id

Without any query, out of the box I get the following fields in influxdb

It goes in the array but it excludes the strings, namely gtw_id and time, both of which are in the metadata.gateways array.

I’d like all of the fields that are in the output including the strings. If I can’t get everything I’d at least like to get gtw_id

This didn’t work. It doesn’t like the wildcard for some reason. I’ve been trying these combinations and getting no where.

If I explicitly put this I get the fields:
json_string_fields = ["metadata_gateways_0_gtw_id","metadata_gateways_1_gtw_id"]

I’m using Telegraf 1.8.3 (git: HEAD f2979106)

Can you try using the 1.10 release? IIRC, there were some changes to the json parser.

telegraf --version
Telegraf 1.10.1 (git: HEAD a6778f46)

Still doesn’t work :expressionless: