Telegraph configuring - json parsing question

Telegraf
1

Hi,

I am looking for a bit of help with configuring Telegraf for parsing Json data. Say, my input data is sent as

        "measurements": [
            {
                "name": "temperature",
                "value": 23.4,
                "units": "℃"
            },
            {
                "name": "moisture",
                "value": 5,
                "units": "%"
            },
            {
                "name": "light",
                "value": 10118,
                "units": "lux"
            },
            {
                "name": "fertility",
                "value": 0,
                "units": "us/cm"
            },
            {
                "name": "time",
                "value": 1555745371410,
                "units": "ms"
            }
        ]
    }

and the Telegraf Json parser is configured as

  json_query = "measurements"

  tag_keys = [
    "name",
    "units"
  ]

then everything looks quite nice:

> select * from mqtt_consumer
name: mqtt_consumer
time                host        name        topic                                                         units value
----                ----        ----        -----                                                         ----- -----
1555745371450794118 *** temperature iot-2/type/TeMoLiFe/id/c67c8dfffe65cb63/evt/PubState/fmt/json ℃     24.3
1555745371450808740 *** moisture    iot-2/type/TeMoLiFe/id/c67c8dfffe65cb63/evt/PubState/fmt/json %     6
1555745371450815925 *** light       iot-2/type/TeMoLiFe/id/c67c8dfffe65cb63/evt/PubState/fmt/json lux   10168
1555745371450822476 *** fertility   iot-2/type/TeMoLiFe/id/c67c8dfffe65cb63/evt/PubState/fmt/json us/cm 58
1555745371450828165 *** time        iot-2/type/TeMoLiFe/id/c67c8dfffe65cb63/evt/PubState/fmt/json ms    1555745371375
1555745371505989365 *** temperature iot-2/type/TeMoLiFe/id/c67c8dfffe65d020/evt/PubState/fmt/json ℃     23.4
1555745371506015142 *** moisture    iot-2/type/TeMoLiFe/id/c67c8dfffe65d020/evt/PubState/fmt/json %     5
1555745371506029556 *** light       iot-2/type/TeMoLiFe/id/c67c8dfffe65d020/evt/PubState/fmt/json lux   10118
1555745371506043834 *** fertility   iot-2/type/TeMoLiFe/id/c67c8dfffe65d020/evt/PubState/fmt/json us/cm 0
1555745371506056170 *** time        iot-2/type/TeMoLiFe/id/c67c8dfffe65d020/evt/PubState/fmt/json ms    1555745371410

besides that I am not sure how to extract the time value into the time column.

If I change the input json to

{
    "time": 1555745371410,
    "measurements": [
        {
            "name": "temperature",
            "value": 23.4,
            "units": "℃"
        },
        {
            "name": "moisture",
            "value": 5,
            "units": "%"
        },
        {
            "name": "light",
            "value": 10118,
            "units": "lux"
        },
        {
            "name": "fertility",
            "value": 0,
            "units": "us/cm"
        }
    ]
}

then I can get the time value in the time column with

  data_format = "json"
  json_time_key = "time"
  json_time_format = "unix_ms"

but I do not know how to extract sensor data as nicely as above. What should be in the json_query field then?

Thank you!

2

I don’t think it is possible to handle this currently, if you have control over the input the best workaround would be to move the timestamp into the measurements:

{
    "measurements": [
        {
            "name": "temperature",
            "time": 1555745371410,
            "units": "℃",
            "value": 23.4
        },
        {
            "name": "moisture",
            "time": 1555745371410,
            "units": "%",
            "value": 5
        },
    ]
}

Do you think you could create a feature request for a way to handle this on the Telegraf issue tracker?

3

Sure, I’ll make a feature request - need some time to figure out what json schema would be the best. In the meantime time I think I can live with the automatically generated timestamps.

I do have control over the input, and I have a feeling that the best option is probably the influx format i.s.o. json. Not completely sure about that yet, because I have other sensor data consumers - need to check if they’d be able to consume influx format.

Another thing that bothers me a bit - how to deal with units. I’d like units come from sensors, but the way how it is done currently is a bit heavy i.m.o. I’ve just started with TSDB, still looking for a nice generic way of how to deal with measurement units. I’d like units be explicit on the wire but in a less verbose way - not sent with every data point.

4

I usually try to put the units in the field name, for example moisture_percent or temp_c. You could try to use a secondary field to store the units, and send the units less frequently: temp=42.0,temp_units="c", but I think tools like Grafana/Chronograf wouldn’t really be able to handle it. I would probably only do this if you are planning to use a custom display tool.

5

I’ve created a feature request, please see here: