Help Telegraf JSON import and how to flatten it's data

Telegraf
1

Hi all

I have just started with telegraf and and therefore i’m a newbie …
My intention is to collect JSON data from my home electricity meter by telegraf then put them into influxdb and visualize by grafana. My meter does support collecting data by JSON download.
Telegraf 1.22.4 is running on Linux.

therefore i need some advice/assistance in configuring telegraf to:

  • get JSON data from electricity meter
  • flatten the JSON data

json readout below; primarily need to flatten the JSON data to Voltage reading as an example:
Position 17: Volt Phase L1 / 240.1
Position 18: Volt Phase L2 / 238.6
Position 19: Volt Phase L3 / 240.1

also need to have “DeviceId” within the flattened data to separate, will have 2 electricity meters later on.

feedback is very much appreciated,
regards

{
   "TimeUTC":"Wed May 18 20:45:00 2022",
   "Timestamp":1652906700,
   "TimezoneOffset":-120,
   "Device":{
      "Id":7,
      "AddressMode":0,
      "PrimaryAddress":0,
      "ManufacturerId":"EMU",
      "Serial":"160755",
      "Version":16,
      "Medium":"Electricity",
      "Active":-1,
      "ReadoutCycle":0,
      "BaudRate":2400,
      "BACNetDevInstNumber":0,
      "Name":"House",
      "Site":"",
      "CostUnit":"",
      "CommentStr":"",
      "LoggerLastReadout":1652906700,
      "LoggerReadoutState":1,
      "LoggerReadoutCycle":60,
      "MediumGroup":1,
      "Battery":false,
      "DeviceType":0,
      "BACNetDataBaseRevision":0,
      "AppResetSubCode":0,
      "ErrorFlags":0,
      "ValueDescs":[
         {
            "Id":7,
            "DeviceId":7,
            "Position":0,
            "DescriptionStr":"Energy",
            "UnitStr":"Wh",
            "ScalePower":0,
            "ScaleMantissa":0,
            "EncodeType":0,
            "ValueType":0,
            "StorageNum":0,
            "Tariff":1,
            "Active":-1,
            "LoggerLastValue":267520.0,
            "CfgDescription":3,
            "CfgUnit":2,
            "CfgPhase":0,
            "CfgFactor":1000.0,
            "CfgStorageNum":0,
            "CfgTariff":1,
            "CfgObisCode":"1-0:1.8.1",
            "Values":[
               {
                  "Id":7,
                  "Timestamp":1652906700,
                  "Val":267520.0
               }
            ]
         },
output ommited
         {
            "Id":24,
            "DeviceId":7,
            "Position":17,
            "DescriptionStr":"Volts (vendor specific)",
            "UnitStr":"V",
            "ScalePower":0,
            "ScaleMantissa":0,
            "EncodeType":0,
            "ValueType":0,
            "StorageNum":0,
            "Tariff":0,
            "Active":-1,
            "LoggerLastValue":240.1,
            "CfgDescription":28,
            "CfgUnit":26,
            "CfgPhase":1,
            "CfgFactor":0.0,
            "CfgStorageNum":0,
            "CfgTariff":0,
            "CfgObisCode":"",
            "Values":[
               {
                  "Id":24,
                  "Timestamp":1652906700,
                  "Val":240.1
               }
            ]
         },
         {
            "Id":25,
            "DeviceId":7,
            "Position":18,
            "DescriptionStr":"Volts (vendor specific)",
            "UnitStr":"V",
            "ScalePower":0,
            "ScaleMantissa":0,
            "EncodeType":0,
            "ValueType":0,
            "StorageNum":0,
            "Tariff":0,
            "Active":-1,
            "LoggerLastValue":238.6,
            "CfgDescription":28,
            "CfgUnit":26,
            "CfgPhase":2,
            "CfgFactor":0.0,
            "CfgStorageNum":0,
            "CfgTariff":0,
            "CfgObisCode":"",
            "Values":[
               {
                  "Id":25,
                  "Timestamp":1652906700,
                  "Val":238.6
               }
            ]
         },
         {
            "Id":26,
            "DeviceId":7,
            "Position":19,
            "DescriptionStr":"Volts (vendor specific)",
            "UnitStr":"V",
            "ScalePower":0,
            "ScaleMantissa":0,
            "EncodeType":0,
            "ValueType":0,
            "StorageNum":0,
            "Tariff":0,
            "Active":-1,
            "LoggerLastValue":240.1,
            "CfgDescription":28,
            "CfgUnit":26,
            "CfgPhase":3,
            "CfgFactor":0.0,
            "CfgStorageNum":0,
            "CfgTariff":0,
            "CfgObisCode":"",
            "Values":[
               {
                  "Id":26,
                  "Timestamp":1652906700,
                  "Val":240.1
               }
            ]
         },
output ommited
2

How did you get the data in the first place? And what methods are available? HTTP? Do you have to connect to it? Any authentication required?

In terms of parsing the JSON, given the nested nature of the data I would prefer the use of the client libraries. You can more easily and probably quickly build something that can parse your JSON and craft the exact data you want.

3

Hi

i got the JSON by http, no auth required.
http://192.168.x.x/app/api/serial/160755.json

4

If you do go down the route of telegraf, you can look at the HTTP plugin. However, I still think if you have a favorite programming language, using it + the client libraries are a better option.

5

ok, thanks so far.
will have a look at it

6

Can you provide a full example of this json? (gist, pastebin…)
Your example seems to be shortened.

7

Hi

yes, please find the full json here.
https://we.tl/t-dUiu2KfZyz

8

if i would like to go down the route with telegraf [[inputs.http]] and data_format = “json_v2” i do need to parse the JSON data to roughly get metrcis format like below example.

currently, i’m struggling building a telegraf.conf which does parse the JSON data to get the metrics.

Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 0 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“267520”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 1 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 2 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 3 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 4 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 5 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 6 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 7 DescriptionStr=“Energy”, UnitStr=“Wh”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 8 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 9 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 10 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 11 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 12 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 13 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 14 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 15 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 16 DescriptionStr=“Power”, UnitStr=“W”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 17 DescriptionStr=“Volts”, UnitStr=“V”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 18 DescriptionStr=“Volts”, UnitStr=“V”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 19 DescriptionStr=“Volts”, UnitStr=“V”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 20 DescriptionStr=“Ampere”, UnitStr=“A”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 21 DescriptionStr=“Ampere”, UnitStr=“A”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 22 DescriptionStr=“Ampere”, UnitStr=“A”, LoggerLastValue=“0”
Device, Serial=“160755”, Medium=“Electricity”, Name=“House”, → Out of ValueDescs 23 DescriptionStr=“Ampere”, UnitStr=“A”, LoggerLastValue=“0”