Electricity data emitted from json file in Telegraf

Telegraf
, , , ,
1

I’m trying to parse the following json data with Telegraf and InfluxDB (both latest docker images) - but I cannot get it to parse the values and the dates to get the correct output.

Here’s sample JSON data:

[
  {
    "MPRN": "123456",
    "Meter Serial Number": "678910",
    "Read Value": "0.305000",
    "Read Type": "Active Import Interval (kW)",
    "Read Date and End Time": "09-12-2023 03:30"
  },
  {
    "MPRN": "123456",
    "Meter Serial Number": "678910",
    "Read Value": "0.278000",
    "Read Type": "Active Import Interval (kW)",
    "Read Date and End Time": "09-12-2023 03:00"
  },
  {
    "MPRN": "123456",
    "Meter Serial Number": "678910",
    "Read Value": "0.336000",
    "Read Type": "Active Import Interval (kW)",
    "Read Date and End Time": "09-12-2023 02:30"
  },
  {
    "MPRN": "10016256272",
    "Meter Serial Number": "23125313",
    "Read Value": "0.379000",
    "Read Type": "Active Import Interval (kW)",
    "Read Date and End Time": "09-12-2023 02:00"
  }
]

here’s my Telegraf config:

# Parse a complete file each interval
[[inputs.file]]
  files = ["/etc/telegraf/json_data.json"]

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "json_v2"

  [[inputs.file.json_v2]]
    measurement_name = "Electricity_Usage" # A string that will become the new measurement name
    timestamp_path = "@this.#.Read Date and End Time" # A string with valid GJSON path syntax to a valid timestamp (single value)
    timestamp_format = "02-01-2006 15:04" # A string with a valid timestamp format (see below for possible values)
    timestamp_timezone = "Europe/Dublin" # A string with with a valid timezone (see below for possible values)

    [[inputs.file.json_v2.object]]
      path = "@this.#" # A string with valid GJSON path syntax

      [[inputs.file.json_v2.field]]
        path = "@this.#.Read Value" # A string with valid GJSON path syntax
        rename = "kW" # A string with a new name for the tag key
        type = "float" # A string specifying the type (int,uint,float,string,bool)

[[outputs.file]]
  ## Files to write to, "stdout" is a specially handled file.
  files = ["stdout"]

which outputs:

Electricity_Usage,host=XxX kW=0.602 1702242360000000000
Electricity_Usage,host=XxX kW=0.398 1702242360000000000
Electricity_Usage,host=XxX kW=0.464 1702242360000000000
Electricity_Usage,host=XxX kW=0.564 1702242360000000000
Electricity_Usage,host=XxX kW=0.754 1702242360000000000
Electricity_Usage,host=XxX kW=0.578 1702242360000000000
Electricity_Usage,host=XxX kW=0.683 1702242360000000000
Electricity_Usage,host=XxX kW=0.815 1702242360000000000
Electricity_Usage,host=XxX kW=0.671 1702242360000000000
Electricity_Usage,host=XxX kW=0.661 1702242360000000000
Electricity_Usage,host=XxX kW=0.74 1702242360000000000
Electricity_Usage,host=XxX kW=0.671 1702242360000000000
Electricity_Usage,host=XxX kW=0.62 1702242360000000000
Electricity_Usage,host=XxX kW=0.868 1702242360000000000
Electricity_Usage,host=XxX kW=0.574 1702242360000000000
Electricity_Usage,host=XxX kW=0.378 1702242360000000000
Electricity_Usage,host=XxX kW=0.411 1702242360000000000
Electricity_Usage,host=XxX kW=0.562 1702242360000000000
Electricity_Usage,host=XxX kW=0.307 1702242360000000000
Electricity_Usage,host=XxX kW=0.344 1702242360000000000
Electricity_Usage,host=XxX kW=0.479 1702242360000000000
Electricity_Usage,host=XxX kW=0.511 1702242360000000000

So it looks like it’s parsing the kW correctly, but the timestamp is now which is incorrect. I’m also not sure if it’s enumerating both correctly - one kW value should be tied to one timestamp - some experimentation I did had them repeating (calling them fields?). I’m new to Telegraf/influxdb - so maybe it’s blindingly obvious and I just can’t see it. I have read up on GJSON and the new JSONv2 Telegraf parser a good bit, as well as looking through examples on GitHub and I can’t crack it yet.

Any help much appreciated !

2

managed to solve the issue after with this config:

[[inputs.file]]
  files = ["json_data.json"]

  data_format = "json_v2"

  [[inputs.file.json_v2]]
    measurement_name = "Electricity_Usage"

    [[inputs.file.json_v2.object]]
      path = "@this" # A string with valid GJSON path syntax
      tags = ["Read_Type"]
      included_keys = ["Read_Type", "Read_Value", "Read_Date_and_End_Time"]
      excluded_keys = ["MPRN", "Meter_Serial_Number"]
      timestamp_key = "Read_Date_and_End_Time" 
      timestamp_format = "02-01-2006 15:04" 
      timestamp_timezone = "Europe/Dublin" 
      
      [inputs.file.json_v2.object.fields]
        Read_Value = "float"