How to parse json array from http source

Telegraf
1

Scenario: I have a sensor pack with various metrics to be collected, and the sensor itself is modular and meant to be reconfigured on the fly with various sensor sources (everything from temperature, pump control, light detection, leak detection, and power usage stats). In line with this, data is returned from the sensor platform like so with multiple metrics stats in a json array:

[{
	"id": "01",
	"temperature": "19.41",
	"pressure": "990.48",
	"humidity": "42.793",
	"gas_resistance": "12535991.793939594",
	"type": "BME680"
}, {
	"id": "02",
	"temperature": "23.21",
	"type": "1wire"
}, {
	"id": "03",
	"temperature": "16.11",
	"type": "1wire"
}]

I’m looking for the best way to get Telegraf to parse this from a http source and stream the metrics to influx as individual events. The key here is that I need it to handle and stream any data that it gets in the json array.

I’ve been fighting this for a while and wondering what the “right” way to do this is. Any insight would be appreciated as I don’t see to much about this exact scenario. I have looked around but haven’t found an answer to this, so if it exists, feel free to point me to it.

2

Hi @M_B,
So the JSON_V2 parser should fit your needs here is an example (I used the file input plugin with your data but you can swap it to the http plugin)


###############################################################################
#                            INPUT PLUGINS                                    #
###############################################################################
[[inputs.file]]
  ## Files to parse each interval.  Accept standard unix glob matching rules,
  ## as well as ** to match recursive files and directories.
  files = ["./data/sensor3.json"]

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "json_v2"
 
 [[inputs.file.json_v2]]

 [[inputs.file.json_v2.object]]
  path = "@this"
  disable_prepend_keys = true
  tags = ["id", "type"]
1 Like
3

Looking at your JSON payload I would also make sure you convert the sensor values from strings to floats, You can do this in InfluxDB but it would save a tone of heartache if you do it before:


# Convert values to another metric value type
[[processors.converter]]
  ## Fields to convert
  ##
  ## The table key determines the target type, and the array of key-values
  ## select the keys to convert.  The array may contain globs.
  ##   <target-type> = [<field-key>...]
  [processors.converter.fields]
    measurement = []
    tag = []
    string = []
    integer = []
    unsigned = []
    boolean = []
    float = ["gas_resistance", "humidity", "pressure", "temperature"]

Full example here: Telegraf-Community-Configs/telegraf-json-v2-sensor-list.conf at master · InfluxCommunity/Telegraf-Community-Configs · GitHub

1 Like
4

Awesome! Testing now but this does not looks the same as my config (Looks like I 100% missed the .json_v2.object block and was defining this in .json_v2.

I took your other advice and rolled the sensor value conversions into my conf too. Will mark as the solution once I successfully test it.

EDIT: Tested and it works flawlessly. Thanks for the assistance here @Jay_Clifford, I seriously appreciate it.

1 Like