Dynamic field names in MQTT payload with json_v2

ga29
Tue, 19 Mar 2024 12:13:41 +0000
Telegraf

Hi, I am new to Influx and Telegraf. I am trying to setup Telegraf to parse JSON payload from an MQTT input. The problem is my payload (not under my control) is having dynamic field values. It is a flat structure something like this below.

{
  'timestamp': '2024-03-19 13:09:00',
  'SN': 'AMG2100-DEMO-001',

  'Dynamic_Field_1': 1,
  'Dynamic_Field_2': 'OU',
  'Dynamic_Field_3': 'NONE',
  'Dynamic_Field_4': 101.1,
  'Dynamic_Field_5': 'PPB',
}

The names of the fields denoted by dynamic above change from product to product although the data type remains same.

I tried the json_v2 parser as a first attempt but I think it does not support dynamic keys. Then I am trying the *XPath parser plugin with the following configuration (Also tried some variations of it). But I am getting errors such as plugin inputs.file: line 169: configuration specified the fields ["xpath"], but they were not used. This is either a typo or this config option does not exist in this version.

My telegraf config looks like this:

[[inputs.mqtt_consumer]]
  servers = ["tcp://MY-MQTT-BROKER:1883"]
  ## Topics that will be subscribed to.
  topics = [
    "MY-TOPIC/SUB-TOPIC/#",
  ]

   username = "USERNAME"
   password = "PASSWORD"

  data_format = "xpath_json"
  [[inputs.mqtt_consumer.xpath]]
      # metric_selection = "/"
      timestamp = "time"
      timestamp_format = "2006-01-02 15:04:05Z"
      field_selection = "*"
      [inputs.file.xpath.tags]
        name   = "string('Sensor Product)"
        # device = "string('Sensor data')"

Am I expecting correct that the xpath json parser is the right plugin for my use case? If yes, then where am I going wrong? If no, is there any other alternative plugin?

Thank you very much,
Gopal

3 posts - 2 participants