[inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:34

Telegraf
,
1

Hello, I am trying to configure Telegraf with mqtt_consumer and data_format = "json_v2" and topic_parsing.

I tried many different configurations, as far as I could figure out was, that the topic_parsing configuration is the problem.

This configuration works:

[[inputs.mqtt_consumer]]
  servers = ["tcp://rabbitmq:1883"]
  topics = [
    "telegraf/stocks"
  ]
  username = "xxx"
  password = "xxx"
  data_format = "json_v2"
  [[inputs.mqtt_consumer.json_v2]]
    measurement_name = "stocks"
    [[inputs.mqtt_consumer.json_v2.object]]
      path = "@this"
      tags = ["isin", "name", "unit"] # can list all tags
      [inputs.mqtt_consumer.json_v2.object.fields]
        val = "float"

With this MQTT-message: (topic = telegraf/stocks)

{
    "isin": "TEST",
    "name": "TestStock",
    "unit": "EUR",
    "date": "2022-03-01",
    "val": 123.456789
}

If I remove the measurement_name from the config and try to parse the topic, in order to set the measurement name dynamically I get the following error:

[[inputs.mqtt_consumer]]
  servers = ["tcp://rabbitmq:1883"]
  topics = [
    "telegraf/stocks"
  ]
  username = "xxx"
  password = "xxx"
  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "telegraf/+"
    measurement = "_/measurement"
  data_format = "json_v2"
  [[inputs.mqtt_consumer.json_v2]]
    [[inputs.mqtt_consumer.json_v2.object]]
      path = "@this"
      tags = ["isin", "name", "unit"] # can list all tags
      [inputs.mqtt_consumer.json_v2.object.fields]
        val = "float"

Error message:

2022-04-02T12:09:47Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:34: "{\"isin\":\"TEST\",\"name\":\"TestStock\",\"unit\":\"EUR\",\"date\":\"2022-03-01\",\"val\":123.456789}"

I tried to look at the source code at GitHub, but could not figure out how to configure the topic_parsing in order to work properly.

Can anyone give me a hint on what is wrong with my config. Or is this maybe a bug?

Thanks in advance and,
Best regards.

2

@samdillard I remember you wrote a blog on this topic: MQTT Topic and Payload Parsing with Telegraf | InfluxData

Any idea what might be going wrong here?

1 Like
3

Try:

  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "telegraf/+"
    measurement = "_/stocks"
2 Likes
4

Hi, thank you for looking into my problem.

Unfortunately it is still the same error message:

2022-04-05T18:53:10Z I! Using config file: /etc/telegraf/telegraf.conf
2022-04-05T18:53:10Z I! Starting Telegraf 1.22.0
2022-04-05T18:53:10Z I! Loaded inputs: mqtt_consumer
2022-04-05T18:53:10Z I! Loaded aggregators:
2022-04-05T18:53:10Z I! Loaded processors:
2022-04-05T18:53:10Z I! Loaded outputs: influxdb_v2
2022-04-05T18:53:10Z I! Tags enabled: host=myhost
2022-04-05T18:53:10Z I! [agent] Config: Interval:1m0s, Quiet:false, Hostname:"myhost", Flush Interval:10s
2022-04-05T18:53:10Z I! [inputs.mqtt_consumer] Connected [tcp://rabbitmq:1883]
2022-04-05T19:00:05Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:34: "{\"isin\":\"TEST\",\"name\":\"TestStock\",\"unit\":\"EUR\",\"date\":\"2022-03-01\",\"val\":123.456789}"
5

I used this config with your data and it works:

[[inputs.mqtt_consumer]]
  servers = ["tcp://mqtt.flespi.io:1883"]
  topics = [
    "telegraf/stocks"
  ]
  username = "xxxxxxxxxx"
  password = "xxxxxxxxxx"
  client_id = "telegraf"
  # name_override = "mqtt"
  data_format = "json_v2"
  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "telegraf/+"
    measurement = "_/stocks"
  [[inputs.mqtt_consumer.json_v2]]
    [[inputs.mqtt_consumer.json_v2.object]]
      path = "@this"
      tags = ["isin", "name", "unit"]
      [inputs.mqtt_consumer.json_v2.object.fields]
        val = "float"
        date = "string"
1 Like
6

Okay, thank you. That is strange. Which Telegraf and Influx version do you use? And are you using the docker containers or have you installed both directly on a host system.

Thanks in advance.

7

Telegraf 1.21.4 on Win10 host

1 Like
8

Okay, I have Telegraf 1.22.0

I now tested with 1.21.4 and it produced the same error.

I also added the date = "string" line, but still getting the same error.

It must be something specific to my setup, or maybe with docker, but docker should not be a problem I think.

Good to know that the config is working on your side. I’ll try to figure out what’s wrong on my side the next days.
Thank you very much for your help.

9

@OidaTiftla are you using the config posted by @Franky1 or the one you posted initially (with some edits on top)?

I made a diff and there might be a misplaced keyword, data_format = "json_v2".
In your config that’s under the [[inputs.mqtt_consumer.topic_parsing]] node, but that should be placed before, directly under [[inputs.mqtt_consumer]] (as in Franky config)…
Afaik in TOML indentation does not matter, it’s all about the order of keywords

1 Like
10

@Giovanni_Luisotto oh, thanks, I did not know that the order is relevant. Now it works. That was a stupid mistake.

:rocket: :tada: @Franky1 and @Giovanni_Luisotto thank you very much for solving my problem.