Telegraf does not create a measurement called mqtt_consumer


I set up MQTT → Mosquitto → Telegraf → InfluxDB 1.8 → Grafana. All of this software is running on Docker on a Raspberry Pi with Raspberry PI OS.

Data is received from Mosquitto and taken by Telegraf to InfluxDB.

My telegraf.conf is the following:


  interval = "30s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  hostname = "BKW"
  omit_hostname = false
  debug = true

  urls = ["http://influxdb:8086"]
  database = "influx"
  timeout = "5s"
  username = "*****"
  password = "******"

  servers = ["tcp://mosquitto:1883"]
  topics = [
  qos = 0
  connection_timeout = "30s"
  username = "*****"
  password = "******"
  data_format = "value"
  data_type = "float"

  topic = "inverter/+/+/+"
  measurement = "measurement/_/_/_"
  tags = "_/name/ch/field"

  topic = "inverter/+/+"
  measurement = "measurement/_/_"
  tags = "_/name/field"

  tag_key = "field"
  value_key = "value"

This generates only the measurement inverter in the influxdb. But no measurement called mqtt_consumer.

The data in inverter is complete, but not useable by Grafana. The table contains for each value an own column. But the transformation in telegraf should actually output a smaller version which contains the field and the value.

In case I run mosquitto_sub with the given topic selection I receive what I need:

mosquitto_sub -h localhost -t inverter/+/+/+ -u -P
inverter/BKW/ch1/YieldDay 274
inverter/BKW/ch1/YieldTotal 12.814
inverter/BKW/ch2/YieldDay 840
inverter/BKW/ch2/YieldTotal 16.973
inverter/BKW/ch0/YieldDay 1114
inverter/BKW/ch0/YieldTotal 29.787

Running telegraf in test mode returned:

2023-04-23T19:20:57Z I! Loading config file: /etc/telegraf/telegraf.conf
2023-04-23T19:20:57Z I! Starting Telegraf 1.26.1
2023-04-23T19:20:57Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-04-23T19:20:57Z I! Loaded inputs: mqtt_consumer
2023-04-23T19:20:57Z I! Loaded aggregators: 
2023-04-23T19:20:57Z I! Loaded processors: pivot
2023-04-23T19:20:57Z I! Loaded secretstores: 
2023-04-23T19:20:57Z W! Outputs are not used in testing mode!
2023-04-23T19:20:57Z I! Tags enabled: host=BKW
2023-04-23T19:20:57Z D! [agent] Initializing plugins
2023-04-23T19:20:57Z D! [agent] Starting service inputs
2023-04-23T19:20:57Z I! [inputs.mqtt_consumer] Connected [tcp://mosquitto:1883]
2023-04-23T19:20:57Z D! [agent] Stopping service inputs
2023-04-23T19:20:57Z D! [inputs.mqtt_consumer] Disconnecting [tcp://mosquitto:1883]
2023-04-23T19:20:57Z D! [inputs.mqtt_consumer] Disconnected [tcp://mosquitto:1883]
2023-04-23T19:20:57Z D! [agent] Input channel closed
2023-04-23T19:20:57Z D! [agent] Processor channel closed
2023-04-23T19:20:57Z D! [agent] Stopped Successfully

Which looks actually normal.

Furthermore the log files of telegraf do not show any issues:

2023-04-23T17:58:13Z I! Loading config file: /etc/telegraf/telegraf.conf
2023-04-23T17:58:13Z I! Starting Telegraf 1.26.1
2023-04-23T17:58:13Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-04-23T17:58:13Z I! Loaded inputs: mqtt_consumer
2023-04-23T17:58:13Z I! Loaded aggregators: 
2023-04-23T17:58:13Z I! Loaded processors: pivot
2023-04-23T17:58:13Z I! Loaded secretstores: 
2023-04-23T17:58:13Z I! Loaded outputs: influxdb
2023-04-23T17:58:13Z I! Tags enabled: host=BKW
2023-04-23T17:58:13Z I! [agent] Config: Interval:30s, Quiet:false, Hostname:"BKW", Flush Interval:10s
2023-04-23T17:58:13Z D! [agent] Initializing plugins
2023-04-23T17:58:13Z D! [agent] Connecting outputs
2023-04-23T17:58:13Z D! [agent] Attempting connection to [outputs.influxdb]
2023-04-23T17:58:13Z D! [agent] Successfully connected to outputs.influxdb
2023-04-23T17:58:13Z D! [agent] Starting service inputs
2023-04-23T17:58:13Z I! [inputs.mqtt_consumer] Connected [tcp://mosquitto:1883]
2023-04-23T17:58:23Z D! [outputs.influxdb] Wrote batch of 10 metrics in 20.122597ms
2023-04-23T17:58:23Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2023-04-23T17:58:33Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2023-04-23T17:58:43Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2023-04-23T17:58:53Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

What I do wrong in order to get the measurement mqtt_consumer with the data Grafana actually needs?


Unfortunately, it does not. Test mode should print out any metrics that Telegraf has received and parsed. However, because you are using the mqtt consumer, a service input, metrics are only created when they are received, not at every interval. As such --test mode will not work with mqtt consumer.

2023-04-23T17:58:23Z D! [outputs.influxdb] Wrote batch of 10 metrics in 20.122597ms

It looks like Telegraf did write something based on this log message. Are you really really sure nothing got written? :slight_smile:

It writes data, but not in mqtt_consumer. But I could solve it by removing the sections inputs.mqtt_consumer.topic_parsing.