Collecting metrics via mqtt_consumer

borpin · February 22, 2021, 5:54am

I am collecting metrics via the mqtt_consumer input and am wondering if I have this set up in an optimal way. It seems to me this results in a database table with lots of gaps.

MQTT setup

[[inputs.mqtt_consumer]]
  servers = ["tcp://192.168.7.50:1883"]
  name_override = "emon_input"

  topics = [
    "emon/tank",
    "emon/Fineoffset-WHx080-236",
  ]
#   topic_tag = ""

Results

The mqtt_consumer is the measurement.

For each row in the table, I have 2 tags (host and topic) with the topic tag having one of 2 values - the 2 topics I subscribe to.

Each row has a field for a combination of the fields from the 2 MQTT Topics i.e. each row has a field for all the potential metrics from both topics.

When data comes in from topic 1, the row is partially filled with the values from that topic leaving the other fields empty.

Have I got that right?
This seems very inefficient - is it the InfluxDB way?
I think I want each topic as it’s own measurement for efficiency. If so, how?

borpin · February 28, 2021, 2:22pm

Any suggestions how lo load MQTT Topics into measurements without multiple consumers?

reimda · March 25, 2021, 7:47pm

I think the simplest way to get what you want is to make another instance of the mqtt_consumer plugin, but since you asked for alternatives, here are two ways:

First, take a look at the processor plugins. telegraf/plugins/processors at release-1.18 · influxdata/telegraf · GitHub You could use clone to make a copy of the metric, then use fieldpass to remove one field in the original and the other field in the copy. telegraf/CONFIGURATION.md at master · influxdata/telegraf · GitHub

Another approach would be to use the starlark processor. With it you can write a short script in a python-like language to do the same thing. There are many examples in the startlark plugin documentation: telegraf/plugins/processors/starlark at release-1.18 · influxdata/telegraf · GitHub

Franky1 · March 26, 2021, 9:54am

As @reimda already said, i would also go for a processors plugin in Telegraf.
If you need any help, please provide the actual snippet of your mqtt input plugin config and a short snippet of your collected data in influx line protocol format.