Regex in MQTT broker topics

1

Hi
I am trying to read multiple topics from the MQTT broker using [[inputs.mqtt_consumer]] but I don’t want to read all the topics from the broker using topics = ["#"].

My topic names like this:

broker.1/0/0.ReportCumKpi.2819
broker.1/0/0.ReportNonCumKpi.2819
broker.1/0/0.ReportAlarms.28191
mainbrokerfile/0/0.Report.28191

like this, I have hundreds of topics that have the data of multiple devices.

Now, I want to process the data from the topic which contains ReportCumKpi and ReportNonCumKpi and exclude everything.
I tried using wildcard + and # but I am not able to exclude other topics.

Is there any way I can exclude the rest of the topics and process interesting topics?

I have followed this blog as well but I am not able to identify the topics of my interest.

2

@Jay_Clifford Any chance you can help here?
Thank you!

3

Hi @Ravikant_Gautam,
So you should be able to do something like this:

 topics = [
    "+/+/0.ReportCumKpi.2819",
    "+/+/0.ReportNonCumKpi.2819",
  ]

In short + and # are the primary wildcards supported by most brokers conforming to the MQTT standard. Your other option is to subscribe a wider range of topics:

 topics = [
"broker.1/0/#"
]

  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "broker.1/0/+"
    tags = "_/_/name"

Then use Telegraf to filter out the topics you do not want with something like tagpass:

# Parse a complete file each interval
[[outputs.file]]
  [inputs.file.tagpass]
    name = [ "0.ReportCumKpi.2819", "0.ReportNonCumKpi.2819" ]
  files = ["./tmp/metrics.out"]
  data_format = "json"
1 Like
4

topics = [
“+/+/0.ReportCumKpi.2819”,
“+/+/0.ReportNonCumKpi.2819”,
]

@Jay_Clifford suppose If I have to ignore the number as well then what will be the pattern

I have tried

 topics = [
    "+/+/0.ReportCumKpi/+",
    "+/+/0.ReportNonCumKpi/+",
  ]

but it’s giving the error.

when I tried this no error is coming but telegraf not able to read the data

topics = [
    "+/+/0.ReportCumKpi/#",
    "+/+/0.ReportNonCumKpi/#",
  ]

Can you please help me with it?

5

Hi @Ravikant_Gautam,
Based on your topic names this is not possible. Topic hierarchy is defined by / . In your case . does not confirm to the MQTT specification that I know of. You would have to look at post-processing to filter. Perhaps through the Starlark processor plugin.