MQTT plugin not writing data to influxdb

Telegraf
1

Hello all! I’m really new to all this so I imagine I may have a few errors here. I am using influxdb v1.8 and telegraf v1.23.

I have two instances of the MQTT plugin, first one for an int data type and the second for json data type. The first instance works fine and writes to my database into the table named mqtt_consumer.

I’m trying to get the second instance to write to a new table called mqtt_sensors for the values received from the MQTT broker. Any help is very much appreciated. Below is my config for the MQTT inputs.

[[inputs.mqtt_consumer]]
   servers = ["tcp://192.168.50.36:1883"]
   topics = [
     "perfscore"
   ]
   topic_tag = "topic"
   qos = 0
   connection_timeout = "30s"
   max_undelivered_messages = 1000
   persistent_session = false
   data_format = "value"
   data_type = "integer"

[[inputs.mqtt_consumer]]
   name_override = "mqtt_sensors"
   servers = ["tcp://192.168.50.36:1887"]
   topics = [
     "meraki/v1/mt/+/ble/+/temperature",
     "meraki/v1/mt/+/ble/+/humidity",
     "meraki/v1/mt/+/ble/+/batteryPercentage"
   ]

   topic_tag = "sensors"
   qos = 0
   connection_timeout = "30s"
   max_undelivered_messages = 1000
   persistent_session = false
   data_format = "json_v2"
   [[inputs.mqtt_consumer.topic_parsing]]
     topic = "+/+/+/+/+/+/temperature"
     tags = "_/_/_/network/_/device/_"
	 [inputs.mqtt_consumer.json_v2]
	   [inputs.mqtt_consumer.json_v2.fields]
	     fahrenheit = "float"
		 celcius = "float"
   [[inputs.mqtt_consumer.topic_parsing]]
     topic = "+/+/+/+/+/+/humidity"
     tags = "_/_/_/network/_/device/_"
	 [inputs.mqtt_consumer.json_v2]
	   [inputs.mqtt_consumer.json_v2.fields]
	   humidity = "float"

The payload for the three topics in the second instance are as follows:
Topic: temperature
Payload: {“ts”:“2021-08-12T17:44:46Z”, “fahrenheit”:72.6, “celsius”:22.6}
Topic: humidity
Payload: {“ts”:“2021-08-12T17:44:46Z”, “humidity”:62}
Topic: batteryPercentage
Payload: {“ts”:“2021-08-12T21:30:48Z”, “battery percentage”:100}

2

Hi @jmo - I’m fairly new as well, but it may be that two occurrences of the [[inputs.mqtt_consumer]] block in the same telegraf.conf file is problematic. It sounds like the first is working and the second is being ignored (i.e., no activity or output). Clearly you understand the plug-in’s usage well enough because the first one is working.

I’d suggest setting debug = true in the agent section and temporarily removing the first one in order to focus on the second. Once you get that working, put the first one back in but after the JSON version and see what happens.

3

Thanks @phill for the recommendation. I have enabled debug and commented out the first [[inputs.mqtt_consumer]] instance. From the logs I see it connecting to the broker for the second instance but thereafter I only see writes to the database. I also am using SNMP so I assume those writes are for that particular data. No change in the database schema to reflect any new table.

Log output:

2022-07-04T00:35:36Z D! [agent] Connecting outputs
2022-07-04T00:35:36Z D! [agent] Attempting connection to [outputs.influxdb]
2022-07-04T00:35:36Z D! [agent] Successfully connected to outputs.influxdb
2022-07-04T00:35:36Z D! [agent] Starting service inputs
2022-07-04T00:35:36Z I! [inputs.mqtt_consumer] Connected [tcp://192.168.50.36:1887]
2022-07-04T00:35:46Z D! [outputs.influxdb] Wrote batch of 60 metrics in 5.281927ms
2022-07-04T00:35:46Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2022-07-04T00:35:56Z D! [outputs.influxdb] Wrote batch of 77 metrics in 5.776582ms
2022-07-04T00:35:56Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

Database output:

> show measurements
name: measurements
name
----
cpu
disk
diskio
kernel
mem
mqtt_consumer
mqtt_perfscore
processes
snmp
swap
system
4

Hi @jmo @phill,
So it is more than acceptable to have two or even more of the same plugin type within your config. In some cases we even encourage it to spread the load with large datasets. Can I propose a config reshuffle:

[[inputs.mqtt_consumer]]
   servers = ["tcp://192.168.50.36:1883"]
   topics = [
     "perfscore"
   ]
   topic_tag = "topic"
   qos = 0
   connection_timeout = "30s"
   max_undelivered_messages = 1000
   persistent_session = false
   data_format = "value"
   data_type = "integer"


[[inputs.mqtt_consumer]]
   name_override = "mqtt_sensors"
   servers = ["tcp://192.168.50.36:1887"]
   topics = [
     "meraki/v1/mt/+/ble/+/temperature",
     "meraki/v1/mt/+/ble/+/humidity",
     "meraki/v1/mt/+/ble/+/batteryPercentage"
   ]

   topic_tag = "sensors"
   qos = 0
   connection_timeout = "30s"
   max_undelivered_messages = 1000
   persistent_session = false
   data_format = "json_v2"

   [[inputs.mqtt_consumer.topic_parsing]]
     topic = "+/+/+/+/+/+/+"
     tags = "_/_/_/network/_/device/_"

 [[inputs.mqtt_consumer.json_v2]]
 [[inputs.mqtt_consumer.json_v2.object]]
path = "@this"
3 Likes
5

Thank you @Jay_Clifford! I now see data in Influx with your proposed change. Much appreciate the help from the community here.

1 Like
6

No worries @jmo ! Happy to help anytime :slight_smile: