Get identifier rejected error while using outputs.mqtt(Mosquitto version is 2.0.12)

Hi ,

I face one issue with outputs.mqtt plugin.

Step 1: Clone the source code from https://github.com/influxdata/telegraf.git and git checkout the tag v1.19.3. I use make to get my own telegraf daemon.

Step 2: docker pull eclipse-mosquitto:1.5

Step 3: docker run -d --name mqttserver eclipse-mosquitto:1.5

Step 4; docker inspect mqttserver and get the container ip.

Step 5: Revise telegraf.conf. Add the above ip to configuration.

Step 6: ./telegraf -config /etc/telegraf/telegraf.conf --debug true, and the result is working well.

Step 7: stop the telegraf, and docker stop mqttserver && docker rm mqttserver.

Step 8: docker run -d --name mqttserver2 eclipse-mosquitto:2.0.12.

Step 9: Since mqttserver2 only allows localhost connection, we have to revise configuration file.
Add the following into /mosquitto/config/mosquitto.conf
listener 1883
allow_anonymous true

Step 10: docker restart mqttserver2

Step 11: docker inspect mqttserver2 to get container ip.

Step 12: Revise telegraf.conf. Add the above ip to configuration.

Step 13: ./telegraf -config /etc/telegraf/telegraf.conf --debug true, and you will see
error was ‘identifier rejected’

The workaround is not using mosquitto2.0.12 or revert this solution https://github.com/influxdata/telegraf/commit/8c932abff64c7a905e58ab708502ebf54c52b6bd#diff-e6317aff44794ed4968727d91c3297c19edc570f5590e6c6e029d231fae4866a

Does anyone have comment for this?

Thanks

Hi Ruke,
I hope you are doing well. So if I understand your use-case a little further you are spinning up a mosquito server on an isolated docker network? i.e. not binding ports to host / running in network mode host? Are you also building telegraf as a docker container in this isolated docker network or running this at a host level. My advise would be to bind the 1883 port to your host network like so:
docker run -d --name mqttserver2 -p 1883:1883 eclipse-mosquitto:2.0.12

Then you will be able to use http://localhost:1883 as shown within the telegraf docs. If there is a requirement for isolating the MQTT server within a docker network let me know.

Hi @Jay_Clifford

For mqtt:

docker run -d --name mqttserver2 eclipse-mosquitto:2.0.12.

docker exec -ti mqttserver2 sh
/ # ip ro
default via 10.0.5.1 dev eth0
10.0.5.0/25 dev eth0 scope link  src 10.0.5.4
/ #

For telegraf:

It’s another alpine container which also runs on the same host.

telegraf.conf is something like the following

 [[outputs.mqtt]]
   servers = ["10.0.5.4:1883"] # required.

Since all container are binding to the some bridge, I can ping mqttserver2 from my telegraf container.

As you can see, if I connect to mosquitto 1.5, I will see the following log

./telegraf -config /etc/telegraf/telegraf.conf --debug true
2021-09-15T01:33:53Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2021-09-15T01:34:03Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2021-09-15T01:34:13Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2021-09-15T01:34:23Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2021-09-15T01:34:33Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2021-09-15T01:34:43Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics
2021-09-15T01:34:53Z D! [outputs.mqtt] Buffer fullness: 0 / 10000 metrics

From my mqttserver, I can see the following log.

1631670208: Client Telegraf-Output-uzAAh disconnected.
1631670253: New connection from 10.0.5.2 on port 1883.
1631670253: New client connected from 10.0.5.2 as Telegraf-Output-krVXp (c1, k30).
1631670253: No will message specified.
1631670253: Sending CONNACK to Telegraf-Output-krVXp (0, 0)
1631670283: Received PINGREQ from Telegraf-Output-krVXp

The behavior with mosquitto 1.5 is working well, but is not working for mosquitto 2.0.12.
Thanks

Hi Ruke,
I managed to reproduce your issue this morning with a simuler setup nice catch. I used Node-Red as a secondary tool to test connectivity which appears to work fine. Have you submitted an issue on the Telegraf GitHub yet? https://github.com/influxdata/telegraf/issues?q=is%3Aissue+is%3Aopen+mqtt
If not we can add your issue there and one of the Telegraf team can take a look.

Do you receive the same issue when using 2.0.12 with authentication?

Thanks,
Jay

Hi @Jay_Clifford

No, I have not posted this issue on Telegraf Github, but I see a similar discussion on Mosquitto.https://github.com/eclipse/mosquitto/issues/2309.

Do you receive the same issue when using 2.0.12 with authentication?
Ans: I have tried using tls, and got the same issue ‘identifier rejected’. I will try using authentication mechanism and update.

Thanks

By the way, if I mark the code from telegraf plugins/outputs/mqtt/mqtt.go.

opts.KeepAlive = 0

The result is good for both mosquitto 1.5 and 2.0.12.

Hi Ruke,
I have reached out to one of our Telegraf team members to see if they have any ideas. I have seen an issue like this in the past where the rejection was due to a bad clientID but this seems to not be the case. I think next steps will be to turn this into an issue.

Hi @Ruke_Chen,

Thanks for the detailed issue! If you could open a related issue in Telegraf that would be great.

I think it should be a simple fix based on the information you’ve provided around the opts.KeepAlive setting we have changed. If you were interested in submitting a PR that would also be welcomed. If not someone from the team will take a look.

1 Like

Hi @helenosheaa

Sure I will open an issue on Telegraf GitHub. Since opts.KeepAlive is related to this commit https://github.com/influxdata/telegraf/commit/8c932abff64c7a905e58ab708502ebf54c52b6bd#diff-e6317aff44794ed4968727d91c3297c19edc570f5590e6c6e029d231fae4866a , I have no idea how this affect . I think I open an issue first, and check the feedback.

Thanks

Hi all,

A related issue is created earlier . https://github.com/influxdata/telegraf/issues/9763. So I add my comment in this post.

Thanks