Excessive queries from EMQX platform into Influx

1

Does anyone have any experience with the EMQX platform? It is nice because it bakes in the rules and actions to forward MQTT messages to Influx, so you no longer have the need to host Telegraf externally. Problem I’m seeing though is that the integration creates an excessive level of queries into Influx. I don’t think anything is configured wrong since it is a very simple configuration. Simply configuring the connection to Influx starts yielding almost 500 queries per hour into Influx. Configuring a rule to actually send data into Influx doubles that. So, I’m getting essentially 1000 queries per hour for a basic integration. The math yields $2.88 in query charges per day for a basic integration.

I already have an open ticket with EMQX, but I’m wondering if anyone is experiencing this because it doesn’t seem correct and is leaning me back towards my Telegraf implementation with HiveMQ.

2

Here are some screen shots of the usage. I disabled the EMQX connector, and the usage went to zero. After re-enabling the connector, it jumped to 500 an hour and after enabling the rule to allow data to flow it jumped to almost 1000 per hour. Incidentally in my tests I am getting two MQTT messages every fifteen minutes, so it is a low volume of data.

image
image2209×1237 140 KB

3

Helllo @sambabic,
Thanks for opening a ticket with them as well. I don’t have any experience there though. Hopefully someone else will here.

4

I am expierencing same issue with more queries that ussuall and it is costing me around 2 dollaras a day, but it is not that so much data is flowing. Ant this change is due changing from telegraf to EMQX native rule engine.

I had setuped 2 rules to to write to 2 different buckets to same organization in influxdb cloud. One rule was writing messages constantly, another is was prepared, but messages was not there, because of sensors was turned off. At that time i was getting around 1000 queries an hour…

image
image1601×482 26.5 KB

I removed one bucket and access right to it before disabling rule in EMQX and i saw constant drop to around 500 queries.

Currently i changed some setting in EMQX InfluxDB connector just to see how it will react in query count. I changed:

  • Health check interval to 1 hour instead of 15s, to my calculation that should take of ~250 queries, because each health check is a single query
  • reduced buffered queue size to 250MB, I do not think I need that much
  • Inflight window set to 1 as it its recomended in EMQX when query mode is Async.

I will post and update how this is changing my query count.