Telegraf: Write MQTT Message instant(!) to InfluxDB

1

Hi there,

I am new with Telegraf and InfluxDB but I think I understood the basic concepts of both.

My problem is now, that I have to write MQTT messages containing metrics instant to influxdb.
The default interval and flush_interval is 10s both, while jitter is 0s, so in the worst case, data is written 10s after reception. I tuned both intervalls to 10ms and now it almost ‘feels’ like real time, but CPU usage gets rather high and with every flush i see the warning “Skipping a scheduled flush because there is already a flush ongoing.” multiple times.

I have to write the data as fast as possible to influxdb!
Is there a way to configure telegraf so that it doesn’t follow the collect and flush principle so much but even more the way MQTT works?

I am thinking about throwing away MQTT and writing my data directly via http to influxdb, but http as a protocol has so many disadvantages compared to MQTT in this use case.

BTW: What happened to the integration of SurgeMQ?

Many Thanks!

2

@stenub If you need to reduce latency to InfluxDB I would definitely advise writing directly to the database. Telegraf is not a streaming engine so you will need to work with what flush_interval offers. I would advise setting that to a minimum of 100ms.

3

Thank you! I solved my problem by writing via udp directly to influxdb! I used this guide to do so:

EDIT: I just recognized that udp listener is also working in a batched way. :frowning:
Trying my luck with blown up http instead…

1 Like
4

@stenub Having any issues with the HTTP interface?