I am ingesting data from Kafka into telegraf. And my output data is going into InfluxDB. I want to find optimum values for my telegraf config:
This is my current configuration. How can I monitor my input and output writes ?Any suggestions on how to configure to reduce memory consumption and avoid buffer overflow?
There are plugins for that as well, if you want to monitor Telegraf check the Input Internal.
Is extremely easy and allows you to monitor lots of useful stuff with a bunch of query:
- points frequency (in-out)
- RAM consumption
- buffered points
- buffer limit
- and even more, just explore a bit the data
Thanks for replying! One more query , these values will change with changes in config right ? Like max_undelivered messages or input interval
“kind of yes” is the best answer I can give you.
The input will monitor Telegraf itself, using the default gathering interval.
Some metrics will be cumulative, like the data points gathered/written (for which you want to use the non_negative_difference and derivative), a small interval just means more data points and more precision in your monitoring.
How easy is it to see the difference made in the config (like max_undelivered) is a totally different topic… it’s all up to how much data you have, at which frequency you gather them, how you visualize them, etc…
here is a screenshot of my current dashboard, so you get an idea of some of the metrics you get… there are several metrics I don’t consider
Hey , could u explain me how did you create this dashboard? It looks really efficient! If you could share how the config files look, it would be much helpful.
From the telegraf side, it’s just the input.internal… which has almost no config.
for the dashboard itself, I might clean it up a little and add it to the Grafana community dashboards.
ill’ edit or reply again once I’ve done that
2 Likes
Hello, I really need your dashboard. Can you share it? Thank you!
