Hey guys,
we are using telegraf with mostly the statsd plugin, all of our applicative metrics are being sent over statsd.
recently we encountered that only 2 cores are active, even if we are using machines with 24CPUs or more.
we saw this limitation is somewhere in the code of the statsd plugin.
does someone know anything about this?
it appears that it once was configurable but now its not:
it looks like it was once possible to configure max_parser_threads but its now reverted
there is also another PR:
which is a draft but also relates to the same point this time with number_workers_threads param.
can you please elaborate on all of those changes? we just need to be able to utilize statsd to the fullest on big machines.
(Also - we have opened an issue on github as well: #12432 (can’t post another link as i am a new user)
Hi @Arnold,
Just tagging @jpowers here. It looks like a design choice made by one of our previous Telegraf developers. Hopefully, Josh could shine some light if this is something we wouldn’t mind adding back in. I belive the decision was taken over performance initially.
recently we encountered that only 2 cores are active, even if we are using machines with 24CPUs or more.
Recall that a goroutine, which those settings in PRs you mentioned control, is not the same as thread. Befor diving to far into these, are you finding issues where Telegraf is not actually able to keep up and is dropping metrics?
Yes, Telegraf is not able to keep up and is dropping a lot of metrics.
When looking at Pod utiliziation only 2 cores are being used, a teamember in the past was able to pinpoint exactly where this is hardcoded but now i can’t find it in telegraf repo
(also might be in a dependecy telegraf uses for statsd for example)
so we are in a situation that we can’t scale telegraf as it will always utilize 2 cores (we are talking about statsd input only)
