Metric Buffer Limit - Deleting Older Messages When Queue is Full

Telegraf
1

How difficult would it be to add an option to an output plugin to discard the older metrics to allow newer metrics to be queued, rather than simply dropping the newer metrics and logging and overflow error?

Can I just do an r,buffer.Dequeue() if r.buffer.Add(metric) failed, then re-do the Add()? Any caveats to doing this?

2

Telegraf is already dropping oldest metrics when adding new ones:

metric_buffer_limit: Maximum number of unwritten metrics per output. Increasing this value allows for longer periods of output downtime without dropping metrics at the cost of higher maximum memory usage. Oldest metrics are overwritten in favor of new ones when the buffer fills up.

3

Thanks for the pointer.

4

Why do you think the newest metric is being dropped?

5

Just a bad assumption on my part.

1 Like
6

Thanks, please mark question as answered.