I am trying to balance pipeline simplicity and data robustness. As an example, I am interested in collecting network metrics (counters et al.) at a very high right (<30seconds) and retaining that data for a short period of time (1month). However, I also want to store the same data in a less aggressive window (~5min).
The first option I cam up with was just run the collector twice, one running every 30s and one running every 5mins. Each output would either go into different buckets or with different tags.
The second option I found was adding quix streams to our pipeline to downsample the data.
I’m wondering if there is a simpler third option to have telegraf do something like this natively. Any advice is appreciated