I have telegraf running on few hundred servers. The data is getting large. I collect data every 1 minute. I would like to downsample data before 30 days to 15 minutes. I am looking on search engines to find a good example of such a workflow but have been unsuccesful.
You need to combine continuous queries with retention policies.
You set the lifetime (= retention time) of your default RP to 15 days and add a continuous query that aggregates the values from that RP and stores the aggregates (via SELECT INTO) into another RP that has a longer or infinite lifetime.
If you don’t want to lose your current data
- create the second retention policy
- create the continuous query
-> the CQ will only act on new data
- use SELECT INTO (and GROUP BY) to aggregate the historical 1m values to 15m values from the default RP into the new RP
- set the retention time of the default RP to 15 days.
The moment you do step four, all data older than 15 days will be deleted from that RP.