Hi Manoj here,
- Retain data for 2 years, with one minute interval.
- After 2 years, data should be down sampled to 5 minute interval.
- One minute interval data should be deleted.
We have come across multiple example of down sampling and retention policy. But we need to achieve this, without applying retention policy on individual measurements and don’t want to down sample data in different measurement, rather in same measurement.
How this can be achieved by batch task in kapacitor ?
Any help shall be appreciated.
I think this presentation may help you: DOWNSAMPLING DATA
I have only done with with continuous queries in InfluxDB directly. Influx has a regex syntax from Go and other shorthand to the same downsampling to all measurements in a retention policy. So in the continuous query it’s something like
CREATE CONTINUOUS QUERY "cq_5min_metrics" ON t128
SELECT LAST("value") as value
INTO LongerData.:MEASUREMENT <- add into "LongerDestination" retention policy with same measurement names
FROM ShorterData./*/ <- selects all measurements in "ShorterData" retention policy
GROUP BY time(5min),* <- do 5 minute rollups, and "*" means keep tags
The Kapacitor documentation explains how to do this with a Kapacitor batch job: Kapacitor as a Continuous Query engine | Kapacitor 1.5 Documentation