Down sample data in same measurement and delete old data

1

Hi Manoj here,

Our requirement,

  • Retain data for 2 years, with one minute interval.
  • After 2 years, data should be down sampled to 5 minute interval.
  • One minute interval data should be deleted.

We have come across multiple example of down sampling and retention policy. But we need to achieve this, without applying retention policy on individual measurements and don’t want to down sample data in different measurement, rather in same measurement.

How this can be achieved by batch task in kapacitor ?

Any help shall be appreciated.

Thanks ,

Manoj.

2

I think this presentation may help you: DOWNSAMPLING DATA

I have only done with with continuous queries in InfluxDB directly. Influx has a regex syntax from Go and other shorthand to the same downsampling to all measurements in a retention policy. So in the continuous query it’s something like

CREATE CONTINUOUS QUERY "cq_5min_metrics" ON t128
BEGIN
    SELECT LAST("value") as value
    INTO LongerData.:MEASUREMENT    <- add into "LongerDestination" retention policy with same measurement names
    FROM ShorterData./*/            <- selects all measurements in "ShorterData" retention policy
    GROUP BY time(5min),*           <- do 5 minute rollups, and "*" means keep tags
END

The Kapacitor documentation explains how to do this with a Kapacitor batch job: Kapacitor as a Continuous Query engine | Kapacitor 1.5 Documentation