I have an instance of InfluxDB that is growing too much.
All the buckets retain the value indefinitely and this is the problem, I know!
I want to move from one bucket all the data older than 6 month to a new bucket that still keeps the data indefinitely, but with a lower data resolution: right now some sensors are recorded every seconds, I’d like to have all of them at about 1 record per 60 seconds.
How can I do that with a whole bucket without specifying every single field?
Thanks!
Hello @Marcx,
It depends on how much data you have.
So you’re asking to:
- Dowsample all data older than 6 mo to 1 min resolution?
- Continue to downsample new data?
The second bit should be easier.
Here is a resource for that:
Downsampling with InfluxDB v2.0 | InfluxData
Specifically you could do something like:
option task = {name: "Downsampling CPU", every: 1m}
from(bucket: "my-bucket")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) =>
(r["_measurement"] == "cpu"))
|> aggregateWindow(every: 10s, fn: mean, createEmpty: false)
// Use the to() function to validate that the results look correct. This is optional.
|> to(bucket: "downsampled", org: "my-org")
Just don’t filter for a specific field and you’ll automatically downsample every field in your measurement.
You can also query all of the data in a bucket with the following (if you don’t want to specify every measurement):
from(bucket: "my-bucket")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
AS for 1. bulk downsampling is challenging. You might have to do that in chunks (depending on how much data you have). It might also be easier to do that type of logic with a client library instead and loop through time ranges.
