what is the retention policy and how exactly it work?
as you know we can set retention policy “check-interval” on config file of influx.
what happen if i set this parameter and after that meet interval, which part of database remove?
what happen to the old data? i mean if i have database of 1 year ago retention will remove it ?
@Indeed_1 The database will only drop whole shards at a time. If you use the
influx cli to run
SHOW SHARDS you should see a listing of your database with their respective shards. Each of those shards has an
check-interval the database will drop any of the shards who’s expiry times are older than
Does that help?
@jackzampolin thank you Jack,
i check my config file , in shard section write this
enabled = true
check-interval = "10m"
advance-period = “30m”
what is mean? each 10 minutes my shard will be remove? but when i enter “SHOW SHARDS” seems all shards remain! would you please clarify this happen?
@Indeed_1 That section is for
shard-precreation. It will create empty shard files ahead of time. The section that deals with retentions (and the defaults) looks as follows:
enabled = true
check-interval = "30m0s"
This means the database will check every 30 minutes to see if the
shard.expiry_time < now(). Also it will only drop shards if there is a
duration > 0d. By the default retention policy that gets created with each database has infinite retention. You can check this with
SHOW RETENTION POLICIES ON "mydb".
@jackzampolin Thank you dear Jack,
I’ve create new RP but now I have two RP :
name duration shardGroupDuration replicaN default
autogen 0s 168h0m0s 1 true
myRP 1h0m0s 1h0m0s 1 false
which will be run? any priority ?
should i remove autogen till myRP work correctly?
another issue after RP run , when i “select count” of tables seems old data still exist ! is that index or something like that?
@Indeed_1 I suggest you check out this documentation on retention policies. That should help clear up some of your questions.
When I apply a new default retention policy to an existing DB I seem to lose access to any data already collected. SHOW SHARDS shows that a new shard with the new retention policy was created, but the old shard still exists. However, queries seem to be answered from the new shard which is empty. Is this the expected behaviour?
@Bill_Gunter You can query the old shard you just need to specify the retention policy. When querying use the fully qualified measurement name:
"database"."retentionPolicy"."measurement". When querying without the fully qualified measurement name the db will return data from the
DEFAULT retention policy, your empty new one. This is intended behavior.
Wow, that makes complete sense. I didn’t make the connection between the retentionpolicy in a query and adding a new retentionpolicy. Thanks!
So how does retention policies map to measurements? If I have 5 RPs and 5 measurements which is applied to which?
I hope the answer over here helps.
hi. is there a way to check when the next retention policy check will run? My config says 30m, but is there a command to check ? Thanks
The closest thing you can do is run
SHOW SHARDS and then check the expiry_time for them