I’ve got an old database with a lot of data. She used the autogen with infinite retention.
I’ve created a new retention on it, with 6 months retention, used by default.
Now, I saw all my old data are not visible. According the documentation, it seems normal and I can get my old data modifying my requests.
I can forget these old datas, so OK, not a problem.
But, how can I be sure my old data from this previous retention autogen is removed?
On the influxdb.conf, the retention is enabled with interval 30m, but after one days no space freed on my server.
@katy On a related note, if I set my retention policy as inf with default shard group duration as 7days, I see that there is an expiry time for every shard created. Does this mean my data will get deleted if the shard expires? If so, how can I avoid this as I’d like to store data indefinitely
I’m new to influx and any help would be appreciated
Thank you
The 7-day default splits the retention policy into 7-day windows. The expiry time is the time when the data is actually deleted.
The database will only drop whole shards at a time. If you use the influx cli to run SHOW SHARDS you should see a listing of your database with their respective shards. Each of those shards has an expiry_time . Each check-interval the database will drop any of the shards who’s expiry times are older than time.Now() .
@katy I understand that. But what if I don’t want my shards to be deleted. I want to keep data infinitely. I’ve set retention policy as inf yet my shards have an expiry after 7 days. Will the shard get deleted at the end of the week? If so, how can I avoid that?
Thank you
@katy, in case of the default retention policy “autogen”, on running SHOW SHARDS, I see that this,
start_time |end_time |expiry_time
2019-02-11T00:00:00Z |2019-02-18T00:00:00Z |2019-02-18T00:00:00Z
The shard is still there even though the expiry time has passed.
Also, if the policy is autogen, what is the significance of the same end_time and expiry_time??
That post says :
“If your retention policy is on autogen (or infinite), the data is not actually stored infinitely—this just means the retention policy matches the shard group duration, so the retention policy is effectively disabled.”
From the answer of sm97 : “The shard is still there even though the expiry time has passed.”
I would conclude that you are right and that the post is wrong …
@MarcV The vocabulary around retention policies and shards is confusing. I think what the blog post is saying is that although it doesn’t explicitly say infinite, the effect is the same. So the default retention policy will keep your data indefinitely, but if you change the shard duration or RP duration, it can cause some different behavior.
,
