Retention policy duration on autogen set to 7d but data never expires

Store
1

Hello! We have a few databases in influxdb. Specific in one database we have a problem that raw data never expires.

show retention policies
name duration shardGroupDuration replicaN default

…
autogen 168h0m0s 24h0m0s 1 true
…

Any ideas what to do? Thanks.

2

Hi @sampler ,

what is the output of : show databases
how is the retention configured in the configuration file ?
something like this ?

[retention]
enabled = true
check-interval = “30m0s”

3

Output of “show databases” prints out all 6 of our databases.
Retention configuration is default (check-inteval and enabled are commented out in .conf file)

All databases are cleaned like expected according to retention period. Just this one doesn’t get trimmed as expected. Specific thing about this database is it contains about 1500 measurements.

4

Can you check the expiry time of the shards in that database with show shards ?

5

show shards
…
id database retention_policy shard_group start_time end_time expiry_time owners

…
3434 … autogen 3434 2019-02-11T00:00:00Z 2020-02-10T00:00:00Z 2020-02-17T00:00:00Z
…

6

the shard you listed will expire in 2020 …

it seems as if this database has a retention policy of 52w is that possible ?

7

Shard group for autogen was once set to 52w but a few weeks ago I explicitly set it to 1d:

alter retention policy autogen on “…” duration 7d replication 1 shard duration 1d

Also “show retention policies” shows that shard group is 24h:
autogen 168h0m0s 24h0m0s 1 true

So how come shard persists at 52w?

8

that is because the shard was created when that retention policy was 52weeks , altering the shard duration will not alter the duraton of existing shards … only for new shards …

I did a little test and altering the retention policy didn’t alter an existing shard …

9

I’m pretty sure I must have changed it this way (so increased it by using alter). Maybe it can be increased but not decreased? Maybe it can be decreased only after it expires and drops past shard - only then it’s recreated according to how rules are set?
Is this behavior documented or is it a bug?
What can we do to lower the shard duration on existing database full of data?

10

No solution to this issue?

11