Influxdb v2 old data with new retention policy

I have an influxdb v2 instance with multiple buckets where:

  • Each bucket has a set of measurements, ideally the a measurement from bucket A will not be repeated in bucket B (or/and any other buckets) [*1]
  • Telegraf is feeding data to all the buckets.
  • For each bucket there is a telegraf configuration file with the different measurements.
  • Initially my data was basically “repeated” in all the buckets [*1], because I was not using any filtering for the telegraf config. I fixed it with name_override for the inputs and namepass for the outputs.

I would like for measurements/tags from bucket A that appear in bucket B to gradually disappear from bucket B [*2] (mainly for the web ui better interactivity).

  • These buckets had an initial retention policy of unknown days (unknown, but big)
  • to speed up the “disappearing” [*2] I changed the retention policy to 1 hour with shard group duration of 5 min.
  • I see that for most buckets measurements/tags that do no not belong to a bucket have disappeared, but one bucket is stubborn (let call it bucket C).

For bucket C the newly added data by telegraf that spills out of the new retention policy (1h) is being deleted. In this case I end up with data from more than X days ago, a big space of no data and data from the last hour.

After some research I think that it can be because of shards expiry date mentioned here.
I do not think is the exactly the same cause, since the issue described there was most probably a influxdb v1 issue and I am using influxdb v2.

Am I on the right path? Is there any commands like “SHOW SHARDS” for flux to get shards details?


P.S. 1: It seems that data from bucket C is being delete, but slowly.
P.S. 2: Judging from bucket C experience, I do not think that the disappearing from all other buckets has something to do with the new retention policy of 1h and shard group duration of 5 min.

Forgot to mentioned, influxdb versions used (info from docker container logs):

version=2.0.7 commit=2a45f0c037 build_date=2021-06-04T19:17:40Z