Hello!
I have an influxdb v2 instance with multiple buckets where:
- Each bucket has a set of measurements, ideally the a measurement from bucket A will not be repeated in bucket B (or/and any other buckets) [*1]
- Telegraf is feeding data to all the buckets.
- For each bucket there is a telegraf configuration file with the different measurements.
- Initially my data was basically “repeated” in all the buckets [*1], because I was not using any filtering for the telegraf config. I fixed it with name_override for the inputs and namepass for the outputs.
I would like for measurements/tags from bucket A that appear in bucket B to gradually disappear from bucket B [*2] (mainly for the web ui better interactivity).
- These buckets had an initial retention policy of unknown days (unknown, but big)
- to speed up the “disappearing” [*2] I changed the retention policy to 1 hour with shard group duration of 5 min.
- I see that for most buckets measurements/tags that do no not belong to a bucket have disappeared, but one bucket is stubborn (let call it bucket C).
For bucket C the newly added data by telegraf that spills out of the new retention policy (1h) is being deleted. In this case I end up with data from more than X days ago, a big space of no data and data from the last hour.
After some research I think that it can be because of shards expiry date mentioned here.
I do not think is the exactly the same cause, since the issue described there was most probably a influxdb v1 issue and I am using influxdb v2.
Am I on the right path? Is there any commands like “SHOW SHARDS” for flux to get shards details?
Thanks!
P.S. 1: It seems that data from bucket C is being delete, but slowly.
P.S. 2: Judging from bucket C experience, I do not think that the disappearing from all other buckets has something to do with the new retention policy of 1h and shard group duration of 5 min.
