Consultation on Using Separate Retention Policies per Organization/Tenant in InfluxDB 1.x

InfluxDB 1
1

Hello InfluxData Team,

We are using an InfluxDB 1.x SaaS cluster in our production environment. Currently we have about 1.8 TB of time-series data. Because of earlier design decisions, all data is stored in a single database, in a single measurement, under the default retention policy, which makes it impossible for us to apply different expiration strategies for different groups of data.

We currently have around 50 organizations (tenants), and this number will continue to grow steadily.
Our goal is:

  • Each organization should have its own retention duration.

  • We should be able to change the retention time for each organization independently.

  • Therefore, we are considering creating one RP per organization.

However, from the community discussions we found that InfluxDB recommends keeping retention policies below ~20 for performance and management reasons.
If we create 50 RPs now, and even more in the future, I’m concerned we may run into performance issues.

Therefore, I’d like to get clarification on the following questions:

1. What is the maximum number of retention policies InfluxDB 1.x can safely support?

Has the InfluxData team conducted internal benchmarks or stress tests on RP scalability?
Would having 50+ RPs (and growing) cause performance degradation or operational issues?

2. If we cannot exceed around 20 RPs, then multiple organizations must share the same RP.

In this case, when one organization needs to change its retention time, we would need to migrate only that organization’s data to a different RP.

  • Does InfluxDB provide any official or recommended approach for partial migration of data between retention policies?

  • Are there tools, best practices, or known patterns for migrating only a subset of a measurement’s data (e.g. filtering by tag such as organization ID)?

3. Based on typical multi-tenant or multi-organization scenarios, what does InfluxData recommend for retention policy design?

Is the “one RP per organization” pattern recommended, discouraged, or should we consider redesigning the database/measurement structure (e.g., multiple databases, separate measurements, or other strategies)?

We want to design a structure that will scale as the number of organizations increases and that remains compatible with InfluxDB 1.x SaaS deployments.

Thank you very much for your guidance.

2

@davidby-influx might be able to offer some key insight here.

3

Since you are a paying customer, please file a ticket in the Support portal. That will let us discuss your situation privately, and with greater ease for our internal team. It will also allow us to examine your installation dashboards and performance directly.

Let me know here if you have any difficulties filing a ticket.

1 Like