Best Practice to outsource old data?

InfluxDB 2
,
1

Hi,

I am using an InfluxDB v 2.4 Instance running in a Docker Container. I have multiple Telegraf Instances writing data to this InfluxDB and some data is now over one year old. I thought about outsourcing data older than 12 months each month to another machine to limit the storage needed for the data. I would like to have all data for e.g. the month of October from one bucket in a file called “backup_exampleBucket_10_2021.something”; one file for each bucket and each month.

My first Idea was to backup a certain chunk of data, defined by start and stop time, as well as the bucket name, but in the documentation to the influx backup command, I can only define a bucket name, not time limits. I also realized that I cannot restore a backup to an Influx Instance in which the specified bucket already exists.

So in order to get my desired result, I would need a lot of additional scripting (backup the whole bucket, delete old data, then to get the data back restore the backup into a new bucket, merge the two buckets to query data). My question is therefore: Is there an other (better) way to achieve these incremental backups or do I have to do it as I described it?

Thanks in advance!

Benjamin

2

Hi @brimstone ,
So a lot of the features you are describing sadly sit within the enterprise version of InfluxDB. So a lot of this functionality you would have to build out yourself as you described. What you could do is look at setting up Edge Data replication to write data from one instance to another and have an InfluxDB with a much larger disk capacity

3

Hi @Jay_Clifford,
thanks for your answer. Sad to hear that this functionality is not available in the OSS Version of Influx, but understandable. I never heard of Edge Data Replication before, but I will look into it, maybe this can be a solution!
Anyway, thanks again for your time!

1 Like