Incremental backup: how to add tags?

[d3banjan]

Continuing the discussion from Restore incremental backups v1.5 OSS:

@aanthony1243

The path you need to take is:

1. backup db ‘dbname’ as you already have
2. import ‘dbname’ but to a new database name:
   influxd restore -portable -db "dbname" -newdb "dbname_tmp" ./new_backup/
3. Use a select query to side-load the data into the existing ‘dbname’:
   use dbname_tmp; SELECT * INTO dbname..:MEASUREMENT FROM /.*/ GROUP BY *

As an aside, you may want to experiment with -start/-end instead of -since. -since will take backups at the file-level, and occasionally cleanup routines will cause old data to appear new at the file level. -start and -end, new in version 1.5, can be used to filter data based on the timestamps on the actual data points. The method you use depends entirely on your use-case.

I found the above recipe when looking for a scheme for incremental backups. For the most part it made sense to me as a relative newcomer.

It also makes sense to our use case because we have to perform incremental backups from a lot of installations to a central one - and this way forced us to let the client nodes do a backup but not write it to the central database - it feels like it should be more secure that way - the central database can load the backup safely.

However, we need to add an additional tag while this sideloading is done. To me, this line SELECT * INTO dbname..:MEASUREMENT FROM /.*/ GROUP BY * makes no syntactic sense - so I couldn’t even figure out to add a tag (denoting which client it was picked up from).

It would be great if someone can link to the part of the documentation where this syntax is explained (I have failed till now) – and maybe suggest how to add a tag.

[Anaisdg]

Hello @d3banjan,

I believe the documentation you’re looking for is here and here.

" The backreference syntax ( :MEASUREMENT ) maintains the source measurement names in the destination database.

INTO <database_name>.<retention_policy_name>.:MEASUREMENT FROM /<regular_expression>/ Writes data to all measurements in the user-specified database and retention policy that match the regular expression in the FROM clause. :MEASUREMENT is a backreference to each measurement matched in the FROM clause.

Regular expressions are surrounded by / characters and use Golang’s regular expression syntax.

To add a tag you could group by a tag_key:

SELECT <function[s]> INTO <destination_measurement> FROM <measurement> [WHERE <stuff>] GROUP BY time(<interval>)[,<tag_key[s]>]

Does this help?

Thanks,

Anais