InfluxDB v2 remote backup and restore

InfluxDB 2
1

Hello influxdb community!

We have a production influxdb host, and a QA influxdb host.
Both hosts have the same admin’s token.

We would like to find a way to somehow refresh our QA host’s data with the production host’s data by creating a remote backup/restore process.

TL;DR: backup works, restore fails with “413 Request Entity Too Large” error.

First step, I backup the production remote host (this is actually working):

# influx backup --host <PRODUCTION_HOST_URL> -o future -t <ADMIN_TOKEN> backup_directory
# ls -l backup_directory
total 1740
-rw------- 1 root root     344 Sep 22 08:34 20230922T083424Z.1.tar.gz
-rw------- 1 root root   17913 Sep 22 08:34 20230922T083424Z.2.tar.gz
-rw------- 1 root root    3652 Sep 22 08:34 20230922T083424Z.3.tar.gz
-rw------- 1 root root   15522 Sep 22 08:34 20230922T083424Z.4.tar.gz
-rw------- 1 root root 1107643 Sep 22 08:34 20230922T083424Z.5.tar.gz
-rw------- 1 root root  608236 Sep 22 08:34 20230922T083424Z.6.tar.gz
-rw------- 1 root root    3798 Sep 22 08:34 20230922T083424Z.bolt.gz
-rw------- 1 root root    4283 Sep 22 08:34 20230922T083424Z.manifest
-rw------- 1 root root    3451 Sep 22 08:34 20230922T083424Z.sqlite.gz

Now when I try to restore these files onto my remote QA host, this fails:

# influx restore --host <QA_HOST_URL> -t <ADMIN_TOKEN> --full .
2023/09/22 08:35:27 INFO: Restoring KV snapshot
2023/09/22 08:35:28 INFO: Restoring SQL snapshot
2023/09/22 08:35:28 INFO: Restoring TSM snapshot for shard 1
2023/09/22 08:35:28 INFO: Restoring TSM snapshot for shard 2
2023/09/22 08:35:28 INFO: Restoring TSM snapshot for shard 3
2023/09/22 08:35:29 INFO: Restoring TSM snapshot for shard 4
2023/09/22 08:35:29 INFO: Restoring TSM snapshot for shard 5
Error: failed to restore TSM snapshot for shard 5: 413 Request Entity Too Large: unable to decode response content type "text/html"

I think this is somehow related to the remote access (by using the --host flag) as if i try to restore a local influxdb this works without any issue.

Do you guys have an idea to deal with this issue?

Thanks a lot for your help!

Max

2

I think this issue is not related to influxdb.

Our influxdb are hosted in kubernetes, and this error is related to its ingress’ nginx blocking the file’s upload as it reaches a limit.

Thanks for your help, I’ll reach my SRE team :slight_smile:

Max

1 Like
3

Hello @max22 thank you for sharing your progress and solution.