Using influx_inspect export for the first time, have 2 databases that I want to dump. Thought I could dump all databases by not using the --database options, but it hits _internal/monitor and then builds a dump big enough to fill all available disk space.
Using Influx 1.1.0.
My /var/lib/influxdb is less than 150MB, but after almost crashing my / volume on my development/test VM I mounted a 100GB filesytem under /dump and tried it again, it got to 85GB before I killed it…
[root@vm-centos-72-mls ~]# systemctl stop influxdb.service
[root@vm-centos-72-mls ~]# influx_inspect export -datadir /var/lib/influxdb/data -waldir /var/lib/influxdb/wal -out /dump/influx.dump
writing out tsm file data for performance_metrics/rp_7d...complete.
writing out wal file data for performance_metrics/rp_7d...complete.
writing out tsm file data for _internal/monitor...^C
And that’s as far as it got, /dump/influx.dump was 85GB and still growing…
@johndburk Can you upgrade to the 1.2.4 release? We have merged some fixes to backup and restore since the 1.1.0 release.
It’s a production server which I’m trying to migrate off of, would like to be able to skip the upgrade if at all possible.
If I manually re-create the users, create empty db’s, then add the retention polices and CQ’s, can I just use the --database arg to dump each DB separately? I have scripts which create the db’s, RP’s, and CQ’s.
1 Like
@johndburk Thats what I would do.
Sometimes the old-fashioned ways are the best.
1 Like
Got my db’s migrated the old-school way, so I’m good to go in that respect.
Then went back to my development host I had running the same version as my original production server (1.1.0) and upgraded it to 1.2.4-1
Same thing: chokes on _internal/monitor, will run and spew to the out file forever. Is there a way for me to file a bug report?
[root@vm-centos-72-mls influxdb]# influx_inspect export -datadir /var/lib/influxdb/data -waldir /var/lib/influxdb/wal -out /dump/influx.dump
writing out tsm file data for _internal/monitor...^C
[root@vm-centos-72-mls influxdb]# influx_inspect export -datadir /var/lib/influxdb/data -waldir /var/lib/influxdb/wal -out /dump/metered_licensing.dump -database metered_licensing
writing out tsm file data for metered_licensing/rp_infinite...complete.
writing out wal file data for metered_licensing/rp_infinite...complete.
writing out tsm file data for metered_licensing/rp_24w...complete.
writing out wal file data for metered_licensing/rp_24w...complete.
writing out tsm file data for metered_licensing/rp_72w...complete.
writing out wal file data for metered_licensing/rp_72w...complete.
[root@vm-centos-72-mls influxdb]# influx_inspect export -datadir /var/lib/influxdb/data -waldir /var/lib/influxdb/wal -out /dump/perf_metrics.dump -database performance_metrics
writing out tsm file data for performance_metrics/rp_30d...complete.
writing out wal file data for performance_metrics/rp_30d...complete.
writing out tsm file data for performance_metrics/rp_7d...complete.
writing out wal file data for performance_metrics/rp_7d...complete.
[jburk@vm-centos-72-mls ~]$ du -sh /dump/*
51G /dump/influx.dump
2.5M /dump/metered_licensing.dump
1.6M /dump/perf_metrics.dump
last few lines of the 50GB dump look like:
tsm1_wal,database=metered_licensing,engine=tsm1,hostname=vm-centos-72-mls,id=3815,path=/var/lib/influxdb/data/metered_licensing/rp_72w/3815,retentionPolicy=rp_72w,walPath=/var/lib/influxdb/wal/metered_licensing/rp_72w/3815 currentSegmentDiskBytes=0i 1497500000000000000
tsm1_wal,database=metered_licensing,engine=tsm1,hostname=vm-centos-72-mls,id=3815,path=/var/lib/influxdb/data/metered_licensing/rp_72w/3815,retentionPolicy=rp_72w,walPath=/var/lib/influxdb/wal/metered_licensing/rp_72w/3815 currentSegmentDiskBytes=0i 1497500010000000000
tsm1_wal,database=metered_licensing,engine=tsm1,hostname=vm-centos-72-mls,id=3815,path=/var/lib/influxdb/data/metered_licensing/rp_72w/3815,retentionPolicy=rp_72w,walPath=/var/lib/influxdb/wal/metered_licensing/rp_72w/3815 currentSegmentDiskBytes=0i 1497500020000000000
@johndburk That would be awesome and much appreciated! Please open an issue on InfluxDB and reference this post.
After:
- uninstalling influxdb
- deleting /var/lib/influxdb
- re-installing 1.2.4.-1
- re-running my db creation scripts
- adding a few series
I’m unable to reproduce this issue. Export completes in under a second, and writes out a 3M uncompressed export.
The database had originally been 1.0, then 1.1.0, then eventually 1.2.4-1, must have been an artifact from 1.0 or upgrading from 1.0 → 1.1.0.
I’ve updated and closed influx_inspect export without -database arg runs forever, fills all available disk space · Issue #8516 · influxdata/influxdb · GitHub as well.
1 Like
@johndburk Thank you for doing the work there! It is very much appreciated. Also always good to have bug reports like this kicking around incase anyone runs into similar.
