# Shard X removed during backup

**URL:** https://community.influxdata.com/t/shard-x-removed-during-backup/24436
**Category:** InfluxDB 2
**Created:** [March 29, 2022, 7:24am UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436 "2022-03-29T07:24:25Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![sineverba](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/sineverba/32/9638_2.png) [@sineverba](https://community.influxdata.com/u/sineverba)
#### Post date: [March 29, 2022, 7:24am UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/1 "2022-03-29T07:24:25Z")

</div>

I got this error:

```auto
2022/03/29 07:20:39 INFO: Backing up TSM for shard 4
2022/03/29 07:20:39 WARN: Shard 4 removed during backup
2022/03/29 07:20:39 INFO: Backing up TSM for shard 5
2022/03/29 07:20:39 WARN: Shard 5 removed during backup
2022/03/29 07:20:39 INFO: Backing up TSM for shard 7
2022/03/29 07:20:39 WARN: Shard 7 removed during backup
2022/03/29 07:20:39 INFO: Backing up TSM for shard 8
2022/03/29 07:20:39 INFO: Backing up TSM for shard 9
2022/03/29 07:20:39 INFO: Backing up TSM for shard 10
2022/03/29 07:20:39 INFO: Backing up TSM for shard 19
2022/03/29 07:20:39 INFO: Backing up TSM for shard 3
2022/03/29 07:20:39 INFO: Backing up TSM for shard 12
2022/03/29 07:20:39 INFO: Backing up TSM for shard 2
2022/03/29 07:20:39 INFO: Backing up TSM for shard 11
2022/03/29 07:20:39 INFO: Backing up TSM for shard 1
2022/03/29 07:20:46 INFO: Backing up TSM for shard 13
2022/03/29 07:20:47 INFO: Backing up TSM for shard 18

```

Is it all ok? I’m using v 2.1 with Docker.

---

<div class="post-metadata">

### Author: ![Anaisdg](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/anaisdg/32/6401_2.png) [@Anaisdg](https://community.influxdata.com/u/Anaisdg)
#### Post date: [April 1, 2022, 5:05pm UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/2 "2022-04-01T17:05:17Z")

</div>

@sineverba was this during a restore?  
Have you seen:

> <https://github.com/influxdata/influxdb/issues/15323>
>
> \<!--
> 
> Thank you for reporting a bug in InfluxDB. 
> 
> \* Please ask usage questi…ons on the Influx Community site.
> \* https://community.influxdata.com/
> \* Please add a :+1: or comment on a similar existing bug report instead of opening a new one.
> \* https://github.com/influxdata/influxdb/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+is%3Aclosed+sort%3Aupdated-desc+label%3Akind%2Fbug+
> \* Please check whether the bug can be reproduced with the latest release.
> \* The fastest way to fix a bug is to open a Pull Request.
> \* https://github.com/influxdata/influxdb/pulls
> 
> \--\>
> 
> \_\_Steps to reproduce:\_\_
> List the minimal actions needed to reproduce the behavior.
> 
> 1. Backup database with:
> \`influxd backup -host ${INFLUXDB\_HOST} -portable -database ${DB} ${DB\_BACKUP\_DIR}\`
> 2. Upload files to the new host
> 3. Try to restore database with:
> \`influxd restore -portable /backup/f3-20191004\_1215\`
> 
> \_\_Expected behavior:\_\_
> Should restore all shards without warnings.
> 
> \_\_Actual behavior:\_\_
> A lot of messages like these:
> \`\`\`
> ts=2019-10-07T10:11:47.626290Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="engine is closed"
> ts=2019-10-07T10:11:47.626467Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=77
> ...
> ts=2019-10-07T10:12:13.487297Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=36
> ts=2019-10-07T10:12:13.490966Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="engine is closed"
> \`\`\`
> 
> \_\_Environment info:\_\_
> 
> \* System info: Run \`uname -srm\` and copy the output here
> source: \`Linux 4.19.25-coreos x86\_64\`
> target: \`Linux 4.9.184-linuxkit x86\_64\`
> \* InfluxDB version: Run \`influxd version\` and copy the output here
> source \`InfluxDB v1.7.8 (git: 1.7 ff383cdc0420217e3460dabe17db54f8557d95b6)\`
> target: \`InfluxDB v1.7.8 (git: 1.7 ff383cdc0420217e3460dabe17db54f8557d95b6)\`
> \* Other relevant environment details: Container runtime, disk info, etc
> Source is running as a docker container on c5.4xlarge AWS EC2 instance (16 CPU, 32GB RAM).
> Target is running as a docker container on a local dev laptop (3 CPUs and 12GB of RAM).
> 
> \_\_Config:\_\_
> Copy any non-default config values here or attach the full config as a gist or file.
> Source modifications:
> \`\`\`
> env:
> - name: INFLUXDB\_GRAPHITE\_ENABLED
> value: "true"
> - name: INFLUXDB\_UDP\_ENABLED
> value: "true"
> - name: INFLUXDB\_HTTP\_FLUX\_ENABLED
> value: "true"
> - name: INFLUXDB\_DATA\_INDEX\_VERSION
> value: "tsi1"
> - name: INFLUXDB\_COORDINATOR\_QUERY\_TIMEOUT
> value: "10s"
> - name: INFLUXDB\_COORDINATOR\_LOG\_QUERIES\_AFTER
> value: "10s"
> - name: INFLUXDB\_BIND\_ADDRESS
> value: "0.0.0.0:8088"
> \`\`\`
> 
> Target runs as
> 
> 
> \_\_Logs:\_\_
> \`\`\`
> ts=2019-10-07T10:10:37.326721Z lvl=info msg="InfluxDB starting" log\_id=0ILR1jZW000 version=1.7.8 branch=1.7 commit=ff383cdc0420217e3460dabe17db54f8557d95b6
> ts=2019-10-07T10:10:37.326877Z lvl=info msg="Go runtime" log\_id=0ILR1jZW000 version=go1.11 maxprocs=3
> ts=2019-10-07T10:10:37.441600Z lvl=info msg="Using data dir" log\_id=0ILR1jZW000 service=store path=/var/lib/influxdb/data
> ts=2019-10-07T10:10:37.445285Z lvl=info msg="Compaction settings" log\_id=0ILR1jZW000 service=store max\_concurrent\_compactions=1 throughput\_bytes\_per\_second=50331648 throughput\_bytes\_per\_second\_burst=50331648
> ts=2019-10-07T10:10:37.445399Z lvl=info msg="Open store (start)" log\_id=0ILR1jZW000 service=store trace\_id=0ILR1k1G000 op\_name=tsdb\_open op\_event=start
> ts=2019-10-07T10:10:37.521508Z lvl=info msg="Reading file" log\_id=0ILR1jZW000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/\_internal/monitor/1/\_00001.wal size=12927
> ts=2019-10-07T10:10:37.533550Z lvl=info msg="Opened shard" log\_id=0ILR1jZW000 service=store trace\_id=0ILR1k1G000 op\_name=tsdb\_open index\_version=inmem path=/var/lib/influxdb/data/\_internal/monitor/1 duration=29.037ms
> ts=2019-10-07T10:10:37.533646Z lvl=info msg="Open store (end)" log\_id=0ILR1jZW000 service=store trace\_id=0ILR1k1G000 op\_name=tsdb\_open op\_event=end op\_elapsed=88.251ms
> ts=2019-10-07T10:10:37.533832Z lvl=info msg="Opened service" log\_id=0ILR1jZW000 service=subscriber
> ts=2019-10-07T10:10:37.533870Z lvl=info msg="Starting monitor service" log\_id=0ILR1jZW000 service=monitor
> ts=2019-10-07T10:10:37.533880Z lvl=info msg="Registered diagnostics client" log\_id=0ILR1jZW000 service=monitor name=build
> ts=2019-10-07T10:10:37.533887Z lvl=info msg="Registered diagnostics client" log\_id=0ILR1jZW000 service=monitor name=runtime
> ts=2019-10-07T10:10:37.533921Z lvl=info msg="Registered diagnostics client" log\_id=0ILR1jZW000 service=monitor name=network
> ts=2019-10-07T10:10:37.533939Z lvl=info msg="Registered diagnostics client" log\_id=0ILR1jZW000 service=monitor name=system
> ts=2019-10-07T10:10:37.533956Z lvl=info msg="Starting precreation service" log\_id=0ILR1jZW000 service=shard-precreation check\_interval=10m advance\_period=30m
> ts=2019-10-07T10:10:37.533963Z lvl=info msg="Starting snapshot service" log\_id=0ILR1jZW000 service=snapshot
> ts=2019-10-07T10:10:37.533968Z lvl=info msg="Starting continuous query service" log\_id=0ILR1jZW000 service=continuous\_querier
> ts=2019-10-07T10:10:37.533973Z lvl=info msg="Starting HTTP service" log\_id=0ILR1jZW000 service=httpd authentication=false
> ts=2019-10-07T10:10:37.533976Z lvl=info msg="opened HTTP access log" log\_id=0ILR1jZW000 service=httpd path=stderr
> ts=2019-10-07T10:10:37.534078Z lvl=info msg="Listening on HTTP" log\_id=0ILR1jZW000 service=httpd addr=\[::\]:8086 https=false
> ts=2019-10-07T10:10:37.534093Z lvl=info msg="Starting retention policy enforcement service" log\_id=0ILR1jZW000 service=retention check\_interval=30m
> ts=2019-10-07T10:10:37.534260Z lvl=info msg="Storing statistics" log\_id=0ILR1jZW000 service=monitor db\_instance=\_internal db\_rp=monitor interval=10s
> ts=2019-10-07T10:10:37.534506Z lvl=info msg="Listening for signals" log\_id=0ILR1jZW000
> ts=2019-10-07T10:10:37.534633Z lvl=info msg="Sending usage statistics to usage.influxdata.com" log\_id=0ILR1jZW000
> ts=2019-10-07T10:11:05.632784Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/32/000000015-000000002.tsm id=0 duration=15.251ms
> ts=2019-10-07T10:11:47.626290Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="engine is closed"
> ts=2019-10-07T10:11:47.626467Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=77
> ts=2019-10-07T10:11:48.468494Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/77/000000193-000000002.tsm id=0 duration=711.420ms
> ts=2019-10-07T10:11:53.062240Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/18/000000015-000000002.tsm id=0 duration=14.591ms
> ts=2019-10-07T10:11:54.739358Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/31/000000016-000000002.tsm id=0 duration=25.957ms
> ts=2019-10-07T10:11:59.444467Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/54/000000022-000000003.tsm id=0 duration=71.394ms
> ts=2019-10-07T10:12:03.309322Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=87
> ts=2019-10-07T10:12:04.234337Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/59/000000031-000000003.tsm id=0 duration=102.396ms
> ts=2019-10-07T10:12:04.332932Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/87/000000156-000000002.tsm id=0 duration=851.227ms
> ts=2019-10-07T10:12:11.866406Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=25
> ts=2019-10-07T10:12:11.992990Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/25/000000071-000000002.tsm id=0 duration=83.623ms
> ts=2019-10-07T10:12:13.487297Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=36
> ts=2019-10-07T10:12:13.490966Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="engine is closed"
> ts=2019-10-07T10:12:13.555212Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/36/000000018-000000002.tsm id=0 duration=31.417ms
> ts=2019-10-07T10:12:14.521182Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=39
> ts=2019-10-07T10:12:14.617060Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/39/000000015-000000002.tsm id=0 duration=27.660ms
> ts=2019-10-07T10:12:41.253858Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/24/000000015-000000002.tsm id=0 duration=20.952ms
> ts=2019-10-07T10:12:43.281293Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/44/000000015-000000003.tsm id=0 duration=165.497ms
> ts=2019-10-07T10:12:45.426243Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/52/000000019-000000004.tsm id=0 duration=53.571ms
> ts=2019-10-07T10:12:52.161687Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/85/000000168-000000002.tsm id=0 duration=613.393ms
> ts=2019-10-07T10:12:58.773898Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/66/000000052-000000003.tsm id=0 duration=107.989ms
> ts=2019-10-07T10:12:59.425764Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T10:13:06.526161Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/69/000000053-000000002.tsm id=0 duration=232.124ms
> ts=2019-10-07T10:13:34.248077Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=3 trace\_id=0ILRCXf0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/03 op\_event=start
> ts=2019-10-07T10:13:34.260067Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=4 trace\_id=0ILRCXi0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/04 op\_event=start
> ts=2019-10-07T10:13:34.273958Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=0 trace\_id=0ILRCXlG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/00 op\_event=start
> ts=2019-10-07T10:13:34.277939Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=5 trace\_id=0ILRCXmG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/05 op\_event=start
> ts=2019-10-07T10:13:34.698322Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=1 trace\_id=0ILRCZQW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/01 op\_event=start
> ts=2019-10-07T10:13:34.714712Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=2 trace\_id=0ILRCZUW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/02 op\_event=start
> ts=2019-10-07T10:13:35.205503Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=7 trace\_id=0ILRCaPG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/07 op\_event=start
> ts=2019-10-07T10:13:35.213474Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=6 trace\_id=0ILRCaRG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/06 op\_event=start
> ts=2019-10-07T10:13:35.583694Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=3 trace\_id=0ILRCXf0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/03 op\_event=end op\_elapsed=1335.642ms
> ts=2019-10-07T10:13:35.813518Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=5 trace\_id=0ILRCXmG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/05 op\_event=end op\_elapsed=1535.590ms
> ts=2019-10-07T10:13:35.973803Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=0 trace\_id=0ILRCXlG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/00 op\_event=end op\_elapsed=1699.868ms
> ts=2019-10-07T10:13:36.035225Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=4 trace\_id=0ILRCXi0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/04 op\_event=end op\_elapsed=1775.173ms
> ts=2019-10-07T10:13:36.388081Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=2 trace\_id=0ILRCZUW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/02 op\_event=end op\_elapsed=1673.380ms
> ts=2019-10-07T10:13:36.543421Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=1 trace\_id=0ILRCZQW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/01 op\_event=end op\_elapsed=1845.108ms
> ts=2019-10-07T10:13:36.584074Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=6 trace\_id=0ILRCaRG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/06 op\_event=end op\_elapsed=1370.679ms
> ts=2019-10-07T10:13:36.840178Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=7 trace\_id=0ILRCaPG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/07 op\_event=end op\_elapsed=1634.687ms
> ts=2019-10-07T10:14:19.107136Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=92
> ts=2019-10-07T10:14:23.798974Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/92/000000177-000000002.tsm id=0 duration=4162.136ms
> ts=2019-10-07T10:14:53.514849Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/34/000000015-000000002.tsm id=0 duration=85.640ms
> ts=2019-10-07T10:14:54.893778Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/42/000000015-000000003.tsm id=0 duration=59.970ms
> ts=2019-10-07T10:14:55.743345Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/45/000000015-000000003.tsm id=0 duration=36.454ms
> ts=2019-10-07T10:24:10.375622Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/47/000000015-000000003.tsm id=0 duration=86.457ms
> ts=2019-10-07T10:24:22.543407Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/49/000000021-000000003.tsm id=0 duration=112.245ms
> ts=2019-10-07T10:24:33.565916Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/57/000000022-000000003.tsm id=0 duration=82.496ms
> ts=2019-10-07T10:24:53.595079Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/67/000000050-000000003.tsm id=0 duration=512.198ms
> ts=2019-10-07T10:24:58.143150Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=97
> ts=2019-10-07T10:24:59.770794Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/97/000000224-000000002.tsm id=0 duration=1073.836ms
> ts=2019-10-07T10:25:22.507769Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/37/000000015-000000002.tsm id=0 duration=22.937ms
> ts=2019-10-07T10:25:27.895507Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/53/000000020-000000003.tsm id=0 duration=65.498ms
> ts=2019-10-07T10:25:41.887707Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/62/000000046-000000003.tsm id=0 duration=134.798ms
> ts=2019-10-07T10:25:42.347925Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/90/000000159-000000002.tsm id=0 duration=762.383ms
> ts=2019-10-07T10:25:47.661753Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=68
> ts=2019-10-07T10:25:47.662854Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T10:25:47.863131Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/68/000000049-000000003.tsm id=0 duration=145.193ms
> ts=2019-10-07T10:25:59.362085Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=72
> ts=2019-10-07T10:25:59.848841Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/72/000000061-000000003.tsm id=0 duration=450.964ms
> ts=2019-10-07T10:26:20.502486Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=75
> ts=2019-10-07T10:26:20.970431Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/75/000000086-000000002.tsm id=0 duration=290.982ms
> ts=2019-10-07T10:27:30.053974Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/51/000000017-000000004.tsm id=0 duration=218.161ms
> ts=2019-10-07T10:28:10.876148Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/70/000000051-000000002.tsm id=0 duration=204.799ms
> ts=2019-10-07T10:29:29.633009Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=79
> ts=2019-10-07T10:29:30.738507Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/26/000000066-000000002.tsm id=0 duration=276.988ms
> ts=2019-10-07T10:29:34.014130Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/35/000000015-000000002.tsm id=0 duration=189.369ms
> ts=2019-10-07T10:29:35.025978Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/79/000000273-000000002.tsm id=0 duration=3411.019ms
> ts=2019-10-07T10:29:41.269560Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/50/000000019-000000003.tsm id=0 duration=100.924ms
> ts=2019-10-07T10:30:27.475770Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=5 trace\_id=0ILSANYW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/05 op\_event=start
> ts=2019-10-07T10:30:27.792054Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=4 trace\_id=0ILSAOoG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/04 op\_event=start
> ts=2019-10-07T10:30:40.306896Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=2 trace\_id=0ILSB9h0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/02 op\_event=start
> ts=2019-10-07T10:30:40.751459Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=0 trace\_id=0ILSBBRW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/00 op\_event=start
> ts=2019-10-07T10:30:40.777773Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=1 trace\_id=0ILSBBY0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/01 op\_event=start
> ts=2019-10-07T10:30:40.929512Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=3 trace\_id=0ILSBC8G000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/03 op\_event=start
> ts=2019-10-07T10:30:56.917830Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=7 trace\_id=0ILSCAaG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/07 op\_event=start
> ts=2019-10-07T10:30:57.531913Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=6 trace\_id=0ILSCCzl000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/06 op\_event=start
> ts=2019-10-07T10:31:13.816393Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=5 trace\_id=0ILSANYW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/05 op\_event=end op\_elapsed=46378.277ms
> ts=2019-10-07T10:31:20.822998Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=4 trace\_id=0ILSAOoG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/04 op\_event=end op\_elapsed=53099.895ms
> ts=2019-10-07T10:31:23.358325Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=2 trace\_id=0ILSB9h0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/02 op\_event=end op\_elapsed=43120.066ms
> ts=2019-10-07T10:31:23.494619Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=3 trace\_id=0ILSBC8G000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/03 op\_event=end op\_elapsed=42631.650ms
> ts=2019-10-07T10:31:23.535097Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=7 trace\_id=0ILSCAaG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/07 op\_event=end op\_elapsed=26651.612ms
> ts=2019-10-07T10:31:23.732939Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=1 trace\_id=0ILSBBY0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/01 op\_event=end op\_elapsed=43021.941ms
> ts=2019-10-07T10:31:23.791646Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=6 trace\_id=0ILSCCzl000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/06 op\_event=end op\_elapsed=26294.225ms
> ts=2019-10-07T10:31:23.845948Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=0 trace\_id=0ILSBBRW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/00 op\_event=end op\_elapsed=43161.123ms
> ts=2019-10-07T10:31:56.593108Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/82/000000179-000000002.tsm id=0 duration=3139.733ms
> ts=2019-10-07T10:34:30.938811Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/80/000000239-000000002.tsm id=0 duration=3544.004ms
> ts=2019-10-07T10:34:30.985692Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/19/000000004-000000002.tsm id=0 duration=80.263ms
> ts=2019-10-07T10:34:37.605651Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/43/000000015-000000003.tsm id=0 duration=180.469ms
> ts=2019-10-07T10:34:57.849726Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/56/000000020-000000003.tsm id=0 duration=256.403ms
> ts=2019-10-07T10:35:14.138001Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T10:35:42.449184Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/64/000000053-000000002.tsm id=0 duration=608.478ms
> ts=2019-10-07T10:35:52.011501Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/81/000000216-000000003.tsm id=0 duration=3813.281ms
> ts=2019-10-07T10:38:43.710175Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000032-000000003.tsm id=0 duration=3819.941ms
> ts=2019-10-07T10:38:45.133952Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000064-000000003.tsm id=1 duration=4551.274ms
> ts=2019-10-07T10:38:45.531769Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000096-000000003.tsm id=2 duration=4172.539ms
> ts=2019-10-07T10:38:46.185389Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000104-000000002.tsm id=3 duration=1498.636ms
> ts=2019-10-07T10:38:46.812193Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000122-000000001.tsm id=6 duration=626.630ms
> ts=2019-10-07T10:38:47.090880Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000121-000000002.tsm id=5 duration=1558.650ms
> ts=2019-10-07T10:38:47.170856Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000123-000000001.tsm id=7 duration=358.130ms
> ts=2019-10-07T10:38:47.190111Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000113-000000002.tsm id=4 duration=2055.730ms
> ts=2019-10-07T10:38:47.594087Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000124-000000001.tsm id=8 duration=502.650ms
> ts=2019-10-07T10:38:47.636348Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000125-000000001.tsm id=9 duration=464.799ms
> ts=2019-10-07T10:38:47.648389Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000126-000000001.tsm id=10 duration=457.460ms
> ts=2019-10-07T10:38:48.111279Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000129-000000001.tsm id=13 duration=432.041ms
> ts=2019-10-07T10:38:48.112530Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000128-000000001.tsm id=12 duration=510.304ms
> ts=2019-10-07T10:38:48.161430Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/99/000000127-000000001.tsm id=11 duration=605.858ms
> ts=2019-10-07T10:39:40.826621Z lvl=info msg="TSM compaction (start)" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group op\_event=start
> ts=2019-10-07T10:39:40.827460Z lvl=info msg="Beginning compaction" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_files\_n=8
> ts=2019-10-07T10:39:40.827540Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=0 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000122-000000001.tsm
> ts=2019-10-07T10:39:40.827586Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=1 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000123-000000001.tsm
> ts=2019-10-07T10:39:40.827628Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=2 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000124-000000001.tsm
> ts=2019-10-07T10:39:40.827672Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=3 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000125-000000001.tsm
> ts=2019-10-07T10:39:40.827747Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=4 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000126-000000001.tsm
> ts=2019-10-07T10:39:40.827853Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=5 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000127-000000001.tsm
> ts=2019-10-07T10:39:40.827931Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=6 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000128-000000001.tsm
> ts=2019-10-07T10:39:40.827977Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=7 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000129-000000001.tsm
> ts=2019-10-07T10:40:13.094951Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/23/000000022-000000002.tsm id=0 duration=34.070ms
> ts=2019-10-07T10:40:25.923091Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/38/000000015-000000002.tsm id=0 duration=208.706ms
> ts=2019-10-07T10:40:28.784790Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/40/000000015-000000002.tsm id=0 duration=196.281ms
> ts=2019-10-07T10:40:38.962713Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/60/000000033-000000003.tsm id=0 duration=166.063ms
> ts=2019-10-07T10:40:57.705778Z lvl=info msg="Compacted file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_index=0 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000129-000000002.tsm.tmp
> ts=2019-10-07T10:40:57.705941Z lvl=info msg="Finished compacting files" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group tsm1\_files\_n=1
> ts=2019-10-07T10:40:57.706043Z lvl=info msg="TSM compaction (end)" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=1 tsm1\_strategy=level trace\_id=0ILSh8yW000 op\_name=tsm1\_compact\_group op\_event=end op\_elapsed=76986.053ms
> ts=2019-10-07T10:40:58.342664Z lvl=info msg="TSM compaction (start)" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group op\_event=start
> ts=2019-10-07T10:40:58.365568Z lvl=info msg="Beginning compaction" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_files\_n=4
> ts=2019-10-07T10:40:58.365777Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_index=0 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000104-000000002.tsm
> ts=2019-10-07T10:40:58.365847Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_index=1 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000113-000000002.tsm
> ts=2019-10-07T10:40:58.365895Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_index=2 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000121-000000002.tsm
> ts=2019-10-07T10:40:58.365957Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_index=3 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000129-000000002.tsm
> ts=2019-10-07T10:41:12.092017Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/65/000000050-000000002.tsm id=0 duration=778.758ms
> ts=2019-10-07T10:42:03.149557Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=94
> ts=2019-10-07T10:42:28.017224Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/94/000000241-000000002.tsm id=0 duration=20781.688ms
> ts=2019-10-07T10:42:35.023943Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/71/000000055-000000002.tsm id=0 duration=3930.645ms
> ts=2019-10-07T10:43:54.798325Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=74
> ts=2019-10-07T10:43:55.401732Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="engine is closed"
> ts=2019-10-07T10:43:56.870576Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/74/000000076-000000002.tsm id=0 duration=773.190ms
> ts=2019-10-07T10:47:03.475081Z lvl=info msg="Compacted file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_index=0 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000129-000000003.tsm.tmp
> ts=2019-10-07T10:47:03.477472Z lvl=info msg="Finished compacting files" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group tsm1\_files\_n=1
> ts=2019-10-07T10:47:03.477575Z lvl=info msg="TSM compaction (end)" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=2 tsm1\_strategy=level trace\_id=0ILSlsuW000 op\_name=tsm1\_compact\_group op\_event=end op\_elapsed=365548.962ms
> ts=2019-10-07T10:47:03.931690Z lvl=info msg="TSM compaction (start)" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group op\_event=start
> ts=2019-10-07T10:47:03.932098Z lvl=info msg="Beginning compaction" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_files\_n=4
> ts=2019-10-07T10:47:03.932163Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_index=0 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000032-000000003.tsm
> ts=2019-10-07T10:47:03.932257Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_index=1 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000064-000000003.tsm
> ts=2019-10-07T10:47:03.932295Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_index=2 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000096-000000003.tsm
> ts=2019-10-07T10:47:03.932328Z lvl=info msg="Compacting file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_index=3 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000129-000000003.tsm
> ts=2019-10-07T10:48:39.878451Z lvl=info msg="Cache snapshot (start)" log\_id=0ILR1jZW000 engine=tsm1 trace\_id=0ILTD2mG000 op\_name=tsm1\_cache\_snapshot op\_event=start
> ts=2019-10-07T10:48:49.236982Z lvl=info msg="Snapshot for path written" log\_id=0ILR1jZW000 engine=tsm1 trace\_id=0ILTD2mG000 op\_name=tsm1\_cache\_snapshot path=/var/lib/influxdb/data/\_internal/monitor/1 duration=9393.894ms
> ts=2019-10-07T10:48:49.240949Z lvl=info msg="Cache snapshot (end)" log\_id=0ILR1jZW000 engine=tsm1 trace\_id=0ILTD2mG000 op\_name=tsm1\_cache\_snapshot op\_event=end op\_elapsed=9397.740ms
> ts=2019-10-07T10:50:05.641388Z lvl=info msg="Retention policy deletion check (start)" log\_id=0ILR1jZW000 service=retention trace\_id=0ILTIHn0000 op\_name=retention\_delete\_check op\_event=start
> ts=2019-10-07T10:50:10.514977Z lvl=info msg="Deleted shard group" log\_id=0ILR1jZW000 service=retention trace\_id=0ILTIHn0000 op\_name=retention\_delete\_check db\_instance=system db\_shard\_group=4 db\_rp=default
> ts=2019-10-07T10:51:55.417955Z lvl=info msg="Compacted file" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_index=0 tsm1\_file=/var/lib/influxdb/data/f3/default/99/000000129-000000004.tsm.tmp
> ts=2019-10-07T10:51:55.418432Z lvl=info msg="Finished compacting files" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group tsm1\_files\_n=1
> ts=2019-10-07T10:51:55.418670Z lvl=info msg="TSM compaction (end)" log\_id=0ILR1jZW000 engine=tsm1 tsm1\_level=3 tsm1\_strategy=level trace\_id=0ILT7BzG000 op\_name=tsm1\_compact\_group op\_event=end op\_elapsed=291836.529ms
> ts=2019-10-07T10:52:03.734994Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=98
> ts=2019-10-07T10:52:08.409491Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/21/000000015-000000002.tsm id=0 duration=143.920ms
> ts=2019-10-07T10:52:12.582001Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/98/000000214-000000002.tsm id=0 duration=4452.406ms
> ts=2019-10-07T10:52:43.819580Z lvl=info msg="failed to store statistics" log\_id=0ILR1jZW000 service=monitor error=timeout
> ts=2019-10-07T10:52:48.111748Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/29/000000021-000000002.tsm id=0 duration=133.747ms
> ts=2019-10-07T10:53:05.861305Z lvl=info msg="Deleted shard" log\_id=0ILR1jZW000 service=retention trace\_id=0ILTIHn0000 op\_name=retention\_delete\_check db\_instance=system db\_shard\_id=3 db\_rp=default
> ts=2019-10-07T10:53:08.224982Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/93/000000199-000000002.tsm id=0 duration=1922.044ms
> ts=2019-10-07T10:53:09.636539Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/33/000000015-000000002.tsm id=0 duration=35.551ms
> ts=2019-10-07T10:53:18.118175Z lvl=info msg="Deleted shard" log\_id=0ILR1jZW000 service=retention trace\_id=0ILTIHn0000 op\_name=retention\_delete\_check db\_instance=system db\_shard\_id=2 db\_rp=default
> ts=2019-10-07T10:53:27.387797Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=55
> ts=2019-10-07T10:53:27.389693Z lvl=info msg="Deleted shard" log\_id=0ILR1jZW000 service=retention trace\_id=0ILTIHn0000 op\_name=retention\_delete\_check db\_instance=system db\_shard\_id=4 db\_rp=default
> ts=2019-10-07T10:53:27.486479Z lvl=info msg="Retention policy deletion check (end)" log\_id=0ILR1jZW000 service=retention trace\_id=0ILTIHn0000 op\_name=retention\_delete\_check op\_event=end op\_elapsed=202085.246ms
> ts=2019-10-07T10:53:28.234590Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/55/000000022-000000003.tsm id=0 duration=394.444ms
> ts=2019-10-07T10:53:56.438611Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=63
> ts=2019-10-07T10:53:56.454984Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=63
> ts=2019-10-07T10:53:56.849842Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/63/000000051-000000003.tsm id=0 duration=233.415ms
> ts=2019-10-07T10:54:45.139955Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=76
> ts=2019-10-07T10:54:46.534079Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/76/000000116-000000002.tsm id=0 duration=1039.682ms
> ts=2019-10-07T10:54:51.279843Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T10:57:59.213842Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/83/000000184-000000003.tsm id=0 duration=2830.564ms
> ts=2019-10-07T10:58:57.384814Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/41/000000016-000000004.tsm id=0 duration=224.794ms
> ts=2019-10-07T10:59:42.794754Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=96
> ts=2019-10-07T11:00:15.373370Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/96/000000217-000000003.tsm id=0 duration=31764.263ms
> ts=2019-10-07T11:00:32.080523Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/27/000000041-000000002.tsm id=0 duration=562.282ms
> ts=2019-10-07T11:00:34.193742Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/30/000000016-000000004.tsm id=0 duration=206.995ms
> ts=2019-10-07T11:00:41.660256Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=91
> ts=2019-10-07T11:00:43.421495Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/91/000000162-000000002.tsm id=0 duration=1516.921ms
> ts=2019-10-07T11:00:51.890567Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=61
> ts=2019-10-07T11:00:51.891083Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T11:00:52.305088Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/61/000000040-000000003.tsm id=0 duration=244.684ms
> ts=2019-10-07T11:04:21.408759Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=78
> ts=2019-10-07T11:04:24.081603Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/78/000000287-000000003.tsm id=0 duration=1995.684ms
> ts=2019-10-07T11:05:41.780354Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/20/000000001-000000001.tsm id=0 duration=49.499ms
> ts=2019-10-07T11:05:55.775088Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/48/000000019-000000003.tsm id=0 duration=199.847ms
> ts=2019-10-07T11:06:20.310040Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=95
> ts=2019-10-07T11:06:23.886693Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/95/000000222-000000002.tsm id=0 duration=3322.943ms
> ts=2019-10-07T11:08:09.857972Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=84
> ts=2019-10-07T11:08:12.296110Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/84/000000175-000000002.tsm id=0 duration=1735.868ms
> ts=2019-10-07T11:08:19.040798Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T11:09:12.548623Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/22/000000016-000000003.tsm id=0 duration=28.334ms
> ts=2019-10-07T11:09:31.910457Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/28/000000027-000000002.tsm id=0 duration=180.718ms
> ts=2019-10-07T11:09:36.239139Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/46/000000016-000000003.tsm id=0 duration=82.460ms
> ts=2019-10-07T11:09:49.551457Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/58/000000029-000000003.tsm id=0 duration=320.920ms
> ts=2019-10-07T11:10:00.512808Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="shard is disabled" db\_shard\_id=89
> ts=2019-10-07T11:10:02.706390Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/89/000000157-000000002.tsm id=0 duration=1540.961ms
> ts=2019-10-07T11:10:48.079293Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/73/000000065-000000002.tsm id=0 duration=4334.105ms
> ts=2019-10-07T11:13:15.350633Z lvl=warn msg="Error while freeing cold shard resources" log\_id=0ILR1jZW000 service=store error="engine is closed" db\_shard\_id=86
> ts=2019-10-07T11:13:17.707782Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/86/000000167-000000002.tsm id=0 duration=1972.604ms
> ts=2019-10-07T11:14:28.389411Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=4 trace\_id=0ILUgZ\_l000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/04 op\_event=start
> ts=2019-10-07T11:14:29.168725Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=5 trace\_id=0ILUgbgl000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/05 op\_event=start
> ts=2019-10-07T11:14:34.999786Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=1 trace\_id=0ILUgyTl000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/01 op\_event=start
> ts=2019-10-07T11:14:36.210421Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=5 trace\_id=0ILUgbgl000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/05 op\_event=end op\_elapsed=7042.665ms
> ts=2019-10-07T11:14:36.635915Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=4 trace\_id=0ILUgZ\_l000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/04 op\_event=end op\_elapsed=8246.708ms
> ts=2019-10-07T11:14:38.811171Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=3 trace\_id=0ILUhCMW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/03 op\_event=start
> ts=2019-10-07T11:14:41.137000Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=0 trace\_id=0ILUhLS0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/00 op\_event=start
> ts=2019-10-07T11:14:42.765176Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=7 trace\_id=0ILUhRoG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/07 op\_event=start
> ts=2019-10-07T11:14:42.768817Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=2 trace\_id=0ILUhRol000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/02 op\_event=start
> ts=2019-10-07T11:14:44.115141Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=1 trace\_id=0ILUgyTl000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/01 op\_event=end op\_elapsed=9115.524ms
> ts=2019-10-07T11:14:51.588801Z lvl=info msg="Series partition compaction (start)" log\_id=0ILR1jZW000 partition=6 trace\_id=0ILUhzH0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/06 op\_event=start
> ts=2019-10-07T11:14:53.812450Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=3 trace\_id=0ILUhCMW000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/03 op\_event=end op\_elapsed=15021.644ms
> ts=2019-10-07T11:14:56.548087Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=0 trace\_id=0ILUhLS0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/00 op\_event=end op\_elapsed=15438.183ms
> ts=2019-10-07T11:14:57.908406Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=2 trace\_id=0ILUhRol000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/02 op\_event=end op\_elapsed=15167.683ms
> ts=2019-10-07T11:15:00.862080Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=7 trace\_id=0ILUhRoG000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/07 op\_event=end op\_elapsed=18123.933ms
> ts=2019-10-07T11:15:02.952306Z lvl=info msg="Series partition compaction (end)" log\_id=0ILR1jZW000 partition=6 trace\_id=0ILUhzH0000 op\_name=series\_partition\_compaction path=/var/lib/influxdb/data/f3/\_series/06 op\_event=end op\_elapsed=11363.563ms
> ts=2019-10-07T11:15:07.782025Z lvl=info msg="Cannot retrieve measurement cardinality" log\_id=0ILR1jZW000 service=store db\_instance=f3 error="shard is disabled"
> ts=2019-10-07T11:15:09.388679Z lvl=info msg="Opened file" log\_id=0ILR1jZW000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/f3/default/88/000000149-000000002.tsm id=0 duration=1331.381ms
> \`\`\`

---

<div class="post-metadata">

### Author: ![sineverba](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/sineverba/32/9638_2.png) [@sineverba](https://community.influxdata.com/u/sineverba)
#### Post date: [April 1, 2022, 5:22pm UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/3 "2022-04-01T17:22:17Z")

</div>

It is during backup.

I use Influx via Docker.

Some day ago I tried to restore backup on aother server, seems all ok. But, when I do backup, I got all that errors.

In next days I will try another restore on another server, just to try…

---

<div class="post-metadata">

### Author: ![HannesKannes](https://avatars.discourse-cdn.com/v4/letter/h/bc8723/32.png) [@HannesKannes](https://community.influxdata.com/u/HannesKannes)
#### Post date: [September 26, 2022, 9:05am UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/4 "2022-09-26T09:05:06Z")

</div>

Hi,  
how can this issue be fixed?  
I have the same failure (shared X removed) during backup.  
 ![grafik](https://us1.discourse-cdn.com/flex023/uploads/influxdata/original/2X/a/a37d9f8671c087c62b10b98838fd06aa887a003e.png)  
I have Influxdb 2.3.0 and also run in Docker  
Regards  
Hannes

---

<div class="post-metadata">

### Author: ![cguy](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/cguy/32/12440_2.png) [@cguy](https://community.influxdata.com/u/cguy)
#### Post date: [January 26, 2023, 11:42pm UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/5 "2023-01-26T23:42:53Z")

</div>

@HannesKannes Did you find a solution? This happens for me as well:

```auto
bash-5.1# influx backup /influxdb2-backup/backup --bucket MyBucket -t MySeCrEtToKeN!
2023/01/26 23:33:24 INFO: Downloading metadata snapshot
2023/01/26 23:33:24 INFO: Backing up TSM for shard 1
2023/01/26 23:33:24 WARN: Shard 1 removed during backup
2023/01/26 23:33:24 INFO: Backing up TSM for shard 2
2023/01/26 23:33:24 WARN: Shard 2 removed during backup
2023/01/26 23:33:24 INFO: Backing up TSM for shard 3
2023/01/26 23:33:24 WARN: Shard 3 removed during backup
2023/01/26 23:33:24 INFO: Backing up TSM for shard 4
2023/01/26 23:33:24 WARN: Shard 4 removed during backup
2023/01/26 23:33:24 INFO: Backing up TSM for shard 5
2023/01/26 23:33:24 WARN: Shard 5 removed during backup
2023/01/26 23:33:24 INFO: Backing up TSM for shard 6
2023/01/26 23:33:24 WARN: Shard 6 removed during backup
2023/01/26 23:33:24 INFO: Backing up TSM for shard 7
2023/01/26 23:33:50 INFO: Backing up TSM for shard 8

```

And then when restoring from that backup, there is data missing.

This is running influxdb 2.3.0:

```auto
InfluxDB v2.3.0+SNAPSHOT.090f681737 Server: 090f681 Frontend: a2bd1f3

```

---

<div class="post-metadata">

### Author: ![MzazM](https://avatars.discourse-cdn.com/v4/letter/m/fbc32d/32.png) [@MzazM](https://community.influxdata.com/u/MzazM)
#### Post date: [March 6, 2023, 1:37pm UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/6 "2023-03-06T13:37:07Z")

</div>

I got the same today (using 2.1.1) . I wonder how to check which data is contained inside of shards that are being removed. I am hoping that that those shard have been deleted because, for example, they fall outside of the Retention policy. How to check this?

---

<div class="post-metadata">

### Author: ![MzazM](https://avatars.discourse-cdn.com/v4/letter/m/fbc32d/32.png) [@MzazM](https://community.influxdata.com/u/MzazM)
#### Post date: [March 6, 2023, 1:56pm UTC](https://community.influxdata.com/t/shard-x-removed-during-backup/24436/7 "2023-03-06T13:56:10Z")

</div>

from my manifest file and the content in the backup folder, it seems that all the shards necessary have been backupped.