Permission Denied: during upgrade from 1 to 2

InfluxDB 2
1

I’m trying to make the switch from 1 to 2 (official docker container; Running Docker Desktop + Ubuntu in WSL2) and getting a permission denied issue during the db upgrade procedure:

{“level”:“error”,“ts”:1628254367.3688755,“caller”:“upgrade/upgrade.go:467”,“msg”:“Database upgrade error, removing data”,“error”:“error getting size of /var/lib/influxdb/data: open /var/lib/influxdb/data/_internal: permission denied”,“stacktrace”:“github .com/influxdata/influxdb/v2/cmd/influxd/upgrade.runUpgradeE\n\t/home/circleci/go/src/github .com/influxdata/influxdb/cmd/influxd/upgrade/upgrade.go:467\ngithub .com/influxdata/influxdb/v2/cmd/influxd/upgrade.NewCommand.func1\n\t/home/circleci/go/src/github.com/influxdata/influxdb/cmd/influxd/upgrade/upgrade.go:157\ngithub .com/spf13/cobra.(*Command).execute\n\t/home/circleci/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842\ngithub .com/spf13/cobra.(*Command).ExecuteC\n\t/home/circleci/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950\ngithub .com/spf13/cobra.(*Command).Execute\n\t/home/circleci/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887\nmain.main\n\t/home/circleci/go/src/github.com/influxdata/influxdb/cmd/influxd/main.go:53\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:204”}

I was running the previous 1.x version with all mounts running as a non root user (“scott”), and during the upgrade process previous to the above error I was getting the perm denied error for a different db:

“error getting size of /var/lib/influxdb/data: open /var/lib/influxdb/data/telegraf: permission denied”

I ls -al and noticed all folders/files were under the user “scott” while the new folders/files so far during the upgrade were “root”, so I went and chown -R root:root <all stuff owned by scott> and retried the upgrade, which then led to the previous error.

As of now, all folders/files are listed as owned by ‘root’, but still get the permission denied error.

Here’s my current tree:

scott@my-Cloud:/opt/docker/configs/InfluxDB$ ls
data  meta  v2  wal
scott@my-Cloud:/opt/docker/configs/InfluxDB$ ls -al
total 24
drwxr-xr-x  6 root  root  4096 Aug  5 01:30 .
drwxrwxrwx 70 scott scott 4096 Jul 26 23:28 ..
drwxr-xr-x  5 root  root  4096 Mar 23 22:38 data
drwxr-xr-x  2 root  root  4096 Aug  4 19:23 meta
drwxr-xr-x  4 root  root  4096 Aug  5 01:30 v2
drwx------  5 root  root  4096 Mar 23 22:38 wal
scott@my-Cloud:/opt/docker/configs/InfluxDB$ cd data/
scott@my-Cloud:/opt/docker/configs/InfluxDB/data$ ls -al
total 20
drwxr-xr-x 5 root root 4096 Mar 23 22:38 .
drwxr-xr-x 6 root root 4096 Aug  5 01:30 ..
drwx------ 4 root root 4096 Jan 29  2021 _internal
drwx------ 3 root root 4096 Mar 26 21:25 telegraf
drwx------ 4 root root 4096 Feb  4  2021 varken
scott@my-Cloud:/opt/docker/configs/InfluxDB/data$ cd _internal/
-bash: cd: _internal/: Permission denied
scott@my-Cloud:/opt/docker/configs/InfluxDB/data$ sudo -s
root@my-Cloud:/opt/docker/configs/InfluxDB/data# ls -la
total 20
drwxr-xr-x 5 root root 4096 Mar 23 22:38 .
drwxr-xr-x 6 root root 4096 Aug  5 01:30 ..
drwx------ 4 root root 4096 Jan 29  2021 _internal
drwx------ 3 root root 4096 Mar 26 21:25 telegraf
drwx------ 4 root root 4096 Feb  4  2021 varken
root@my-Cloud:/opt/docker/configs/InfluxDB/data# cd _internal/
root@my-Cloud:/opt/docker/configs/InfluxDB/data/_internal# ls -la
total 16
drwx------  4 root root 4096 Jan 29  2021 .
drwxr-xr-x  5 root root 4096 Mar 23 22:38 ..
drwxr-xr-x 10 root root 4096 Jan 29  2021 _series
drwx------  9 root root 4096 Aug  4 19:23 monitor

and my current Compose:

  influxdb:
    hostname: influxdb
    image: influxdb:latest
    environment:
      TZ: ${TZ}
      DOCKER_INFLUXDB_INIT_USERNAME: 'Scott'
      DOCKER_INFLUXDB_INIT_PASSWORD: "redacted"
      DOCKER_INFLUXDB_INIT_ORG: "J. Scott Elblein"
      DOCKER_INFLUXDB_INIT_BUCKET: "default_bucket"
      DOCKER_INFLUXDB_INIT_RETENTION: ''
      DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: ''
      DOCKER_INFLUXDB_INIT_MODE: 'upgrade'
    volumes:
      - /opt/docker/configs/InfluxDB:/var/lib/influxdb:rw
      - /opt/docker/configs/InfluxDB/v2/data:/var/lib/influxdb2:rw
      - /opt/docker/configs/InfluxDB/v2/conf:/etc/influxdb2:rw
      - /etc/localtime:/etc/localtime:ro
    networks:
      - odb
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.labels.MainDaemon == true
      restart_policy:
        condition: any

Admittedly, I’m not the most experienced Linux user on the planet, and this is a permissions issue so I assume it’s a fairly easy fix, but I’m not sure what it is?

2

The official influxdb container also runs as a non-root user by default. Users who have hit this in the past reported that running this:

chown -R 1000:1000

On the directory containing v1 data fixed the problem. Can you give it a try?

2 Likes
3

@dan-moran That did the trick, ty!

I just ran that on the entire Influxdb dir for everything that gets mounted to the container and it worked.

Strange to me, since that data dir was originally already under the “scott” user, which is normally UID/GID 1000, and running that cmd set it back to that same user/gid?

I did also previously sniff around for a way to set the UID/GID in the container via an environment variable (normally I just do something like user: "${UID}:1000" or PUID: 1000 PGID: 1000) but came up empty; did I miss it somewhere, or is it not available? (and would that have also worked?)