Unable to create operator token

InfluxDB 2
,
1

Issue Summary
Current installation of InfluxDB2 does not have an operator token / root authorization token. Have tried the following command line option: influx auth create --operator and received the following message:

Error: could not write auth with provided arguments: 403 Forbidden: permission read:authorizations is not allowed: read:authorizations is unauthorized

Tried the recovery method using the following commands:

root@influxdb:/# influx config set --config-name default --active
Active  Name    URL                     Org
*       default http://localhost:8086   instantdreams
root@influxdb:/# influx org list
ID                      Name
b962535ddb0d5f55        instantdreams
root@influxdb:/# influx user list
ID                      Name
0aa65c66faa7d000        admin
root@influxdb:/# influxd recovery auth create-operator --username admin --org instantdreams --bolt-path /var/lib/influxdb2/influxdb.bolt
2024-12-09T18:17:36.514759Z     info    Resources opened        {"log_id": "0tNA5~~0000", "system": "bolt-kvstore", "path": "/var/lib/influxdb2/influxdb.bolt"}
Error: bucket "authorizationsv1": bucket not found

When searching for this issue there are a number of suggestions, all of which do not seem to resolve the problem.

Version Details

  • Host: Debian 6.1.112-1 (2024-09-30) x86_64 GNU/Linux
  • Docker: 27.3.1, build ce12230
  • Docker Compose: v2.29.7
  • InfluxDB: InfluxDB v2.7.11 (git: fbf5d4ab5e) build_date: 2024-12-02T17:48:15Z
  • InfluxDB2 CLI: Influx CLI dev (git: a79a2a1b82[…] build_date: 2024-04-16T14:34:32Z

Configuration Details
compose.yaml content:

services:
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    ports:
      - 8086:8086 # web ui
    env_file:
      - .env
    volumes:
      - /srv/influxdb/data:/var/lib/influxdb2
      - /srv/influxdb/config:/etc/influxdb2
      - /srv/influxdb/backup:/var/lib/backup
      - /etc/localtime:/etc/localtime:ro
    restart: unless-stopped

.env content:

DOCKER_INFLUXDB_INIT_MODE=setup
DOCKER_INFLUXDB_INIT_USERNAME=[username]
DOCKER_INFLUXDB_INIT_PASSWORD=[password]
DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=[token]
DOCKER_INFLUXDB_INIT_ORG=[org]
DOCKER_INFLUXDB_INIT_BUCKET=[bucket]

Note that the [token] provided is the same as the admin token for the default org, instantdreams, and is not an operator token.

Configuration details:

root@influxdb:/# influx config list
Active  Name            URL                     Org
*       default         http://localhost:8086   instantdreams
        homeassistant   http://localhost:8086   homeassistant
        scrutiny        http://localhost:8086   scrutiny

Org instantdreams auth details:

root@influxdb:/# influx auth list
ID                      Description     Token                                                                                           User Name       User ID                     Permissions
0acbf650a257e000        instantdreams   [redacted]        admin           0aa65c66faa7d000    [read:orgs/b962535ddb0d5f55/annotations write:orgs/b962535ddb0d5f55/annotations read:orgs/b962535ddb0d5f55/authorizations write:orgs/b962535ddb0d5f55/authorizations read:orgs/b962535ddb0d5f55/buckets write:orgs/b962535ddb0d5f55/buckets read:orgs/b962535ddb0d5f55/checks write:orgs/b962535ddb0d5f55/checks read:orgs/b962535ddb0d5f55/dashboards write:orgs/b962535ddb0d5f55/dashboards read:orgs/b962535ddb0d5f55/dbrp write:orgs/b962535ddb0d5f55/dbrp read:orgs/b962535ddb0d5f55/documents write:orgs/b962535ddb0d5f55/documents read:orgs/b962535ddb0d5f55/labels write:orgs/b962535ddb0d5f55/labels read:orgs/b962535ddb0d5f55/notebooks write:orgs/b962535ddb0d5f55/notebooks read:orgs/b962535ddb0d5f55/notificationEndpoints write:orgs/b962535ddb0d5f55/notificationEndpoints read:orgs/b962535ddb0d5f55/notificationRules write:orgs/b962535ddb0d5f55/notificationRules read:/orgs/b962535ddb0d5f55 read:orgs/b962535ddb0d5f55/remotes write:orgs/b962535ddb0d5f55/remotes read:orgs/b962535ddb0d5f55/replications write:orgs/b962535ddb0d5f55/replications read:orgs/b962535ddb0d5f55/scrapers write:orgs/b962535ddb0d5f55/scrapers read:orgs/b962535ddb0d5f55/secrets write:orgs/b962535ddb0d5f55/secrets read:orgs/b962535ddb0d5f55/sources write:orgs/b962535ddb0d5f55/sources read:orgs/b962535ddb0d5f55/tasks write:orgs/b962535ddb0d5f55/tasks read:orgs/b962535ddb0d5f55/telegrafs write:orgs/b962535ddb0d5f55/telegrafs read:/users/0aa65c66faa7d000 write:/users/0aa65c66faa7d000 read:orgs/b962535ddb0d5f55/variables write:orgs/b962535ddb0d5f55/variables read:orgs/b962535ddb0d5f55/views write:orgs/b962535ddb0d5f55/views]

Org homeassistant auth details:

root@influxdb:/# influx config set --config-name homeassistant --active
Active  Name            URL                     Org
*       homeassistant   http://localhost:8086   homeassistant
root@influxdb:/# influx auth list
ID                      Description     Token                                                                                           User Name       User ID                     Permissions
0c395d1b25697000        Home Assistant  [redacted]        admin           0aa65c66faa7d000    [read:orgs/31766a4bc0dce764/annotations write:orgs/31766a4bc0dce764/annotations read:orgs/31766a4bc0dce764/authorizations write:orgs/31766a4bc0dce764/authorizations read:orgs/31766a4bc0dce764/buckets write:orgs/31766a4bc0dce764/buckets read:orgs/31766a4bc0dce764/checks write:orgs/31766a4bc0dce764/checks read:orgs/31766a4bc0dce764/dashboards write:orgs/31766a4bc0dce764/dashboards read:orgs/31766a4bc0dce764/dbrp write:orgs/31766a4bc0dce764/dbrp read:orgs/31766a4bc0dce764/documents write:orgs/31766a4bc0dce764/documents read:orgs/31766a4bc0dce764/labels write:orgs/31766a4bc0dce764/labels read:orgs/31766a4bc0dce764/notebooks write:orgs/31766a4bc0dce764/notebooks read:orgs/31766a4bc0dce764/notificationEndpoints write:orgs/31766a4bc0dce764/notificationEndpoints read:orgs/31766a4bc0dce764/notificationRules write:orgs/31766a4bc0dce764/notificationRules read:/orgs/31766a4bc0dce764 read:orgs/31766a4bc0dce764/remotes write:orgs/31766a4bc0dce764/remotes read:orgs/31766a4bc0dce764/replications write:orgs/31766a4bc0dce764/replications read:orgs/31766a4bc0dce764/scrapers write:orgs/31766a4bc0dce764/scrapers read:orgs/31766a4bc0dce764/secrets write:orgs/31766a4bc0dce764/secrets read:orgs/31766a4bc0dce764/sources write:orgs/31766a4bc0dce764/sources read:orgs/31766a4bc0dce764/tasks write:orgs/31766a4bc0dce764/tasks read:orgs/31766a4bc0dce764/telegrafs write:orgs/31766a4bc0dce764/telegrafs read:/users/0aa65c66faa7d000 write:/users/0aa65c66faa7d000 read:orgs/31766a4bc0dce764/variables write:orgs/31766a4bc0dce764/variables read:orgs/31766a4bc0dce764/views write:orgs/31766a4bc0dce764/views]

Org scrutiny auth details:

root@influxdb:/# influx config set --config-name scrutiny --active
Active  Name            URL                     Org
*       scrutiny        http://localhost:8086   scrutiny
root@influxdb:/# influx auth list
ID                      Description     Token                                                                                           User Name       User ID                     Permissions
0acbf131b397e000        Scrutiny        [redacted]        admin           0aa65c66faa7d000    [read:orgs/36b6c4934c44d5be/annotations write:orgs/36b6c4934c44d5be/annotations read:orgs/36b6c4934c44d5be/authorizations write:orgs/36b6c4934c44d5be/authorizations read:orgs/36b6c4934c44d5be/buckets write:orgs/36b6c4934c44d5be/buckets read:orgs/36b6c4934c44d5be/checks write:orgs/36b6c4934c44d5be/checks read:orgs/36b6c4934c44d5be/dashboards write:orgs/36b6c4934c44d5be/dashboards read:orgs/36b6c4934c44d5be/dbrp write:orgs/36b6c4934c44d5be/dbrp read:orgs/36b6c4934c44d5be/documents write:orgs/36b6c4934c44d5be/documents read:orgs/36b6c4934c44d5be/labels write:orgs/36b6c4934c44d5be/labels read:orgs/36b6c4934c44d5be/notebooks write:orgs/36b6c4934c44d5be/notebooks read:orgs/36b6c4934c44d5be/notificationEndpoints write:orgs/36b6c4934c44d5be/notificationEndpoints read:orgs/36b6c4934c44d5be/notificationRules write:orgs/36b6c4934c44d5be/notificationRules read:/orgs/36b6c4934c44d5be read:orgs/36b6c4934c44d5be/remotes write:orgs/36b6c4934c44d5be/remotes read:orgs/36b6c4934c44d5be/replications write:orgs/36b6c4934c44d5be/replications read:orgs/36b6c4934c44d5be/scrapers write:orgs/36b6c4934c44d5be/scrapers read:orgs/36b6c4934c44d5be/secrets write:orgs/36b6c4934c44d5be/secrets read:orgs/36b6c4934c44d5be/sources write:orgs/36b6c4934c44d5be/sources read:orgs/36b6c4934c44d5be/tasks write:orgs/36b6c4934c44d5be/tasks read:orgs/36b6c4934c44d5be/telegrafs write:orgs/36b6c4934c44d5be/telegrafs read:/users/0aa65c66faa7d000 write:/users/0aa65c66faa7d000 read:orgs/36b6c4934c44d5be/variables write:orgs/36b6c4934c44d5be/variables read:orgs/36b6c4934c44d5be/views write:orgs/36b6c4934c44d5be/views]

Resources Used

Additional Steps

  1. Install boltbrowser binary:
    wget https://github.com/br0xen/boltbrowser/releases/download/2.2/boltbrowser.linux64
chmod +x boltbrowser.linux64
  2. Create a copy of influxdb.bolt and edit with boltbrowser:
    cp data/influxd.bolt influxd.bolt
./boltbrowser.linux64 influxd.bolt
  3. Edit according to russorat’s instructions on GitHub but no operator entry found to add the details

Help Request
At this point I’m stuck - I think I’ve explored all the usual options but cannot connect the dots to the last bit I need to generate an operator token in a configuration that lacks an authorizationsv1 entry for Admin’s token.

What am I missing here?

2

Hello @instantdreams,
Thanks for providing so much detail and for trying various options.
I’m asking around, thanks for your patience.

1 Like
3

@instantdreams
I know you used influxd recovery auth create-operator, but I’m wondering if you are able to try the following please:

  1. Stop influxd
  2. Run influxd recovery auth create-operator --org example-org --username example-user to generate a new operator token. If he’s using a non-default installation of InfluxDB, he’ll also need to provide the filepath of his boltdb using the --bolt-path flag.
  3. Store the newly generated operator token securely.
  4. Restart influxd
  5. Attempt to authorize with InfluxDB using the new operator token.

Thanks!!

4

This might relate to Best practice of stopping InfluxDB inside a docker container without stopping the container. I would be happy to try this. I’ve installed InfluxDB2 using the standard docker compose instructions.

I access my container shell using docker exec -it influxdb bash. What command would I run inside the container to stop influxd? I’ve tried the following:

$ docker exec -it influxdb bash
root@influxdb:/# sudo service influxd stop
bash: sudo: command not found
root@influxdb:/# service influxd stop
influxd: unrecognized service
root@influxdb:/# systemctl stop influxdb
bash: systemctl: command not found
root@influxdb:/# influxd stop
Error: unknown command "stop" for "influxd"
See 'influxd -h' for help

I’ll absolutely try to run this if I can just stop influxd in the docker container.

5

I tried the following, just in case:

$ docker compose down
[+] Running 1/1
 ✔ Container influxdb  Removed                                                                                                                         0.4s
$ docker exec influxdb influxd recovery auth create-operator --org instantdreams --username admin
Error response from daemon: No such container: influxdb
$ docker compose up --detach
[+] Running 1/1
 ✔ Container influxdb  Started

I feel the answer will likely involve something like docker run --rm --entrypoint /bin/bash influxdb:latest influxd recovery auth create-operator --org instantdreams --username but I hope you and the team can assist me.

6

Just following up to see if there is any update on this issue - how do I stop influxd within my docker container?

7

Now that the winter break is over, any chance this could be reviewed?

8

Checking to see if there is any direction from the team on how to stop influxdb from within a running docker container.

9

@instantdreams You should be able to exec into the container and kill the process from inside:

# open a shell session inside of your docker container
docker exec -it <container-id> sh

# From inside your Docker container, find the influxd process
ps aux | grep [i]nfluxd

# Use the influxd process ID (PID) and kill the influxd process
kill <influxd-PID>

You should then be able to run the recovery commands while you’re still in the container.

1 Like
10

Unfortunately it looks like ps isn’t part of the base image for influx:

$ docker exec -it influxdb bash
root@influxdb:/# ps aux | grep [i]nfluxd
bash: ps: command not found
root@influxdb:/#

I tried a few alterative commands:

root@influxdb:/# ps
bash: ps: command not found
root@influxdb:/# top
bash: top: command not found
root@influxdb:/# htop
bash: htop: command not found
root@influxdb:/# procs
bash: procs: command not found

None were included in the image, and I couldn’t install them:

root@influxdb:/# apt install ps
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package ps
root@influxdb:/# apt install top
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package top
root@influxdb:/# apt install htop
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package htop
root@influxdb:/# apt install procs
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package procs
root@influxdb:/#

…which is good, because we want the image to restrict people adding things. Any idea how I could get around this?

11

@scott @Anaisdg Just checking in to see if there is any more progress with being able to stop the influxdb process inside a running docker container. I just retried using shell rather than bash with the same result:

$ docker exec -it influxdb sh
# ps
sh: 1: ps: not found
# ps aux
sh: 2: ps: not found
# top
sh: 3: top: not found
# htop
sh: 4: htop: not found
# procs
sh: 5: procs: not found
# proc
sh: 6: proc: not found
# apt install ps
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package ps
#

Any direction is most welcome!

12

@scott @Anaisdg Pinging this thread to see if there’s any movement!

13

Wondering if there is any advancement on this issue!

14

Hello @instantdreams,
Sorry for the delay thanks for pinging.
Can you please try:

docker exec -it influxdb sh -c "killall influxd"

Or from inside the container:

cat /proc/[0-9]*/cmdline | grep -n influxd

and

kill <PID>

or perhaps

docker exec -it influxdb sh -c "pkill influxd"

You can also try installing ps
``
docker exec -it influxdb sh
apk add --no-cache procps

you can verify with

ps aux

15

Hey @Anaisdg I do appreciate you following up with this. However, no luck.

Using killall didn’t work:

$ docker exec -it influxdb sh -c "killall influxd"
sh: 1: killall: not found
exit status 127

Same with pkill:

$ docker exec -it influxdb sh -c "pkill influxd"
sh: 1: pkill: not found
exit status 127

Connecting to the container to examine the /proc/ directory structure wasn’t a success:

$ docker exec -it influxdb bash
root@influxdb:/# cat /proc/[0-9]*/cmdline | grep -n influxdb
grep: (standard input): binary file matches

Installing ps using your suggeted method didn’t work:

$ docker exec -it influxdb sh
# apk add --no-cache procps
sh: 1: apk: not found
# apt install procps
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package procps

I was able to install it using the following:

$ docker exec -it influxdb sh
# apt-get update && apt-get install procps
[...]
Fetched 9306 kB in 2s (4977 kB/s)
[...]
The following additional packages will be installed:
  libproc2-0 psmisc
The following NEW packages will be installed:
  libproc2-0 procps psmisc
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
[...]
Setting up psmisc (23.6-1) ...
Setting up libproc2-0:amd64 (2:4.0.2-3) ...
Setting up procps (2:4.0.2-3) ...
Processing triggers for libc-bin (2.36-9+deb12u9) ...
# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
influxdb       1  0.5  0.9 3466928 152924 ?      Ssl  Feb25  15:32 influxd
root         122  0.0  0.0   2576   888 pts/0    Ss   12:12   0:00 sh
root         305  0.0  0.0   8088  3936 pts/0    R+   12:15   0:00 ps aux
# kill 1
# exit status 137
$

This immediately kicked me out of the shell. I checked and the container had restarted, so I logged back in:

$ docker exec -it influxdb sh
# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
influxdb       1  1.6  1.3 3466672 211828 ?      Ssl  12:17   0:02 influxd
root          83  1.5  0.0   2576   880 pts/0    Ss   12:19   0:00 sh
root          89  0.0  0.0   8088  3868 pts/0    R+   12:19   0:00 ps aux
#

The influxdb process was running again. Killing it just restarts the container.

I tried the command adter reconnecting just in case:

$ docker exec -it influxdb sh
# influxd recovery auth create-operator --org example-org --username example-user
2025-02-27T19:25:01.541616Z     info    Resources opened        {"log_id": "0uzDktsW000", "system": "bolt-kvstore", "path": "/root/.influxdbv2/influxd.bolt"}
Error: bucket "authorizationsv1": bucket not found
See 'influxd -h' for help

Same issue as before.

The entrypoint for the influxdb container is /entrypoint.sh, which appears to run influxd as a daemon in the container. I wonder if you could check to see if I could use this to run the command. Here’s the start of the script:

$ docker exec -it influxdb bash
root@influxdb:/# cat entrypoint.sh
#!/bin/bash
set -eo pipefail

## READ ME
##
## This script handles a few use-cases:
##   1. Running arbitrary shell commands other than `influxd`
##   2. Running subcommands of `influxd` other than `run`
##   3. Running `influxd run` with no auto-setup or auto-upgrade behavior
##   4. Running `influxd` with automated setup of a fresh 2.x DB
##   5. Running `influxd` with automated upgrade from a 1.x DB

Use case 2 is exactly what I want here, we just need to check with the influxdb docker team.

Thank you for your patience with this!

16

Reviewing this, I tried the following:

$ docker compose down
[+] Running 1/1
 ✔ Container influxdb  Removed   
$ docker run --rm --entrypoint /entrypoint.sh influxdb:latest influxd recovery auth create-operator --org instantdreams --username admin --bolt-path /var/lib/influxdb2/influxdb.bolt
ts=2025-02-27T19:47:50.331890Z lvl=info msg="Resources opened" log_id=0uzF3RiG000 system=bolt-kvstore path=/var/lib/influxdb2/influxdb.bolt
Error: bucket "authorizationsv1": bucket not found
See 'influxd -h' for help
exit status 1

Same issue - that authorizationsv1 bucket isn’t found. Taking this back to basics, I wanted to check the paths for the docker container.

First, the default location:

$ docker exec influxdb ls -lha /root/.influxdbv2/
total 24K
drwx------ 2 root root 4.0K Feb 27 12:54 .
drwx------ 1 root root 4.0K Feb 27 12:54 ..
-rw------- 1 root root  16K Feb 27 12:54 influxd.bolt

Second, the configuration directory set up by the docker container:

$ docker exec influxdb ls -lha /var/lib/influxdb2
total 352K
drwx------ 3 influxdb influxdb 4.0K Dec  6 11:54 .
drwxr-xr-x 1 root     root     4.0K Feb 27 12:49 ..
drwx------ 5 influxdb influxdb 4.0K Feb 23  2023 engine
-rw-rw-r-- 1 influxdb influxdb 128K Feb 27 12:49 influxd.bolt
-rw-r--r-- 1 influxdb influxdb 128K Dec  6 11:54 influxd.bolt.bak
-rw-rw-r-- 1 influxdb influxdb 120K Jan 25  2023 influxd.sqlite
-rw------- 1 influxdb root      16K Dec  6 11:03 influxdb.bolt

In the config directory there are two bolt databases, influxd.bolt and influxdb.bolt.

If we call the influxd recovery command with no path set, it shows us the default location it is looking for:

$ docker exec influxdb influxd recovery auth create-operator --org instantdreams --username admin
ts=2025-02-27T20:01:38.271363Z lvl=info msg="Resources opened" log_id=0uzFqyrl000 system=bolt-kvstore path=/root/.influxdbv2/influxd.bolt
Error: bucket "authorizationsv1": bucket not found
See 'influxd -h' for help
exit status 1

We get the same message if we explicitly set the default path:

$ docker exec influxdb influxd recovery auth create-operator --org instantdreams --username admin --bolt-path /root/.influxdbv2/influxd.bolt
ts=2025-02-27T20:01:40.889220Z lvl=info msg="Resources opened" log_id=0uzFr86G000 system=bolt-kvstore path=/root/.influxdbv2/influxd.bolt
Error: bucket "authorizationsv1": bucket not found
See 'influxd -h' for help
exit status 1

In my original message in this thread I was using /var/lib/influxdb2/influxdb.bolt and according to the default I should be using the /var/lib/influxdb2/influxd.bolt database. Let’s test that.

$ docker exec influxdb influxd recovery auth create-operator --org instantdreams --username admin --bolt-path /var/lib/influxdb2/influxd.bolt
Error: unable to open boltdb file unable to open boltdb file timeout
See 'influxd -h' for help
exit status 1

Aha, now at least it’s a different message. Perhaps the timeout is because influxd is still running. Let’s take the container down and run the image with the influxd command:

$ docker compose down
[+] Running 1/1
 ✔ Container influxdb  Removed                                                                                                                         0.4s
$ docker run --rm --entrypoint /entrypoint.sh influxdb:latest influxd recovery auth create-operator --org instantdreams --username admin --bolt-path /var/lib/influxdb2/influxd.bolt
ts=2025-02-27T20:11:31.639175Z lvl=info msg="Resources opened" log_id=0uzGQBgl000 system=bolt-kvstore path=/var/lib/influxdb2/influxd.bolt
Error: bucket "authorizationsv1": bucket not found
See 'influxd -h' for help
exit status 1

Oh bugger. Back to square one.

17

Quick ping to @Anaisdg to determine if there is a means to create an operator token or recover the original configuration for a container hosted instance of influxdb2.