Successful upgrade from 1.8 to 2.x!

In the course of debugging horrible performance with the InfluxDB engine, I tried to upgrade our server from 1.8.9 to the latest 2.6 following InfluxData’s automatic upgrade guide. This was not straightforward and the forums have many unanswered questions relating to upgrades, so I’m sharing what I learned. Hopefully it can help you upgrade.

It’s possible to roll back to 1.8

First of all, don’t be scared, you can easily re-install influxdb if you need to roll back from influxdb2 because the upgrade copies data and leaves your original 1.8 data intact.

You need to setup the 1.8 users first

The upgrade guide implies that Secure by Default is optional, but I believe these things are mandatory unless you’re some Influx guru who can fix the permissions later:

  • You need an admin account (ALL with no database given). It won’t get transferred, but if it doesn’t exist it seems the 1.8 permissions commands are broken.
  • You need a non-admin account that can read/write your databases (ALL with a database given). This will get transferred over.
  • You need to enable auth-enabled = true, and I’d recommend testing a query with the new settings.

Without doing these I never got a working token (see the last problem in this post) so I wasn’t able to do anything with the new 2.x server, including give myself permissions.

You must run the upgrade as user influxdb

By default, the 2.6 built-in influx upgrade command hilariously migrates the 1.8 database into your user home directory :joy:. This is why you must run the command as the influxdb user, which has its $HOME set to \var\lib\influxdb, which is where the data resides.

Either use sudo like the guide recommends, or set a default shell for that user and login directly, or use any other Linux tricks you know to accomplish the same.

The upgrader stores data in the wrong location

The new server version expects files to be at /var/lib/influxdb, but influx upgrade incorrectly migrates to and initializes files into a subdirectory /var/lib/influxdb/.influxdbv2/. You need to move everything within .influxdbv2 up one directory. If you don’t fix this, the web portal will ask you to create a new admin account the first time you visit, and you won’t get any of your old data.

Hidden token options on the command line

The built-in command line help only showed me the -h option to view the help, but not the other options. On the website, I found that most commands have a -t option that is essentially required, otherwise you’ll keep getting 401: Unauthorized errors. Use -t TOKEN where TOKEN is a long magic string you can find in /var/lib/influxdb/configs.

Conclusion

Hopefully something here will help someone out there who’s struggling to migrate their data to the latest version of Influx. And if anyone’s curious, after spending a few days and much frustration tracking down these upgrade-breaking quirks, I was able to run my benchmark and see that InfluxDB2 is still limited to slow single-threaded queries, but it is about 5% faster than InfluxDB, so that’s something.

2 Likes

Hello @influxdan,

I actually recommend holding off on upgrading to 2.x. Instead I’d wait for 3.x OSS to release later this year.
It offers significant performance benefits and is centered around interoperability so you don’t have to learn Flux. Instead you can use InfluxQL and SQL with 3.x OSS.

InfluxDB 3.0 is up to 45x Faster for Recent Data Compared to InfluxDB Open…

In this post, we look at recent benchmarks comparing InfluxDB 3.0 to InfluxDB Open Source (OSS) 1.8. influxdb

So, it will be possible to upgrade from 1.8 direct to 3.x?

Antony.

Thanks I was just thinking about going to 2.

Do you still need 2x the storage, or is this now being done in shards? 1x storage +8GB or so?