What's with data in WAL-Files when running a Backup

Hello everyone,

i’m new to influxdb and the last few days i try to understand how influxdb is working behind the scenes. I think i got a rough understanding about this, but a few things are not clear to me and i didn’t found an answer on the internet.

I’m using influxdb 1.8 and playing around a little bit with backup and restore. In the documentation there is a Note: " Note: backup ignores WAL files and in-memory cache data.".

As i read so far, influxdb write’s data to the WAL-Files and after a given time or a specific amount of space the data of the WAL-Files is written to TSM-Files.

Does this mean, if i’m running a Backup via “influxd backup”, that all data that is currently stored in the WAL-Files is lost?

Thanks!

1 Like

Hello @Chris_272,
Good question. The backups start at the most recently written TSM generation file written to disk.

Hello @Anaisdg,

thanks for your answer, but this doesn’t really answer my question. But

start at the most recently written TSM generation file written to disk.

also sounds like the data in the WAL-Files is lost. So say i have two databases and each database is short before converting WAL-Files to TSM-Files (let’s say 100MB of Data in WAL-Files for each of these two databases), than this would mean, if i run a backup at that time 200MB of Data will be lost (maybe more when using more databases).

This seems a little bit odd fore someone like me, who is coming out of the “SQL-Server”-World :sweat_smile:.

When i have time, i’m going to test it out and if this is really the case :sweat_smile:

For everyone who cares, i finally got a little time to test and it looks like, that when you run a backup, that the .wal-Files, are being written to a .tsm-File before the backup runs.

Weird that nobody cares or knows about this, but now i know and i’m happy with it :slight_smile:

I copied the folder of influx database folder where wal was also stored and also used the influxd command it worked.