Hello! I had a strange behavior of InfluxDB last night. The main DB is about 640GB, and the partition to it is about 788G. When Influx started full compaction of database group, the CPU went mad, Heap in Use went mad and after a while disk got full and the Influx just stopped working. I am using InfluxDB 1.2.1.
My question: are there any basic recommendations about free disk space and how much I need it for 640GB DB to go through compaction process successfully?
Marijus
Hi Marijus,
Full group compactions create temporary duplicates of the tsm files inside the shard being processed.
In the worst case scenario, if all of your shards have been modified in the last cycle and are picked up for full compaction at the same time, you would need 2x the space used at rest.
Typically, you need to have at least twice the size of your current hot shards in reserve for compactions to run.
Hello Kostas,
Thank You for the reply.
Looks like I know what caused the problem. I was rewriting/changing some historical data to my InfluxDB, as I understand it was compacted already and since it got a bunch of new data it has to do the full compaction process again?
By the way is there a way to determine current hot shards?
Marijus
