Sudden 100% swap usage on influxdb

We have a situation where the Swap has suddenly spiked up to 100%. When does such a situation occur and how do we debug this? During such scenario, the VM become unresponsive or slow.

If I check the free -lh I see as follows:

influxdb3/8b37314c-a9ff-47a9-9ccb-86e67f1fec1e:~# free -lh
               total        used        free      shared  buff/cache   available
Mem:            61Gi        40Gi       425Mi       614Mi        20Gi        22Gi
Low:            61Gi        61Gi       425Mi
High:             0B          0B          0B
Swap:            9Gi         9Gi       0.0Ki

Available memory is 22 GB (which I guess is free+shared+buff). But if 22Gb is free why is 100% swap used?
Also CPU during didn’t seem high.
The VM runs 16CPU and 64GB Memory. The load went up to 10. So I assume the system isn’t overloaded either.

When does influx prefer to take swap over free memory? and how do we debug if it’s a specific query or ingestion that is causing the issue

Running of influxdb 1.8 on 22.04.1-Ubuntu

Hello @vipinvkmenon,
Memory concerns are a long standing concern with InfluxDB. What version of InfluxDB are you using?
Do you have any:

  • Long-running queries or queries that require a lot of memory?
  • High write rates that could cause InfluxDB to consume more memory temporarily

Previous versions of InfluxDB are notorious for grabbing all available RAM and there isn’t operator control over that :frowning: