Memory usage is remaining high after a finished query

[Ruke_Chen]

Hello All,

We are using InfluxDB v1.8.3 (git: 1.8 563e6c3) as our time-series database in our startup. I create a github issue influxd keeps high memory usage · Issue #21965 · influxdata/influxdb · GitHub, We don’t have any problem for writing data to the database. What we faced is after several queries to the database, our Memory (RAM)usage goes up and never comes down even after the query has finished.

So if we continue to execute a new query to Influxdb, it uses more memory, and at some point, it causes memory and OOM occurs.

[Anaisdg]

Hello @Ruke_Chen,
Could this be related to your issue?

High memory usage on TSI mode

We are running a stress test with importing certain amount of history records from scratch using InfluxDB v1.8.3 OSS. The goal of the test is to discover the needed parameters to set and the hardware spec in order to gain the best out of InfluxDB. By switching to TSI mode, we expect to decrease memory usage. Unfortunately, we’re pretty much stuck in the same situation where the memory usage is still high and InfluxDB service was up and down periodically. We’ve tried some ways mentioned in other …

[Ruke_Chen]

Hi @Anaisdg

I had already checked out this topic, and added this configuration GODEBUG=madvdontneed=1 in /etc/default/influxdb, restarted influxd, and did my test on 2021/08/03. The result is the same. Memory usage is remaining high after querying.

Thanks

[wbaker]

@Ruke_Chen

1.8.3 (git: 1.8 563e6c3) as our time-series database in our startup. I create a github issue influxd keeps high memory usage · Issue #21965 · influxdata/influxdb · GitHub, We don’t have any problem for writing data to the database. What we faced is after several queries to the database, our Memory (RAM)usage goes up and never comes down even after the query has finished.

So if we continue to execute a new query to Influxdb, it uses more memory, and at some point, it causes memory and OOM occurs.

@Ruke_Chen - are you running influxd in a docker container? I wasn’t 100% sure from your description here or in the issue. We’ve got a few open issues related to memory usage, particularly running docker containers, that we are investigating. It’s not a quick fix, but I appreciate you taking the time to write up such clear instructions on how to reproduce the issue so that we can include it in our investigation!

[Ruke_Chen]

Hi @wbaker ,

Yes. I am running influxd in alpine docker container. (influxdata-docker/influxdb/1.8/alpine at 0d341f18067c4652dfa8df7dcb24d69bf707363d · influxdata/influxdata-docker · GitHub).
Within this docker file, we build our own docker image which contains InfluxDB 1.8.3. The difference is we change the index-version to “tsi1”. The detail step is mentioned in influxd keeps high memory usage · Issue #21965 · influxdata/influxdb · GitHub. Although I make a simple modification for generating testing data, I think it doesn’t matter. Anyway, if you need further information, please let met know, Thanks

[Ruke_Chen]

Hi,

Any update?

Thanks

[Ruke_Chen]

Hi @Anaisdg and @wbaker

Andy good news?

Thanks

[wbaker]

@Ruke_Chen - we’re working pretty hard on performance improvements and memory usage is on the radar. I don’t have any other news right now but we definitely haven’t forgotten about it!

[Ruke_Chen]

Hi @wbaker

I use the following command to launch influxd

env GODEBUG=madvdontneed=1 /usr/bin/influxd

The memory usage looks much better. The memory usage of influxd would increase while querying, but after a while the memory usage is decrease.

Thanks