Tsi1 index uses more memory than without, preventing upgrade to influx2

InfluxDB 1
1

Background: I’m running influxdb in docker on a Raspberry Pi 3 Model B Rev 1.2 with 1Gb ram for tracking some home automation variables. My database backup is 50-100MB and I write 5-6 entries per minute have in total ~10 series, and I query the database once or twice a week.

I recently decided to jump from influxdb version 1.7 to 2.6. Initial testing on my laptop went fine, but when I tried to deploy to the raspberry I noticed that memory consumption had jumped from ~50MB reserved memory to ~300MB reserved memory which didn’t fit with my other containers. I tried tuning all memory related config options, but could not make any significant dents into the memory usage. These were my final settings:

  - INFLUXD_HTTP_READ_TIMEOUT=20s
  - INFLUXD_HTTP_WRITE_TIMEOUT=20s
  - INFLUXD_METRICS_DISABLED=true
  - INFLUXD_PPROF_DISABLED=true
  - INFLUXDB_REPORTING_DISABLED=true
  - INFLUXD_QUERY_INITIAL_MEMORY_BYTES=262144
  - INFLUXD_QUERY_MEMORY_BYTES=5242880
  - INFLUXD_QUERY_QUEUE_SIZE=3
  - INFLUXD_STORAGE_CACHE_MAX_MEMORY_SIZE=5242880
  - INFLUXD_STORAGE_CACHE_SNAPSHOT_MEMORY_SIZE=5242880
  - INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=1
  - INFLUXD_STORAGE_MAX_INDEX_LOG_FILE_SIZE=262144
  - INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=1
  - INFLUXD_STORAGE_SERIES_ID_SET_CACHE_SIZE=0

Restricting the amount of available memory via docker just terminated influxdb.

I then went back to influxdb 1.7 and tried turning on INFLUXDB_DATA_INDEX_VERSION=tsi1 and again the memory jumped to around 300MB.

I would like to use influxdb 2, since I think the API is much nicer, so I’m wondering both why tsi1 is using more memory, and also if there is any way to reduce memory usage with tsi1 so I can switch to influxdb2?

2

Hello @goodbyte,
Unfortunately there is no way to operate memory usage with influxdb 2.x this is a feature that we’re looking into adding with the new storage engine eventually.
I’m not sure why it’s consuming more memory.
Perhaps you could offload some of the work of your queries to tasks that run periodically. But outside of that I don’t know that there’s much you can do. Im sorry.

3

There are no queries running while I made those measurements. Thanks anyway, I’ll retest with a later release when the new storage engine is available.