One or many databases for memory performance

influxdb

#1

Hi! I’m using InfluxDB to store real-time data from various energy monitors. I’ve had out of memory problems and would like to know if I’ve missed something obvious.

My hardware is 2Gb t2.medium AWS instance, with 8 databases. Each database is written to via HTTP at 10s intervals in a batch. Each batch contains about 10 readings. Based on the “hardware recommendations” this too little memory, but quite low requirements.

Generally the posting of data is fine. Memory problems occurs on compaction (solved via config file " max-concurrent-compactions = 1") and continuous queries (unsolved). My schema is generally:

use iotalcibne
Using database iotalcibne
show measurements
name: measurements
name
----
hourly
iotawatt
select * from iotawatt limit 10
time                 PF    Volts  Watts   ct                device  units
----                 --    -----  -----   --                ------  -----
2018-12-15T08:10:00Z              255.01  Lights1           ilcibne Watts
2018-12-15T08:10:00Z       269.43         Voltage           ilcibne Volts
2018-12-15T08:10:00Z 0.748                Lights1PF         ilcibne PF
2018-12-15T08:10:00Z              0.11    Power1Kitchen     ilcibne Watts
2018-12-15T08:10:00Z              0.23    Power4SouthOffice ilcibne Watts
2018-12-15T08:10:00Z              0.24    Power5OpenPlan    ilcibne Watts
2018-12-15T08:10:00Z              0.22    Power2Rack        ilcibne Watts
2018-12-15T08:10:00Z              1012.44 TotalCircuits     ilcibne Watts
2018-12-15T08:10:00Z              1011.41 TotalLights       ilcibne Watts
2018-12-15T08:10:00Z              756.4   Lights2           ilcibne Watts

I think this is ‘good’ for memory usage: there are limited numbers of distinct tag values under “ct”. However, I have 8 databases with different names, each like this. The series count is 120

My question (finally!): from a memory-use / performance point of view, would I be better off have one single database? (This would have an additional tag-name “input-device” which would identify which of the 8 units posted the data.)

My “SHOW STATS” is attached: stats.txt (898.0 KB)


#2

Perhaps this was TLDR, so I’ll rephrase as:

Is it better to have a single database instead of multiple databases for memory consumption?

Thanks in advance, Brett.