Influxdb have performance problem when query data

1

Hello, I have a problem when I read data from influxdb by http. I find my influxdb use too many memory and cpu of my computer.And the disk io is in high usage,too.But I find My frequency of reading data from influxdb is not high.It’s highest frequency is 116qps.I’m very curious about why influxdb use too many resources of my computer.

This is my requests frequency:

64tu
64tu.png835×261 16.7 KB

This is system info when reading from influxdb
64IO
64IO.png854×318 24.2 KB

mem
mem.png888×190 34.8 KB

And my request is like this

select *,day_epoch::integer, created_at::integer from one_year.load where symbol = 'node1' order by desc limit 6&epoch=ms

this is the debug info I created by folloing commands while I quering data from influxdb

curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"

curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt

iostat.txt (10.8 KB)
profiles.tar.gz (977.3 KB)
vars.txt (98.5 KB)

Do you have any suggestions to help performance tunning?

2

which version of influx? How many data points are you writing?

Based on my limited knowledge

Whenever i have had issues with memory its down to high cardinality. You can index teh data on disk if you have SSD (i think. They were working on something like this) It’s mentioned here.

That could help. There is also a decent tool on GitHub by one of the guys from Influx. You can use it to bench mark Influx on your kit. I can’t find the link but the tool is called INCH (Influx Benchmark).

Also, Grafana is terribly slow imo when viewing the data in the browser so if you are hosting the TICK stack and Grafana on the same machine.

3

The version of influx is 1.3.8, and the datapoints I write is over 600 millions. I will see if INCH is helpful to my problem.
Thanks for your reply!

4

Hello, I find inch tests the performance while writing points into influxdb, but my problem happened at the time of quering data.Do you have antother tools help with my problem?

5

Hi, apologies for delayed reply.

I’m afraid i can’t offer much more i’m still wrapping my head around it myself. The later versions of Influx should allow you to send some of the data to disk which should free up the memory.

It might be better if you were to sample the data and downsize it, then query the sampled data. I don’t know if that would help, i haven’t tried it myself - I’m just prepping to update to the latest version to try and fix the memory issues i’m having.

EDIT: Downsampling and data retention | InfluxDB OSS 1.4 Documentation

6

HI mate,
single instance influx db is very good at handling super frequent write (insert a vast number of lines data per second).
However, I have to say single instance influx-db is super weak for frequent read but powerful in occasional reads.
when you read very fast without waiting between each read query, write will be significantly slowed down.

So, I usually sleep a while (100ms) between each query that has a very big time range (up to 1d).
you can use enterprise or cloud influx-db that provides you r/w split but it is reasonably expensive (thousands dollars for 2 or 3 instances per year).