Small Query is super slow on a fairly large machine

Telegraf
1

Here’s the situation I had:

  • Influxdb version: 1.4. Machine: Linux.
  • The machine is fairly strong, with 4-core CPU, 16GB memory.
  • 5-30 queries/second. Each query is same.
  • Query: select COUNT(*) from Trades where time > 1512498660000000000 and time < 1512498660000000000 + 2h
  • Only about 8000-18000 data points will be returned per query.
  • Very high CPU usage.
  • most of the queries didn’t across the shard.
  • We stored data in ns.

What I observed:

  • The query speed is irrelevant to the size of the result.
  • On small machines, it OOM. On big machine, it used a lot of CPU.

Any insight for this weird situation? I think our workload is fairly small for a time series database.