Slow Queries, not using all memory

I am testing with influx 2.0 compared to influx 1.8 and I see a significant increase in query times between the two. A similar query on 1.8 runs for about 3-4 seconds where on 2.0 it runs for 15-17 seconds.

The piece I find strange is there is no difference in memory utilization that I can see on the machine when the query is running vs when it is not running. The box has ~64G of memory but influx will only use about 21G of memory. I have the query limit set to query-memory-bytes = 3221225472 but the memory usage does not change when a query is ran. The DB size is still small, only about 6.5G at the moment

Here is the query

from(bucket: "prometheus")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_field"] == "lb_requests_completed_by_code")
  |> filter(fn: (r) => r["code"] == "1xx" or r["code"] == "2xx" or r["code"] == "3xx" or r["code"] == "4xx" or r["code"] == "5xx")
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
  |> yield(name: "last")

Current config

bolt-path = "/var/lib/influxdb/influxd.bolt"
engine-path = "/data/influxdb/engine"
http-bind-address = "omitted"
log-level = "info"
storage-cache-max-memory-size = 32212254720
query-memory-bytes = 3221225472

Versions

Influx CLI 2.0.7 (git: 2a45f0c037) build_date: 2021-06-04T19:17:40Z
CentOS Linux release 8.1.1911 (Core)
1 Like

Hello @tchameroy,
Welcome! Unfortunately I don’t have the answer you’re looking for. Essentially, we traded speed for features, sorry. However, the Flux team is constantly making improvements to performance. Also, InfluxDB v2 will support querying with InfluxQL, SQL, and Python out of the box in the future. Read about it here:

Finally, you can currently query v2 with InfluxQL with the forward compatibility endpoints. You might find increased performance that way: