How to properly get sum per group?

InfluxDB 2
1

I’m scraping InfluxDB’s own /metrics endpoint into a bucket.

Now I want to see the disk space each bucket takes up over time.

The data looks like this:

image
image2494×435 120 KB

The query I wrote is:

from(bucket: "default")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "storage_tsm_files_disk_bytes")
  |> filter(fn: (r) => r["_field"] == "gauge")
  |> group(columns: ["_time", "bucket"])
  |> sum()
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)

This seems to be wrong in some way because I’m getting more than one row per bucket and time:

image
image1564×435 62.8 KB

How can I query this correctly?

Extra questions:

  • Is storage_tsm_files_disk_bytes the correct measurement to find the disk size of a bucket?
  • When I omit |> sum() I still see some values in the chart. What do these values then represent, are they arithmetic means?
2

@AndreKR From what I can see, this query should work.

Not directly, but you should be able to use it to calculate the disk size of a bucket. This metric reports the disk size of shards, but your approach of grouping by the bucket and summing should return the sum of all shards associated with the bucket.

These likely just represent the size of each shard over time.

I think to actually get the results you’re looking for (changes over time; not the current state), you’d do this:

from(bucket: "default")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "storage_tsm_files_disk_bytes")
  |> filter(fn: (r) => r["_field"] == "gauge")
  |> group(columns: ["bucket"])
  |> aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false)
3

That query gives completely different results depending on whether I query the last hour or the last week. I think that is because aggregateWindow(every: v.windowPeriod, fn: sum, createEmpty: false) will sum up more than one report when v.windowPeriod becomes longer than the scrape interval (which I think is 10 seconds).

4

Ok, you should probably explicitly set every to 10s. v.windowPeriod is derived from the the time range queried divided by the number of pixels available in the cell so the dashboard only displays enough data to render a point per pixel. You may lose some query efficiency as you query larger time ranges, but the behavior should stay the same.