Visualizing downsample data influxdb 2.0

Grafana
1

Hi, does anyone have any idea on how to visualize the downsampled data from influxdb 2.x to Grafana? I have created tasks to aggregate the raw data, but still at a dead end on how to visualize the downsampled data dynamically using the time range control just like in influxdb 1.x…

2

Hello @faiz,
Welcome! What does your task look like? Where are you writing the downsampled data to?
Can you please share your task and the query you’re using to try and visualize the results? Are you able to visualize the results of your task with InfluxDB?

4

I am writing the downsampled data into a different bucket . The downsampled data can be visualized with InfluxDB by itself but it cannot be displayed dynamically in Grafana i.e. only display downsampled data when the time range selected is beyond the source bucket retention period.

Task I am running:
|option task = {name: “cq_mem_20sto30m”, every: 30m, offset: 10m}
from(bucket: “telegraf”)
|> range(start: -task.every)
|> filter(fn: (r) =>
(r._measurement == “vsphere_vm_mem”))
|> filter(fn: (r) =>
(r._field == “active_average” or r._field == “swapin_average” or r._field == “latency_average” or r._field == “swapout_average” or r._field == “usage_average” or r._field == “vmmemctl_average”))
|> aggregateWindow(every: task.every, fn: mean)
|> set(key: “_measurement”, value: “:MEASUREMENT”)
|> group(columns: [“vmname”])
|> filter(fn: (r) =>
(exists r._value))
|> to(org: “organization”, bucket: “telegraf_ds”)|

Previously, we did this method to select which data to use (raw or downsampled) in InfluxDB 1.x. Is there any equivalent approach to this?
|CREATE RETENTION POLICY “persistent” ON mydb DURATION INF REPLICATION 1
USE mydb
INSERT INTO persistent grafana_rp rp=“1_Minute” 604800000000000
INSERT INTO persistent grafana_rp rp=“10_Minutes” 2592000000000000|

5

I am writing the downsampled data into a different bucket . The downsampled data can be visualized with InfluxDB by itself but it cannot be displayed dynamically in Grafana i.e. only display downsampled data when the time range selected is beyond the source bucket retention period.

Task I am running:

option task = {name: “cq_mem_20sto30m”, every: 30m, offset: 10m}
from(bucket: “telegraf”)

range(start: -task.every)
filter(fn: (r) =>
(r._measurement == “vsphere_vm_mem”))
filter(fn: (r) =>
(r._field == “active_average” or r._field == “swapin_average” or r._field == “latency_average” or r._field == “swapout_average” or r._field == “usage_average” or r._field == “vmmemctl_average”))
aggregateWindow(every: task.every, fn: mean)
set(key: “_measurement”, value: “:MEASUREMENT”)
group(columns: [“vmname”])
filter(fn: (r) =>
(exists r._value))
to(org: “organization”, bucket: “telegraf_ds”)|

Previously, we did this method to select which data to use (raw or downsampled) in InfluxDB 1.x. Is there any equivalent approach to this?
CREATE RETENTION POLICY "persistent" ON mydb DURATION INF REPLICATION 1 USE mydb INSERT INTO persistent grafana_rp rp="1_Minute" 604800000000000 INSERT INTO persistent grafana_rp rp="10_Minutes" 2592000000000000

6

Hello @faiz,
I’m sorry, I’m not that familiar with grafana so I’m having trouble understanding.
Can you query data from that downsampled bucket in grafana?

7

Yes, I can query the data from the downsampled bucket in grafana. What I’m looking for is, does InfluxDB 2.0 provides the equivalent to this where we create a cq with one function and writing the results to another retention policy?

> CREATE CONTINUOUS QUERY minnie_jr ON world BEGIN SELECT min(mouse) INTO world."7days".min_mouse FROM world."1day".zoo GROUP BY time(30m) END
8

I want to share my implementation to this problem. Getting downsampled data to show up in Grafana graphs was the hard part of implementing downsampling but I found a way to do it with only needing three line changes to each panel. I wrote up a guide with the Flux code and how to use it on my website: InfluxDB v2 Downsampling — Robpol86.com

image
image2172×814 97.1 KB

This is the before and after for one panel:

// Before
from(bucket: "${BUCKET}")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r.host == "${HOST}")
  |> filter(fn: (r) => r._measurement == "mem")
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)

// After
dsQuery = (bucket, start, stop) =>
from(bucket)
  |> range(start, stop)
  |> filter(fn: (r) => r.host == "${HOST}")
  |> filter(fn: (r) => r._measurement == "mem")
  |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false)
${dsPost}

Basically I put a Flux script in a Grafana query variable that refreshes every time the dashboard time range changes. This only works with Flux, so InfluxDB v2 is supported but not v1 or v3.

1 Like