Write back results from query (e.g. every hour)

Guys, just trying to improve performance of my queries where I do have a quite a lot queries plus some of them are quite complex and time consuming to calculate. So when I have a graph showing last 3 months average it takes minutes to calculate.

Here is example of simple query and I am thinking of using flux to run every hour and write back the results to influxdb. Therefore I can have just easy query to calculate average of all the values when using graphs for more than 1 hour and only keep calculating results up to 1 hour.

Does anybody knows how am I able to run query e.g. every hour and write back the results to influxdb with new field i.e. r_value_avg_quality ? What are the options here ?

I expect that using of python for example may be one of the possible way…if so, any example for such a cases ?

Thanks

cycle = from(bucket: "test")
  |> range($range)
  |> filter(fn: (r) =>
    r._measurement == "fautest" and
    r.trigger =~ /cycle|interval/ and
    r._field == "countfld")
|> group(columns: ["_measurement", "_field"])
|> sum(column: "_value")


|> map(fn:(r) => ({
     r with _time: r._time,
     _value_zero:
 if r._value == 0 then "reduce"
     else "keep"
   })
)
|> filter(fn: (r) =>
    r._value_zero == "keep")
    
reject = from(bucket: "test")
  |> range($range)
  |> filter(fn: (r) =>
    r._measurement == "fautest" and
    r.trigger =~ /reject|interval/ and
    r._field == "countfld")
|> group(columns: ["_measurement", "_field"])
|> sum(column: "_value")

quality = join(tables:{reject:reject, cycle:cycle}, on:["_start"])
|> map(fn:(r) => ({
_value: ((float(v: r._value_cycle) - float(v: r._value_reject)) / float(v: r._value_cycle)) * 100.00 }))

quality|> yield()

@salvq What version of InfluxDB are you using?

@scott I am running v2.0.0-beta.4

Oh, perfect. Just use the to() function.

// ...
  |> to(bucket: "bucket-name", org: "org-name")

I’d recommend setting up a task that downsamples data on a regular schedule.

This is great :+1: let me study and try it out

Just a question for influxdb version, in several older servers I do have older version of Influxdb < 1.7 or lower, any procedure to make an upgrade to 2.0 ?

For some I need to keep 1.7 as they are in production mode but several older I can upgrade as they are not critical data storages.

@salvq, we are in the process of building out a graceful 1.x to 2.0 migration path.

In the mean time, you could export the data in your 1.x instances as line protocol (blog post about it here, remove the comments and the CREATE DATABASE ... statement from the export file, then write that file to your 2.0 instance.