Creating tasks, agreggates - help

InfluxDB 2
1

Hi,

this is my first try with influxdb tasks.

So i have my main “mydb”.
Than i have 3 tasks “aggregates5min”, “aggregatesHourly”, “aggregatesDaily”.
Aggregates5min was created on mydb.
AggregatesHourly was created on Aggregates5min.
AggregatesDailywas created on AggregatesHourly.

I am getting data in all my buckets aggregates but i think they are off. Let me explain on temperature example reading.

If i compare MAX temperature reading between aggregates5min and aggregatesHourly i would expect to have the same MAX temp in some period. But this is not the case. In aggregates5min max temp is 65C in aggregatesHourly is let say 62C. So the problem is probably how i create this aggregates.

Aggregates5min from mydb (This one is ok. Correct data.)

option task = {name: "aggregates5min", every: 5m}

data =
    from(bucket: "mydb")
        |> range(start: -5m)
        |> filter(fn: (r) => r["_measurement"] == "sensors")
        |> filter(
            fn: (r) =>
                r["device"] == "raspberry-pi-4" or r["device"] == "water-irrigation" or r["device"]
                    ==
                    "weather-station",
        )
        |> filter(
            fn: (r) =>
                r["_field"] == "available_memory" or r["_field"] == "cpu_temperature" or r["_field"] == "waterflow"
                    or
                    r["_field"] == "battery" or r["_field"] == "deviceboot" or r["_field"] == "infrared" or r["_field"]
                    ==
                    "light" or r["_field"] == "lux" or r["_field"] == "moisture" or r["_field"] == "rain" or r["_field"]
                    ==
                    "temperature",
        )
        |> filter(
            fn: (r) =>
                r["sensor"] == "cpu_tmp-1" or r["sensor"] == "cpu_available_memory" or r["sensor"] == "flow-1"
                    or
                    r["sensor"] == "flow-2" or r["sensor"] == "battery-procent" or r["sensor"] == "battery-units"
                    or
                    r["sensor"] == "battery-voltage" or r["sensor"] == "deviceboot-1" or r["sensor"] == "ir-1"
                    or
                    r["sensor"] == "light-1" or r["sensor"] == "lux-1" or r["sensor"] == "moisture-1" or r["sensor"]
                    ==
                    "moisture-2" or r["sensor"] == "rain-1" or r["sensor"] == "reboot-tmp-1" or r["sensor"] == "tmp-1"
                    or
                    r["sensor"] == "tmp-2" or r["sensor"] == "tmp-3",
        )

data
    |> aggregateWindow(every: 5m, fn: mean, createEmpty: false)
    |> set(key: "_field", value: "mean")
    |> yield(name: "mean")
    |> to(bucket: "aggregates5min")

data
    |> aggregateWindow(every: 5m, fn: max, createEmpty: false)
    |> set(key: "_field", value: "max")
    |> yield(name: "max")
    |> to(bucket: "aggregates5min")

data
    |> aggregateWindow(every: 5m, fn: min, createEmpty: false)
    |> set(key: "_field", value: "min")
    |> yield(name: "min")
    |> to(bucket: "aggregates5min")

data
    |> aggregateWindow(every: 5m, fn: sum, createEmpty: false)
    |> set(key: "_field", value: "sum")
    |> yield(name: "sum")
    |> to(bucket: "aggregates5min")

AggregatesHourly from Aggregates5min

option task = {name: "aggregatesHourly", every: 1h}

data =
    from(bucket: "aggregates5min")
        |> range(start: -1h)
        |> filter(fn: (r) => r["_measurement"] == "sensors")
        |> filter(
            fn: (r) =>
                r["device"] == "raspberry-pi-4" or r["device"] == "water-irrigation" or r["device"]
                    ==
                    "weather-station",
        )
        |> filter(fn: (r) => r["_field"] == "mean")
        |> filter(
            fn: (r) =>
                r["sensor"] == "cpu_tmp-1" or r["sensor"] == "cpu_available_memory" or r["sensor"] == "flow-1"
                    or
                    r["sensor"] == "flow-2" or r["sensor"] == "battery-procent" or r["sensor"] == "battery-units"
                    or
                    r["sensor"] == "battery-voltage" or r["sensor"] == "deviceboot-1" or r["sensor"] == "ir-1"
                    or
                    r["sensor"] == "light-1" or r["sensor"] == "lux-1" or r["sensor"] == "moisture-1" or r["sensor"]
                    ==
                    "moisture-2" or r["sensor"] == "rain-1" or r["sensor"] == "reboot-tmp-1" or r["sensor"] == "tmp-1"
                    or
                    r["sensor"] == "tmp-2" or r["sensor"] == "tmp-3",
        )

data
    |> aggregateWindow(every: 1h, fn: mean, createEmpty: false)
    |> set(key: "_field", value: "mean")
    |> yield(name: "mean")
    |> to(bucket: "aggregatesHourly")

data
    |> aggregateWindow(every: 1h, fn: max, createEmpty: false)
    |> set(key: "_field", value: "max")
    |> yield(name: "max")
    |> to(bucket: "aggregatesHourly")

data
    |> aggregateWindow(every: 1h, fn: min, createEmpty: false)
    |> set(key: "_field", value: "min")
    |> yield(name: "min")
    |> to(bucket: "aggregatesHourly")

data
    |> aggregateWindow(every: 1h, fn: sum, createEmpty: false)
    |> set(key: "_field", value: "sum")
    |> yield(name: "sum")
    |> to(bucket: "aggregatesHourly")

I have also tried to replace this line:

|> filter(fn: (r) => r[“_field”] == “mean”)

with this line:

|> filter(fn: (r) => r[“_field”] == “mean” or r[“_field”] == “max” or r[“_field”] == “min” or r[“_field”] == “sum”)

but than i get values instead of around 60 let say around 300

So what is wrong if anything?

2

Example for cpu raspberry 4 temperature:

mydb(ok):

agreggates5min(ok):

1784×370 39.9 KB

aggregatesHourly(not ok weird numbers; it should be around 60C):
1794×383 34.3 KB

3

I just dont get it … why is see difference?

My main bucket is showing max peak temp at 64.7C

1187×360 23.6 KB

And my bucket aggregates5min showing max peak at temp 65.7C
1053×371 38 KB

Why is the difference? Max value from aggregates5min should be the same (64.7C) as peak value from main bucket? What i am not seeing?

I am using Task fro 5min aggregate:

option task = {name: "aggregates5min", every: 5m}

data =
    from(bucket: "mydb")
        |> range(start: -5m)
        |> filter(fn: (r) => r["_measurement"] == "sensors")
        |> filter(
            fn: (r) =>
                r["device"] == "raspberry-pi-4" or r["device"] == "water-irrigation" or r["device"]
                    ==
                    "weather-station",
        )
        |> filter(
            fn: (r) =>
                r["_field"] == "available_memory" or r["_field"] == "cpu_temperature" or r["_field"] == "waterflow"
                    or
                    r["_field"] == "battery" or r["_field"] == "deviceboot" or r["_field"] == "infrared" or r["_field"]
                    ==
                    "light" or r["_field"] == "lux" or r["_field"] == "moisture" or r["_field"] == "rain" or r["_field"]
                    ==
                    "temperature",
        )
        |> filter(
            fn: (r) =>
                r["sensor"] == "cpu_tmp-1" or r["sensor"] == "cpu_available_memory" or r["sensor"] == "flow-1"
                    or
                    r["sensor"] == "flow-2" or r["sensor"] == "battery-procent" or r["sensor"] == "battery-units"
                    or
                    r["sensor"] == "battery-voltage" or r["sensor"] == "deviceboot-1" or r["sensor"] == "ir-1"
                    or
                    r["sensor"] == "light-1" or r["sensor"] == "lux-1" or r["sensor"] == "moisture-1" or r["sensor"]
                    ==
                    "moisture-2" or r["sensor"] == "rain-1" or r["sensor"] == "reboot-tmp-1" or r["sensor"] == "tmp-1"
                    or
                    r["sensor"] == "tmp-2" or r["sensor"] == "tmp-3",
        )

data
    |> aggregateWindow(every: 5m, fn: mean, createEmpty: false)
    |> set(key: "_field", value: "mean")
    |> to(bucket: "aggregates5min")
    |> yield(name: "mean")

data
    |> aggregateWindow(every: 5m, fn: max, createEmpty: false)
    |> set(key: "_field", value: "max")
    |> to(bucket: "aggregates5min")
    |> yield(name: "max")
data
    |> aggregateWindow(every: 5m, fn: min, createEmpty: false)
    |> set(key: "_field", value: "min")
    |> to(bucket: "aggregates5min")
    |> yield(name: "min")
data
    |> aggregateWindow(every: 5m, fn: sum, createEmpty: false)
    |> set(key: "_field", value: "sum")
    |> to(bucket: "aggregates5min")
    |> yield(name: "sum")`Preformatted text`
4

Hello @mojInfluxDb,
Welcome!
Thanks for giving tasks a try!
I agree that’s super weird I don’t know what’s happening.

Can you please share some data from aggregates5min like query it and limit to the last hour and share with me in annotatedCSV?

When you run the AggregatesHourly query outside of a task do you also get unexpected values? I.E when you run:

data =
    from(bucket: "aggregates5min")
        |> range(start: -1h)
        |> filter(fn: (r) => r["_measurement"] == "sensors")
        |> filter(
            fn: (r) =>
                r["device"] == "raspberry-pi-4" or r["device"] == "water-irrigation" or r["device"]
                    ==
                    "weather-station",
        )
        |> filter(fn: (r) => r["_field"] == "mean")
        |> filter(
            fn: (r) =>
                r["sensor"] == "cpu_tmp-1" or r["sensor"] == "cpu_available_memory" or r["sensor"] == "flow-1"
                    or
                    r["sensor"] == "flow-2" or r["sensor"] == "battery-procent" or r["sensor"] == "battery-units"
                    or
                    r["sensor"] == "battery-voltage" or r["sensor"] == "deviceboot-1" or r["sensor"] == "ir-1"
                    or
                    r["sensor"] == "light-1" or r["sensor"] == "lux-1" or r["sensor"] == "moisture-1" or r["sensor"]
                    ==
                    "moisture-2" or r["sensor"] == "rain-1" or r["sensor"] == "reboot-tmp-1" or r["sensor"] == "tmp-1"
                    or
                    r["sensor"] == "tmp-2" or r["sensor"] == "tmp-3",
        )

data
    |> aggregateWindow(every: 1h, fn: mean, createEmpty: false)
    |> set(key: "_field", value: "mean")
    |> yield(name: "mean")

?

Also sorry for the delay, the team was onsite last week. Typically we respond much faster.