aggregateWindow() drops needed columns

InfluxDB 2
1

I am trying to write a data consolidation task with aggregateWindow(), and am stuck with flux’s behavior of dropping unreferenced columns with AggregateWindow. My specific issue is the “units” string is needed elsewhere in my system.

Is there any way to avoid this behavior natively, or am I forced to do some kind of lookup table from remaining data to try to recover the lost columns?

There is another odd behavior of flux sometimes dropping columns and sometimes not dropping them, but I don’t have enough information to tell when/why it is happening.

Thanks in advance!

2

What is the look of your data and what is your query ?
What do you mean by “unreferenced columns” ? Is your colums another value ?
I don’t see any drop of columns on my side with aggregate windows and group()/keep()/drop() are working quite nicely for me.

3

My data before aggregateWindow() looks like this (rows truncated and time windows were different so the results don’t match):

table _measurement _value _start _stop _time _unit device_class_str entity_id friendly_name state_class_str
0 sensor.ws_pressure_absolute 1012.92 2023-03-15T13:06:53.490Z 2023-03-15T13:07:53.490Z 2023-03-15T13:07:14.000Z hPa pressure ws_pressure_absolute Atmospheric Pressure measurement

After the aggregateWindow() it looks like this:

table _measurement _value _start _stop _time entity_id friendly_name
0 sensor.ws_dewpoint_outside 20.62 2023-03-15T13:11:51.718Z 2023-03-15T13:21:51.718Z 2023-03-15T13:15:00.000Z ws_dewpoint_outside Outside Dewpoint (Roof)
1 sensor.ws_humidity_outside 78 2023-03-15T13:11:51.718Z 2023-03-15T13:21:51.718Z 2023-03-15T13:15:00.000Z ws_humidity_outside Outside Humidity
2 sensor.ws_indoor_dewpoint 20.92 2023-03-15T13:11:51.718Z 2023-03-15T13:21:51.718Z 2023-03-15T13:15:00.000Z ws_indoor_dewpoint Indoor Dewpoint

Note the missing columns for _unit, device_class_str, state_class_str
My full query is the following; the results for input and output just comment out the aggregateWindow():

    from(bucket: "Hassbucket")
        |> range(start: -10m)
        |> filter(fn: (r) => r["_measurement"] =~ /^sensor.ws_*/)
        |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value")
        |> drop(columns: ["icon_str", "domain", "source"])
        |> rename(columns: {value: "_value", unit_of_measurement_str: "_unit"})
        |> aggregateWindow(every: 5m, fn: mean, createEmpty: false)
        |> yield()