Sorry, wrote that out too quickly. I updated the query to reflect those changes.
aggregateWindow
is a really powerful tool that segments data by time, runs some kind of aggregation, then removes the time-based segmentation to output aggregated/downsampled data. The fn
parameter defines the function used to aggregate the data.
aggregateWindow
’s fn
parameter is designed to accept any function with column
and tables
parameters (aggregate and selector functions all have these, which is why you can use them with aggregateWindow
).
In your particular use case, you need to run multiple aggregations on each time window. To do this, for the fn
parameter, I gave it an anonymous function with the two parameters required by aggregateWindow
– column
and tables
.
(tables=<-, column) => tables |> distinct() |> count()
tables
is set to <-
which represents all input data. column
just needs to be present, but it doesn’t necessarily need to be defined because it inherits the value of aggregateWindow
’s column
parameter (_value
by default). So the anonymous function takes each input table (now segmented by time) and runs distinct()
and count()
on the _value
column.