Table field causing difficulty

I’m trying to use window() and then last() with difference() but it returns no records.

When I take a closer look, I can get aggregateWindow() to work with difference(). It seems there’s a field created called “table”. If I use aggregateWindow, table has a value of zero. If I use window() and then last() in place of aggregateWindow(), table has a number of different values. When I try to use map to rewrite table to zero, I just get a new field with the same name.

What is the field table? And how can I use window(), last(), and difference() together?

The following works:
data
|> aggregateWindow(every: 1h, fn: last)
|> difference()

But how can I get the following to work:
data
|> window(every: 1h)
|> last()
|> difference()

Hello @Unique,
You would have to do:

|> window(every: 1h)
|> last()
|> group() 
|> difference()

Flux returns table streams. Multiple tables can part of the same results stream. Tables are created/identified by their group keys. A column is part of a group key if every record or row in the table is the same. In this case the window() function groups data by generated _start and _stop time values based off of the every parameter. Then you select the last function and every table in the stream has only one value. So when you go apply the difference function there isn’t multiple rows with which to calculate any difference. The group function removes the _start and _stop columns from the group key which is like saying “a table can exist with multiple start and stop times” so all the data gets grouped together. Now there are multiple rows in the same table and the difference function produces results.

This isn’t an issue with aggregateWindow() because aggreagateWindow automatically groups data.

I recommend using multiple yield() functions as print statements to understand how each flux function is transforming my data.

|> window(every: 1h)
|> yield(name: "after window") 
|> last()
|> yield(name: "after last") 
|> difference()
|> yield(name: "etc") 

I also frequently use the limit() function to reduce the output so it’s easier to visualize how the transformations are working.

1 Like