[Kapacitor] [TICKscript] Why would I want overlapping data in a stream?

Telegraf
1

Hi all,

I was reading through some Kapacitor documentation about the WindowNode. Specifically with the WindowNode, there’s two properties:

  • period(): Time range of collected data
  • every(): Frequency window is passed to the next node in pipe

Both of these make sense, but I was a little curious about the practicality of the example in the docs:

stream
    |window()
        .period(10m)
        .every(5m)
    |httpOut('recent')

The docs explain what this is doing, but not why:

This example emits the last 10 minute period every 5 minutes to the pipeline’s httpOut node. Because every is less than period, each time the window is emitted it contains 5 minutes of new data and 5 minutes of the previous period’s data.

If this were your script, what would the advantage be of having overlapping data in the stream? Is there a concern of some of it being dropped? I was curious and wanted to get a better understanding of why someone would want to do this.

Thanks!

2

I’m curious, too.

One thought would be you’d use the overlap to prevent missing any data if period and every were the same and some data points “fell through the cracks” between periods. But that seems like a lot of overlap to prevent that, and it seems like it shouldn’t happen, anyway.

It’d be nice to hear from the experts.

1 Like
3

Hi there,

One benefit of being able to grab a window of overlapping data in a stream would be in the instance of having to find some kind of aggregate of a moving window of data (such as in finding a moving average for example). You only want the window to shift forward slightly in time as you perform some kind of transformation to it.

Hope that helps a bit.

2 Likes
4

This makes a little more sense now. It would be cool to see this added to the docs – I remember when I first read it and I was confused as to what the use case would be.