Batch data from internal kapacitor stream (with no backing influxdb data)

Telegraf
1

We’re running with the concepts from prometheus_metrics_normalizer, which uses

stream
    |from()
    |where(...)

to process all of the raw data coming from the kapacitor scraper of /metrics info.

So far, so good.

But, we’re trying to make a batch processor instead of a stream processor, so we can write more complex grouping logic – and I’m trying to figure out, what’s the batch equivalent of this?

That is – there’s no actual query that applies, since the data is just coming through the kapacitor stream and not from the influxdb yet… but when i try batch|query().period(1m).every(20s).groupBy(time(10s), '__name__') – then i get an error that query can’t be empty.

Suggestions? What am i missing?

2

Update: Courtesy of @jackzampolin from the #influxdb slack channel, the answer here is to use the window() function to turn our streams into batches.

eg, rather than attempting this (which doesn’t work due to empty query):

batch
    |query()
        .period(1m)
        .every(20s)
        .groupBy('__name__')

we can instead use this:

stream
    |from()
    |window()
        .period(1m)
        .every(20s)
    |groupBy('__name__')
1 Like
3

@shawnfh Glad you were able to get that working!