Kapacitor :: union 4 streams with offset (solved)

InfluxDB 1 Kapacitor
1

Hi,

I’m in the need to perform a tick based on batch nodes. I need to query 2 times on almost current time to extract the requests and the errors; and perform the same operation with offset to extract the requests and errors from a previous time (5m, 1d, 7d…). This part is working and is like:

var period = 1m
var every = 30s

var conteo = batch
    |query('SELECT sum("responseTime.count") AS "conteo_actual" FROM "metrics"."server"')
        .period(period)
        .every(every)
        .groupBy('httpStatus', 'uri')
        .align()
    |log()

var errores = batch
    |query('SELECT sum("responseTime.count") AS "errores_actual" FROM "metrics"."server" WHERE "httpStatus" !~ /((2\d+)|(3\d+))/')
        .period(period)
        .every(every)
        .groupBy('httpStatus', 'uri')
        .align()

var conteo_prev = batch
    |query('SELECT sum("responseTime.count") AS "conteo_anterior" FROM "metrics"."http.server"')
        .period(period)
        .every(every)
        .groupBy('httpStatus', 'uri')
        .align()
        .offset(5m)
    |log()

var errores_prev = batch
    |query('SELECT sum("responseTime.count") AS "errores_anterior" FROM "metrics"."http.server" WHERE "httpStatus" !~ /((2\d+)|(3\d+))/')
        .period(period)
        .every(every)
        .groupBy('httpStatus', 'uri')
        .align()
        .offset(5m)

This data extraction is working, but now is the tricky part. As far as i know, i can do:
previous
|shift(5m)
|join(current)

But i can no make 2 to 2:
previous,previous_errors
|shift(5m)
|join(current,current_errors)

I tryied with union the four nodes, and make the calculation there…but it neither work…

Any idea / suggestion / next step to move ahead? I’m blocked trying to think waht i’m missing??

I have as another option to store the calculation on a new field, and write a separate tick comparing the actual percent with the prior one (that’s the end of all this!!!) but want to know if the 4 union should work (or even the join…) for this purpose!

Thanks a lot!

2

Hi,

What i’m currently trying, complete, is something like:

var period = 1m
var every = 30s

var conteo = batch
|query(‘SELECT sum(“responseTime.count”) AS “conteo_actual” FROM “app_metrics”.“four_weeks”.“http.server”’)
.period(period)
.every(every)
.groupBy(‘uri’)
.align()

var errores = batch
|query(‘SELECT sum(“responseTime.count”) AS “errores_actual” FROM “app_metrics”.“four_weeks”.“http.server” WHERE “httpStatus” !~ /((2\d+)|(3\d+))/’)
.period(period)
.every(every)
.groupBy(‘uri’)
.align()

var conteo_prev = batch
|query(‘SELECT sum(“responseTime.count”) AS “conteo_anterior” FROM “app_metrics”.“four_weeks”.“http.server”’)
.period(period)
.every(every)
.groupBy(‘uri’)
.align()
.offset(5m)

var errores_prev = batch
|query(‘SELECT sum(“responseTime.count”) AS “errores_anterior” FROM “app_metrics”.“four_weeks”.“http.server” WHERE “httpStatus” !~ /((2\d+)|(3\d+))/’)
.period(period)
.every(every)
.groupBy(‘uri’)
.align()
.offset(5m)

var current = conteo
|join(errores)
.as(‘count’, ‘errors’)
|eval(lambda: float(“errors.errores_actual” / “count.conteo_actual”) * 100.0)
.as(‘percent_cur’)
.keep()
|log()

var previous = conteo_prev
|join(errores_prev)
.as(‘count_prev’, ‘errors_prev’)
|eval(lambda: float(“errors_prev.errores_anterior” / “count_prev.conteo_anterior”) * 100.0)
.as(‘porcentaje_ant’)
.keep()
|log()

previous
|shift(5m)
|join(current)
.as(‘previous’, ‘current’)
|log()
|alert
.crit(lambda: current.percent_cur > previous.porcentaje_ant + 10.0)

After some try/error this script is now working. Main mistake was trying to maintain the httpStatus. Before removing this grouping, the only % saw were 0 or 100, because of that one! Making the grouping by just uri (i maybe include as well httpMethodName as it will give more detail on what’s going on!) gives good results.