BATCH - Eval broken with join

#1

kapacitor version : Kapacitor 1.2.0 (git: master 5408057e5a3493d3b5bd38d5d535ea45b587f8ff)
OS: Ubuntu 16.04.1 LTS
Kernel: 4.4.0-53-generic x86_64

Hi Team,

When i’m using several batch with a join including eval, eval dont works.
Values

  • newvaluepending
  • newvalueok

are not present because eval not ran.

Maybe i’m doing something wrong but look like a bug around join using eval.

I have also tested with .tolerance(1m) and .tolerance(5m) and .tolerance(48h) inside |join but still not working.

Thanks for your feedback. All informations are bellow.

See bellow my tick:

var data_pending_5min = batch
        |query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='Pending' ''')
        .groupBy('channel','result')
        .every(1m)
        .period(5m)
        .align()
        |default()
                .field('value', 0)
        |log()
                .prefix('[DEBUG]-1')
                .level('DEBUG')

var data_ok_5min = batch
        |query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='OK' ''')
        .groupBy('channel','result')
        .every(1m)
        .period(5m)
        .align()
        |default()
                .field('value', 0)
        |log()
                .prefix('[DEBUG]-2')
                .level('DEBUG')

data_ok_5min
        |join(data_pending_5min)
                .as('data_pending_5min', 'data_ok_5min')
        |eval(lambda: "data_pending_5min.value",
              lambda: "data_ok_5min.value")
                .as('newvaluepending',
                    'newvalueok')
                .keep()
        |log()
                .prefix('[DEBUG]-3')
                .level('DEBUG') 

Find bellow the kapacitor show:

var data_pending_5min = batch
    |query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='Pending' ''')
        .groupBy('channel', 'result')
        .every(1m)
        .period(5m)
        .align()
    |default()
        .field('value', 0)
    |log()
        .prefix('[DEBUG]-1')
        .level('DEBUG')

var data_ok_5min = batch
    |query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='OK' ''')
        .groupBy('channel', 'result')
        .every(1m)
        .period(5m)
        .align()
    |default()
        .field('value', 0)
    |log()
        .prefix('[DEBUG]-2')
        .level('DEBUG')

data_ok_5min
    |join(data_pending_5min)
        .as('data_pending_5min', 'data_ok_5min')
    |eval(
        lambda: "data_pending_5min.value",
        lambda: "data_ok_5min.value"
    )
        .as(
            'newvaluepending',
            'newvalueok'
        )
        .keep()
    |log()
        .prefix('[DEBUG]-3')
        .level('DEBUG')

DOT:
digraph alerting_reg_batch {
graph [throughput="0.00 batches/s"];

query4 [avg_exec_time_ns="0s" batches_queried="2" points_queried="2" query_errors="0" ];
query4 -> default5 [processed="2"];

default5 [avg_exec_time_ns="0s" fields_defaulted="0" tags_defaulted="0" ];
default5 -> log6 [processed="2"];

log6 [avg_exec_time_ns="0s" ];
log6 -> join8 [processed="2"];

query1 [avg_exec_time_ns="0s" batches_queried="2" points_queried="2" query_errors="0" ];
query1 -> default2 [processed="2"];

default2 [avg_exec_time_ns="0s" fields_defaulted="0" tags_defaulted="0" ];
default2 -> log3 [processed="2"];

log3 [avg_exec_time_ns="0s" ];
log3 -> join8 [processed="2"];

join8 [avg_exec_time_ns="0s" ];
join8 -> eval9 [processed="0"];

eval9 [avg_exec_time_ns="0s" eval_errors="0" ];
eval9 -> log10 [processed="0"];

log10 [avg_exec_time_ns="0s" ];
}

Log from kapacitor :

[alerting_reg_batch:query4] 2017/04/06 16:15:00 D! starting next batch query: SELECT last(value) AS value FROM metrics_business.autogen.reg WHERE metric = 'last5min' AND result = 'OK' AND time >= '2017-04-06T16:10:00Z' AND time < '2017-04-06T16:15:00Z' GROUP BY channel, result
[alerting_reg_batch:query1] 2017/04/06 16:15:00 D! starting next batch query: SELECT last(value) AS value FROM metrics_business.autogen.reg WHERE metric = 'last5min' AND result = 'Pending' AND time >= '2017-04-06T16:10:00Z' AND time < '2017-04-06T16:15:00Z' GROUP BY channel, result

[alerting_reg_batch:log3] 2017/04/06 16:15:00 D! [DEBUG]-1 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel1,result=Pending","tags":{"channel":"channel1","result":"Pending"},"points":[{"time":"2017-04-06T16:14:01.526306187Z","fields":{"value":16},"tags":{"channel":"channel1","result":"Pending"}}]}

[alerting_reg_batch:log6] 2017/04/06 16:15:00 D! [DEBUG]-2 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel1,result=OK","tags":{"channel":"channel1","result":"OK"},"points":[{"time":"2017-04-06T16:12:01.732661132Z","fields":{"value":2},"tags":{"channel":"channel1","result":"OK"}}]}

[alerting_reg_batch:log3] 2017/04/06 16:15:00 D! [DEBUG]-1 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel2,result=Pending","tags":{"channel":"channel2","result":"Pending"},"points":[{"time":"2017-04-06T16:14:01.526306187Z","fields":{"value":8},"tags":{"channel":"channel2","result":"Pending"}}]}

[alerting_reg_batch:log6] 2017/04/06 16:15:00 D! [DEBUG]-2 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel2,result=OK","tags":{"channel":"channel2","result":"OK"},"points":[{"time":"2017-04-06T16:14:01.526306187Z","fields":{"value":5},"tags":{"channel":"channel2","result":"OK"}}]}

#2

Same here - join do not process further.