DerivativeNode in Kapacitor

1

Hello,

I am trying to use Kapacitors DerivativeNode in batch mode to calculate desired values, but when my TICK script reaches Derivative Node it doesn’t return any values. The TICK script that I am using:

# kapacitor show one_by_one 
ID: one_by_one
Error: 
Template: 
Type: batch
Status: disabled
Executing: false
Created: 02 Mar 17 10:09 UTC
Modified: 11 Jan 18 14:09 UTC
LastEnabled: 01 Jan 01 00:00 UTC
Databases Retention Policies: ["collectd"."default" "history"."autogen"]
TICKscript:
var stats = batch
    |query('''
        SELECT mean("value") as "value" FROM "collectd"."default"."nginx_value"
        WHERE "host" =~ /some-server/ AND "type" = 'nginx_requests'
    ''')
        .period(1h)
        .every(1h)
        .groupBy(time(1h), *)
        .align()
    |log()
    |derivative('value')
        .unit(1s)
        .nonNegative()
        .as('value')
    |log()
    |influxDBOut()
        .database('history')
        .measurement('web_stats')

DOT:
digraph one_by_one {
query1 -> log2;
log2 -> derivative3;
derivative3 -> log4;
log4 -> influxdb_out5;
}

I define the task, record past 3h of data and replay it with:

kapacitor define one_by_one -type batch -tick one_by_one.tick -dbrp collectd.default -dbrp history.autogen
kapacitor record batch -task one_by_one -past 3h
kapacitor replay -task one_by_one -recording 93212075-0d99-4d1b-b296-f06f36c70df2 -rec-time

Log of one_by_one task:
one.txt (7.9 KB)

# kapacitor version
Kapacitor 1.4.0 (git: HEAD fcce3ee9e6abcee5595fd61066bfc904edb1e113)

dpkg -l | grep influx
ii  influxdb                       1.3.6-1                          amd64        Distributed time-series database.

If I am not using “|derivative” I am getting the results. Any help would be appreciated.

2

Hi Marijus,
I met the same situation with Kapacitor as you and it took me some amount of time to figure out the solution (as i was not able to find proper examples or good docs in the Kapacitor section of the site).
Here is what is working for me now:

ID: total_traffic_test-2
Error: 
Template: 
Type: batch
Status: enabled
Executing: true
Created: 18 Jan 18 11:55 PST
Modified: 19 Jan 18 14:05 PST
LastEnabled: 19 Jan 18 14:05 PST
Databases Retention Policies: ["mdt_db"."test6h"]
TICKscript:
dbrp "mdt_db"."test6h"

var received = batch
    |query('''
       SELECT sum("bytes-received")  as "received"
       FROM "mdt_db"."test6h"."Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"
       WHERE "interface-name" =~ /Bundle.*/ AND time > now() - 1m GROUP BY time(1s),"Producer"
     ''')
        .groupBy(time(1s), 'Producer')
        .period(1m)
        .every(10s)
        .align()
    |derivative('received')
        .unit(1s)
        .nonNegative()
        .as('received')

var sent = batch
    |query('''
       SELECT sum("bytes-sent")  as "sent"
       FROM "mdt_db"."test6h"."Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"
       WHERE "interface-name" =~ /Bundle.*/ AND time > now() - 1m GROUP BY time(1s),"Producer"
     ''')
        .groupBy(time(1s), 'Producer')
        .period(1m)
        .every(10s)
        .align()
    |derivative('sent')
        .unit(1s)
        .nonNegative()
        .as('sent')

received
    |join(sent)
        .as('received', 'sent')
        .tolerance(5s)
    |eval(lambda: "received.received" - "sent.sent")
        .as('difference')
    |alert()
        .info(lambda: "difference" > 100000000)
        .post('http://10.30.110.38:5000/relay')
        .stateChangesOnly()

DOT:
digraph total_traffic_test-2 {
graph [throughput="0.00 batches/s"];

query3 [avg_exec_time_ns="32.65558ms" batches_queried="2971" errors="0" points_queried="35449" working_cardinality="0" ];
query3 -> derivative4 [processed="2971"];

derivative4 [avg_exec_time_ns="1.07µs" errors="0" working_cardinality="6" ];
derivative4 -> join6 [processed="2971"];

query1 [avg_exec_time_ns="136.80986ms" batches_queried="2971" errors="0" points_queried="35446" working_cardinality="0" ];
query1 -> derivative2 [processed="2971"];

derivative2 [avg_exec_time_ns="2.426µs" errors="0" working_cardinality="6" ];
derivative2 -> join6 [processed="2971"];

join6 [avg_exec_time_ns="22.642µs" errors="0" working_cardinality="6" ];
join6 -> eval7 [processed="2971"];

eval7 [avg_exec_time_ns="2.369µs" errors="0" working_cardinality="6" ];
eval7 -> alert8 [processed="2971"];

alert8 [alerts_triggered="2" avg_exec_time_ns="2.667µs" crits_triggered="0" errors="0" infos_triggered="1" oks_triggered="1" warns_triggered="0" working_cardinality="6" ];
}

You also need to make sure your .period(1m) is equal to time period in query.
If somebody from Influx team could enhance documents about Kapacitor, it would be great.
thanks.