I’ve got data that looks like:
"series" : [
{
"name" : "measurement_name",
"columns" : [
"time",
"count"
]
"values" : [
[
"2017-10-17T17:04:20Z",
2
]
],
"tags" : {
"pool" : "a",
"reporting_lb" : "1"
},
},
{
"name" : "measurement_name",
"columns" : [
"time",
"count"
],
"values" : [
[
"2017-10-17T17:04:20Z",
2
]
]
"tags" : {
"pool" : "a",
"reporting_lb" : "2"
},
}
]
I process this to get the average of the value across all "pool"s, using TICKscript like so:
var flaps = stream
|from()
.database(db)
.retentionPolicy(rp)
.measurement('loadbalancing_member_events')
// use only down events (going down then up is a full flap; don't count halves)
.where(lambda: "event" == 'readiness change' AND "transition" == 'down')
|delete()
.field('text')
.tag('transition')
|window()
.period(5m)
.every(10s)
|groupBy(['pool', 'reporting_lb'])
|count('event')
|groupBy(['pool'])
|mean('count')
The calculated average will sometimes lag minutes behind the input data. There seems to be an accumulating delay at the mean() node.
These stats might help. The mean() node is I believe “mean7” here:
stream0 [avg_exec_time_ns="0s" errors="0" working_cardinality="0" ];
from9 [avg_exec_time_ns="521ns" errors="0" working_cardinality="0" ];
window10 [avg_exec_time_ns="1.25µs" errors="0" working_cardinality="1" ];
groupby11 [avg_exec_time_ns="1.635879ms" errors="0" working_cardinality="233" ];
mean12 [avg_exec_time_ns="15.782µs" errors="0" working_cardinality="0" ];
influxdb_out13 [avg_exec_time_ns="9.7µs" errors="0" points_written="1148901" working_cardinality="0" write_errors="0" ];
from1 [avg_exec_time_ns="2.843µs" errors="0" working_cardinality="0" ];
delete2 [avg_exec_time_ns="4.833µs" errors="0" fields_deleted="155872" tags_deleted="155872" working_cardinality="0" ];
window3 [avg_exec_time_ns="4.046µs" errors="0" working_cardinality="1" ];
groupby4 [avg_exec_time_ns="3.79276ms" errors="0" working_cardinality="10" ];
count5 [avg_exec_time_ns="43.52µs" errors="0" working_cardinality="0" ];
groupby6 [avg_exec_time_ns="4.889µs" errors="0" working_cardinality="0" ];
mean7 [avg_exec_time_ns="14m18.119870102s" errors="0" working_cardinality="208" ];
join15 [avg_exec_time_ns="24.9µs" errors="0" working_cardinality="235" ];
eval17 [avg_exec_time_ns="69.2µs" errors="0" working_cardinality="206" ];
alert19 [alerts_triggered="92" avg_exec_time_ns="24.613µs" crits_triggered="8" errors="0" infos_triggered="0" oks_triggered="44" warns_triggered="40" working_cardinality="206" ];
influxdb_out20 [avg_exec_time_ns="6.025µs" errors="1" points_written="90" working_cardinality="0" write_errors="1" ];
influxdb_out18 [avg_exec_time_ns="3.005µs" errors="3" points_written="16737" working_cardinality="0" write_errors="3" ];
influxdb_out16 [avg_exec_time_ns="2.475µs" errors="0" points_written="16743" working_cardinality="0" write_errors="0" ];
influxdb_out8 [avg_exec_time_ns="6.733µs" errors="0" points_written="160454" working_cardinality="0" write_errors="0" ];
Any ideas why this might be happening?
Thanks.