So I have a spike at around 12:58PM for example. When I replay the data against the tickscript below, the timestamp I get seems to be the beginning of the first window that includes the spike/anomaly, so around 12:48PM. I am guessing this is working as intended, but I am not clear if it would work the same when not doing a replay.
The way I imagine it working when streaming data live, is at 12:48PM it looks back to 12:38PM and checks for anomalies in that 10m window. Then at 12:49PM it looks back to 12:39PM. And so on. In the replay though it seems to be looking forward instead of back. Is that just a difference between replays and live streaming? Or am I looking at this fundamentally incorrectly? If so, how can I do what I want, which is to compare each current minute with the previous 10 to detect anomalies?
var data = stream
|from()
.measurement(measurement)
.groupBy('shard')
|window()
.period(period)
.every(every)
.align()
// Send each window to Morgoth
@morgoth()
.field(field)
.scoreField(scoreField)
.minSupport(minSupport)
.errorTolerance(errorTolerance)
.consensus(consensus)
.sigma(sigmas)
var alert = data
|alert()
.details('')
.crit(lambda: TRUE)
.critReset(lambda: FALSE)
.stateChangesOnly()
.log('/tmp/' + field + '.log')
There should be no difference between replay and live streaming. Could you share some example data that exhibits the issue? Your understanding seems correct to me.
I blew away the install without saving the replay or data, but here’s similar data.
time value
---- ----------
1499719440000000000 95
1499719500000000000 101
1499719560000000000 99
1499719620000000000 96
1499719680000000000 98
1499719740000000000 84
1499719800000000000 103
1499719860000000000 103
1499719920000000000 85
1499719980000000000 120
1499720040000000000 118
1499720100000000000 94
1499720160000000000 89
1499720220000000000 91
1499720280000000000 110
1499720340000000000 98
1499720400000000000 98
1499720460000000000 82
1499720520000000000 86
1499720580000000000 120
1499720640000000000 102
1499720700000000000 99
1499720760000000000 118
1499720820000000000 284
1499720880000000000 203
1499720940000000000 146
1499721000000000000 146
1499721060000000000 157
1499721120000000000 120
1499721180000000000 103
1499721240000000000 156
1499721300000000000 100
1499721360000000000 118
1499721420000000000 107
1499721480000000000 101
1499721540000000000 91
1499721600000000000 103
1499721660000000000 90
Unfortunately, and this sorta veers into a different problem, I can’t get the replay of that data to work like I had before. I get “no field value is not a float or int” then “morgoth3: write error: write |1: broken pipe.”
But I am pretty sure the timestamp it was giving was the start of the anomalous window rather than the timestamp of the anomalous point.
