Kapacitor's Alert Issue

Store
1

dbrp “ABC”.“autogen”

var data = stream
|from()
.database(“ABC”)
.retentionPolicy(“autogen”)
.measurement(memoryUtil)
.groupBy(host)
|window()
.period(1m)
.every(1m)
.align()
|median(‘available_percent’)
.as(‘value’)

data
|alert()
.warn(lambda: “value” < warn)
.crit(lambda: “value” < crit)
.stateChangesOnly()
.message(message)
.topic(topic)

Let assume at a particular time My data is:

name: memoryUtil
tags: host=ip-111-111-17-208
time median

1528787490000000000 34.31948838933012

name: memoryUtil
tags: host=ip-111-111-53-24
time median

1528787510000000000 19.15578168915138

name: memoryUtil
tags: host=ip-111-111-84-49
time median

1528787530000000000 14.391669240940798

Question: When I set my critical to less that 30%, I am getting only one alert instead of two. How to get alerts for both the hosts.
e.g. .crit(lambda: “value” < 30)

2

Because one of them is 34%, which is greater than 30%. If the samples you provided are real

1528787490000000000 34.31948838933012

name: memoryUtil
tags: host=ip-111-111-53-24
time median

1528787510000000000 19.15578168915138

name: memoryUtil
tags: host=ip-111-111-84-49
time median

So only one will alert. What was the threshold before?

3

@philb

I set my critical alert to less that 30 e.g. crit(lambda: “value” < 30). So two machines ip-111-111-53-24 and ip-111-111-84-49 should be generating critical alerts but I get only one.

4

Sorry i misread your first post.

Have the machines gone into the OK state since generating the first alert? You could try removing the stateChangesOnly and increasing the threshold so they recover - then change the thresholds back.

1 Like