Hi,
Some alerts we cannot reset automatically using metrics from Telegraf. For this, we made a TICK reset script, that resets any arbitrary alert identified using alarmId, and send this as a metric in influx line protocol using curl to kapacitor.
Strange thing is this reset works only once. The second time we want to reset it (if anotehr script raised the alert level once again), it won’t reset but stay at the same level. Is clearing twice from same script not allowed? No errors appear in the stats for the task.
I did eventually find a workaround, to disable and enable the reset task, then send clear metric, but it would be so much nicer, if this would work, and I would understand what the issue was…
The TICK script is as follows:
dbrp “telegraf”.“autogen”
var data=stream
|from()
.measurement(‘clear_function’)
|alert()
.id(’{{ index .Tags “alarmId” }}’)
.message(‘Manual setting of alarmId {{ index .Tags “alarmId” }} by {{ in
dex .Fields “user”}}’)
.critReset(lambda: “reset” == “critReset”)
.topic(‘prod.em_raw_alarms’)
Best Regards,
Menno Bot
