Is it possible to define a ‘back to normal’ version of the message string that you define in an alert?
For example lets say I’m counting the number of api requests made every couple minutes, and then via a udf detecting spikes above/below some fixed multiplier…
|alert()
.id('request_{{ index .Tags "request_type"}}')
.warn(lambda: "multiplier" > max_multiplier)
.warnReset(lambda: "multiplier" <= max_multiplier)
.stateChangesOnly()
.message('{{ index .Fields "count" }} calls to {{ index .Tags "request_type" }} in the last 2 minutes is x{{ index .Fields "multiplier" }} more than normal')
My first warning (in slack) might say…
(orange-icon) 400 calls to login in the last 2 minutes is x10.87 more than normal
When reset kicks-in the new message would be something like…
(green-icon) 36 calls to login in the last 2 minutes is x1.45 more than normal
I guess there’s no way to change the reset message to be something different, i.e. to include the phrase “is back to normal”?
One thought would be to generate the message, or some element of it, in the UDF and pass it back to kapacitor.
How would you guys normally manage the message strings?