Filter Kapacitor alerts through a UDF

InfluxDB 1 Kapacitor
1 
stream
   | from()
         .database(db)
         .measurement(msmt)
   | alert()
         .id('kapacitor/{{ .TaskName }}')
         .crit(lambda: "usage_idle" < 90)
         .message('{{ .ID }} is {{ .Level}}:  ({{ index .Fields "usage_idle" }}) is < 90 at {{ .Time }}')
   @alertFilter() //custom udf

Is there a way to access strings evaluated by alert() node in my alertFilter UDF? Basically I want to filter alerts generated by alertNode on some criteria and then send an alert on email/slack.

2

This is possible look at the AlertNode | InfluxData Documentation Archive property and the related properties.

What kind of criteria and why do you think it requires a UDF?

3

Thanks for the quick reply. messageField looks interesting. How do I name the field, looks like it takes only one argument?

The FieldsString in my UDF looks like this: fieldsString:<key:"{{ .ID }} is {{ .Level}}: Average storage capacity ({{ index .Fields “usage_idle” }}) is above 90 at {{ .Time }}" value:“kapacitor/us
age_idle is CRITICAL” >

We have lot of tick scripts and they generate lots of alerts, we want a way to filter these, optionally cap the number of alerts per day etc… So I’m thinking of using a socket based UDF which will look at a yaml/json file for notification policy and apply that to every alert that gets generated by alert node. Modifying a yaml file is much easier than modifying tick script IMO.

4

Ignore “How do I name the field” part. Following works and it automatically adds the message as value.

| alert()
   .message( 'blah')
   .messageField('message')

Feel free to comment on the UDF implementation part, if you have better ideas. Thanks

5

I think you are going to want to use the new topics system for alerts. It was first introduced in v1.2 of Kapacitor and is being improved for the v1.3 release.

The basic idea would be that you can send all your alerts to the same topic and then filter/aggregate them in there.

There are not a whole lot of docs yet on how this works as the improvements are just getting ready to be released but here is a good place to start kapacitor/DESIGN.md at master · influxdata/kapacitor · GitHub

Also if you have specific questions I can answer them.

Until v1.3 lands the UDF is probably a simple solution, but the hope would be the the topics system is robust enough to be able to do want you want natively.

NOTE: You can get the id, message, and level as fields or tags on the data just check for the corresponding {id,message,level}{Field,Tag} property.