Knowing Kapacitor Alert State

Advait_Deo · June 19, 2019, 10:52pm

Hi,

is there a way where we can know the current state of alert in kapacitor?
I see that kapacitor provide details about how many alerts it has fired and we can see number of warns_triggered, crits_triggered and oks_triggered.

But looking at the alert output from kapacitor, I don’t see anything which tells me whether an alert is currently in alarm state or OK state.

Also, number of warns_triggered, crits_triggered and oks_triggered does not add up every time. In case if alert changes and we disable/enable alert, everything is reset to 0.

Any idea on how we can know if kapacitor alert is in alarm state or OK state?

Regards,

Advait Deo

MarcV · June 20, 2019, 6:30am

Hi @Advait_Deo, welcome back

You can send the alert to a measurement in the database and then you can find the last alert in the measurement to know the state ,
Hope this helps ,
Best regards

Advait_Deo · June 20, 2019, 8:26am

Thanks @MarcV. We are doing that but problem is we cannot set taskname with influxdbOut

github.com/influxdata/kapacitor

Templates in InfluxDBOutNode tag

opened 11:44AM - 19 Oct 16 UTC

tomer-amir-vonage

enhancement

Hi, I am trying to use the InfluxDBOutNode to write the results of my tick scri…pt back to InfluxDB, and I would like to add a tag that indicates the task who wrote the line, something like this: ``` var data = stream |from() .measurement('cpuStats') |groupBy('host') |window() .period(1h) .every(1m) |mean('idle') |eval(lambda: int(100.0 - "mean")) .as('load') //Send alert data |alert() ... //Save data to influxdb for dashboard data |influxDBOut() .database('clusterStatus') .retentionPolicy('default') .measurement('cpuStatus') .tag('kapacitor', 'true') .tag('cluster_task', '{{ .TaskName }}') ``` unfortunately, this writes "{{ .TaskName }}" in the database, instead of e.g. "cpu_monitor_task". Is there another way to do this?

Also, even if we send that alert data to measurement, it also sends all the tags and fields that are used in all the alerts which make it difficult to process the measurement and show the output in GUI. Because we have too many tags and fields in that measurement. We are not using Chronograf in our setup.

MarcV · June 20, 2019, 8:44am

You can add the taskname ( hardcoded ) in the measurement using tag in influxDBout ,
is that an option for you ?

In the kapacitor logfile the task name is logged …

ts=2019-06-20T10:52:00.008+02:00 lvl=info msg=“batch point” service=kapacitor task_master=main task=health node=log4 prefix= name=mem group=host=myhost,service= tag_host=mytaghost tag_service= field_last=35.549647085827964 time=2019-06-20T08:51:00.001Z

Advait_Deo · June 20, 2019, 10:02am

I think I can workaround that and get the tag name hardcoded.
Do we have a way to exclude all other tags and fields when writing to measurement using influxdbOut?
I want to send only specific tags and field to this measurement.

MarcV · June 20, 2019, 10:27am

Do you mean the deleteNode

Advait_Deo · June 21, 2019, 8:36am

Great. Thanks I will use that.

MarcV · June 21, 2019, 8:46am

Ok ! Have a nice weekend

Topic		Replies	Views
Kapacitor: Emit alert on startup if OK	5	1328	April 4, 2017
Kapacitor - Send OK alerts when info, warn or crit are not triggered Kapacitor	3	2171	August 21, 2018
Get count of all alert triggerd Kapacitor	1	481	June 10, 2019
Kapacitor- Handle Alert criteria based on global state and recurring window with a timer firing every 5 secs with emission of data Welcome & Getting Started	0	356	August 13, 2019
Kapacitor - Alert on same measurement but different tag generating too many alerts kapacitor	3	1083	April 26, 2019

Knowing Kapacitor Alert State

Related Topics