I’ve created simple kapacitor task to watch disk usage data and alert on certain threshold. Disk usage data gets updated in 10s interval. I wonder how kapacitor checks and raise alerts? Is kapacitor also creates alerts in every 10s in this case ? If it’s, is it configurable or it directly depends on data collection interval. Kapacitor sends alerts via SMTP or some other service and looks like it’s sending mails for each control.
Can I configure kapacitor to send the alert for once for each warning level change (ok to crit or crit to ok or warn to crit…) and may be accelarete the warning level and send again after some time (like after 30 minutes, ‘hey this is still crit’). Currently it spams so many alerts.
Is alerts cheap? How should I monitor the performance or if there is waiting queue?