Hi,
I have a metric (number of events) that is collected every 5 mins by a custom agent. The metric is classified with a couple of tags (e.g. host and client). When looking at a 7 day plot, there’s a clear pattern for each series, that reflects very low traffic during the weekends / holidays, and high traffic during weekdays, that grows from early morning into mid-day, and then starts descending into the evening.
I’d like to implement automated alerting on top of these metrics. In a nutshell, have Kapacitor look at the the pattern traffic of at least the last 4 weeks for all series, and evaluate every hour if the past hour average deviates too much (I guess a tolerance would have to be defined somehow) from the what would be expected at that time of day (on that day of the week), for the series.
Given that, I’d appreciate some guidance on best approach to achieve that, and also to have a visualization that highlights the anomaly (the current values vs the expected)
Thanks!
