Help with Deadman

InfluxDB 1 Kapacitor
,
1

Hi,

I would to create an alert that triggers an event if there have not been any point during the last hour but during the previows hour there had been at least one point.

For example:

Between the 4 - 5 pm the tag “client” == ‘myself’ has 3 points, but between 5 - 6 pm there is no points for ‘myself’. It should trigger an alert for any possible client with points in the previows hour.

Is it possible?

I want something like:

var data = stream
	|from()
		.database('database_example')
		.retentionPolicy('policy_example')
		.measurement('measurement_example')
		.groupBy('tag1','tag2')

data
	|deadman(1.0, 10s)
		.stateChangesOnly(1h)
		.slack()
		.channel('#example_channel')
		.iconEmoji(':scream:')
		.username('Kapacitor - No Traffic')
		.message('NO TRAFFIC DETECTED {{ .Level }} {{ index .Tags "tag1" }} - {{ index .Tags "tag1" }}')

The problem here is that if for a specific tag1 and tag2 there are no points in the last hour but there are points in the previous hour, there will not be any alert for that tags.

How can I get an alert for the the tags that have stopped to write points?

Thanks and best regards,
Marcos.

1 Like
2

Can you share some example data that is not working as expected?

The behavior of the deadman node is to alert for data that is missing that existed at any previous time period so what you describe should work.

3

Hi @nathaniel,

This is my main problem now:

My code:

var data = stream
    |from()
            .database('stats_prv')
            .retentionPolicy('xtgpolicy')
            .measurement('hotel_avail')
            .groupBy('hus','hpr')
data
        |deadman(0.0, 10s)
                .stateChangesOnly(20s)
                .slack()
                .channel('#ntf-tgx')
                .iconEmoji(':scream:')
                .username('Kapacitor - No Traffic')
                .message('NO TRAFFIC DETECTED {{ .Level }} {{ index .Tags "hus" }} - {{ index .Tags "hpr" }}')

The alert message:

The problem here is that the tags are not set in the alert messages so I’m not able to know for which tags is the event triggered.

4

Tags should be set, but you might be running into a small edge case of the deadman node.

The deadman node will alert if no data has been received, this means if it has never received a single point then it will trigger an alert with no tags, since it doesn’t know the value of any of the tags. But once the node has received at least one point then it will track that tag group and should always have the tags. In other words it is possible for the deadman to fire an alert if it never receives a single point during the first intervals, which in your case is 10s.

Try ensuring that the deadman is in an OK state first by sending valid data and then stop the data stream and see what the error message is like after it stops. At that point it should contain all the tag information.

1 Like
5

Thanks, I’ve set 2 minutes interval and now it works.