Help with DeadMan Checks

InfluxDB 1 Kapacitor
1

Hi all,

As I dont have the option to send alerts from Influxdb it has to be via Grafana for now - has anyone got any information on how I can use the Deadman check that is within the awesome Influxdb within Grafana - I cant seem to find any information about it - some people have said to use inputs ping but I dont seem to have the same success that the deadman would have as when telegraf service stops the info stops and the metric stays the same.

Does the deadman setup get posted into the timeseries db anywhere that I can pull?

Any help appreciated

2

What version of InfluxDB?

Influx 2.0 I don;t know about, but with 1.8.x and Kapacitor you can utilise the deadman function.

3

Influx 2.0 :face_with_raised_eyebrow:

4

If you’ve setup a deadman check in InfluxDB you should be able to read the results of it from the _monitoring bucket.

Open your Data Explorer tab and run this:

from(bucket: "_monitoring")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "statuses")
  |> filter(fn: (r) => r["_type"] == "deadman")
5

Thats great thanks for the info it was what I was looking for, sadly for some reason I am unable to access the _monitoring bucket from Grafana, it only shows user created buckets.

I even tried to create a token to show all buckets but to no avail.

6

Is there a way to automatically copy the data for the deadman check from _monitoring into the normal bucket in which my case is telegraf?

Anyone have any pointers?

7

You can use a Task to move data from _monitoring into another bucket

8

Thanks @mhall119, in a separate issue this has kinda been resolved as can now access the _monitoring bucket from Grafana, I would like to know however how to copy data from one bucket to another - as the current solution is a temporary one and would need to backup the buckets / db and switch to new hardware in the future - has anyone had to do this yet?

9

you can create a task in influxdb containing a query that returns the data you want to use and then you pipe it into the “to()” function to write it into a different bucket.

1 Like