Recently we experienced a network failure. It took use a day and looking at grafana to see that one of our data source ( a db in influx ) wasn’t sending data anymore or that said data wasn’t coming in anymore. Is there way to get alerted when such a scenario arises ? If so what is the best way to monitor that data sources are sending data to influx ?
You have two scenarios here
- agents that don’t send data to influx
- influx nodes that are down
In regards to 1, you can create a deadman alert via kapacitor or some other tool to look for nodes being down.
In regards to 2. Never monitor your monitoring thing with your monitoring thing. Basically use another InfluxDB instance to monitor your other influxdb instances.
1 Like
agents that don’t send data to influx
That’s the case that I’m chasing.
https://docs.influxdata.com/kapacitor/v1/reference/nodes/alert_node/ ?
This topic was automatically closed 60 minutes after the last reply. New replies are no longer allowed.
