Alert to detect wrong timestamp

InfluxDB 1 Kapacitor
1

Hi, I’d like to make an alert that monitor if there’s any data coming with wrong timestamp.

For example, if there’s a server has wrong OS clock setting, like, it’s a day behind than real time, it’s telegraf will send any metric with that timestamp.
This will corrupt that day’s statistics, and I’d like to spot these server ASAP.

So i tried these eval node, hoping to catch cpu metric’s timestamp and use it to trigger alert

var data = stream
|from()
.database(‘Default’)
.retentionPolicy(‘autogen’)
.measurement(‘cpu’)
.groupBy(‘host’)
|eval(lambda: “time”)
.as(‘time_field’)
.keep(‘cpu_idle’, ‘host’, ‘time_field’)
|eval(lambda: int(unix(“time_field”)))
.as(‘time_unix’)
.keep(‘cpu_idle’, ‘host’, ‘time_unix’)
|log()

And only got these errors :

Kapacitor
ERROR: error evaluating expression

What am I doing wrong here?

2

Hello @shan1,
Unfortunately I’m not sure and Influxdata doesn’t offer free support on Kapacitor anymore. Your best bet is to find old kapacitor questions and reach out to those users. And see if they can help :slight_smile:

3

really… well, I found below post helpful and was able to made alert anyway.

Thanks!

4

So, I made alert anyway.
For anyone looking for same answer, here’s mine :

var message = 'time_unix test: [{{.Level}}] [{{.ID}}] OS time : {{.Time}}'

var OKmessage = 'time_unix test: Server time gap is now [OK], [{{.Group}}] OS time : {{.Time}}'

var data = stream
    |from()
        .database('Default')
        .retentionPolicy('autogen')
        .measurement('cpu')
        .groupBy('host')
    |eval(lambda: unixNano(now()) - unixNano("time"))
        .as('time_nano')
    |eval(lambda: if("time_nano" < 0, (-1 * "time_nano"), "time_nano") / 60000000000)
        .as('time_min')

var trigger = data
    |alert()
        .warn(lambda: "time_min" > 10)
        .stateChangesOnly(10m)
        .message('{{if eq .Level "OK"}}' + OKmessage + '{{else}}' + message + '{{end}}')