Filtering data from many column at the same time!

Telegraf
,
1

Hi everybody,

I got real time data from over 50 sensors per second. And for some difficult reasons, I can not filter the spike errors from those sensors’ side so I want to filter it out at one when the sensors’ data arrive Influxdb.

The filter would be simple that ( * > mean( * ) + stddev( * )) with * is all the data points. However, the problem is that I can not filter it out in Kapacitor . I can not use IF or CASE , or just some simple comparison like > , < in SELECT clause . And in WHERE clause, I can not use * there to filter data . Finally is the TICK script, I can not find any For Loop or While Loop to compare the each data point with the range to filter the spike errors out. It’s over 50 column so I can not go 1 by 1, I need to filer it all at 1 , for example in 30s interval.

May be I miss something or there is some other way to do it. So if anybody has any ideas about how to do this, please let me know !

Thank you in advances !

2

Hi @niki,

So TICK scripts are not like bash scripts, or other scripting languages with which you might be familiar. TICK scripts run inside Kapacitor, so Kapacitor handles the looping, etc. internally.

Depending on how you want to group your query, you may not be able to do it in Kapacitor and may need to do it via a Continuous Query (CQ). instead which you would then have to send the output of to Kapacitor to generate the alert.

TICK Scripts don’t support wildcard operators like * so if you’re trying to select across multiple measurements, etc. you’ll need to use CQs instead of a TICK Script.

CQs are limited to things expressible in InfluxQL

  • No rolling windows
  • No cross measurement joins

For what it appears you’re trying to do, a CQ to evaluate the data that feeds into a new measurement field which a TICK script can monitor and generate alerts on might be your best bet.

Best Regards,
dg