Processing irregular (in time) sensor data

datalifecycle
time
iot
influxql
kapacitor
#1

I collect sensor data with MQTT every 5 seconds and I use the InfluDB timestamp as time marker (thus ignoring network delay).
The data I am collecting are cumulative, I count pulses from an anemometer, resetting count after each successful data transfer.
There is a number of reasons why I do so, so please try to avoid giving suggestions for changes in this setting;-) alghouth I may do it eventually :frowning:

Most of the time it works fine but sometimes, due to network, I get long interval (as reported by f.i. elapsed() ). This is a problem because the cumulative value goes up and up, which destroy my evaluation of “max” and even “mean” if the network outage is too long (in my case, longer than 5 minutes)

I would like to do one of the following:

  • calculating the pulse per seconds an storing them
  • or at least filter out the event when elapsed() reports more than 5 seconds (or say5.1). It would leave 99% of the points

What I tried so far:

  • directly calculating the rate in influx. it doesn’t work because one cannot mix elapsed() with the value of a key-value field.
  • enhance the measurement (SELECT INTO) with elapsed(). I was not able to do so, it just create an independant point, moreover without any tags (btw, is it possible to “tag” the result of a select into?). I was not able to perform math with 2 different points
  • read a lot about how I could, may be, sove this with kapacitor. I would like to avoud that, I startet playing with influxdb last week, may be I missed some important point in the way to work with influxdb.

I googled a lot and read the forum, but I found nothing. Sorry if I missed something obvious;-)