Filtering or removing flickering values from sensors

[Chris_KIP]

Hello everyone,

currently we’re using InfluxDB to store data derived from parking cameras which send us data about the current occupation of various parking lots.
We’re using the nodejs client library (read and write apis).
The problem is that sometimes the cameras send false positives/negatives at short intervals, so when saving every message without filtering, we end up with some “flickering” (see attached screenshot).

What would be a good way to either filter those values before saving them, removing them when already saved (like some cleanup procedure), or simply ignoring them when running queries over the dataset?

My colleagues suggest to just cleanup the database at certain intervals, like every hour, and remove those flickering values. But given that Influx is a timeseries DB, it think there have to be better methods.

Thanks in advance!

[scott]

@Chris_KIP Probably the best thing to do would be to clean up the flickering before writing the data to InfluxDB. It may be easier to identify the flickering in your application before writing the data rather than identifying it at query time.

While this may not completely remove the flickering, you could “normalize” the data a query time by querying the flickering field, using aggregateWindow() to apply mode() to the interval-based windows of time. This will give you the more common value of the flicker. This too could be inaccurate.