Hi, would it be possible to implement LTTB (BlackOps - Time-series Graphs and Downsampling) in kapacitor?
@Fredrik_Skeel_Lokke I took a quick look into LTTB, and I found this implementation in Go, lttb package - github.com/dgryski/go-lttb - Go Packages. You could write a UDF to take batches of data and perform the LTTB transformation and return the downsampled data. See the docs on writting UDFs Writing a socket based UDF | InfluxData Documentation Archive
The LTTB method has a threshold argument which if I understand correctly represents the number of points you want returned at a maximum from the LTTB transformation. This means that you will probably have to pick a few different sizes of graphics and precompute those downsampled values and then point your visualization layer at the right downsampled data accordingly.
If you put something together and are wiling to open source it, I would love to take a look.
Lastly I want to point out that the LTTB downsampling algorithm is purely about producing good visualizations of the data and is not targeted as a anti-aliasing safe downsampling method. Meaning that you should only use the LTTB downsampled data to draw graphs and not perform any further mathematical analysis of the data as it will likely introduce artifacts in the downsampled data that are not present in the raw data.
Thank you very much for the feedback nathaniel. I’ll let you know if we decide to go in this direction (+ open source…). And yes this is purely for presentation where we are interested in quickly spotting outliers…
