I would like to execute a task every day to process a data of last 10s window. so wrote an associated tick script
var data = stream
but i guess this causes all kind of memory side-effect.
I believe this is expected behavior, as i am instructing pipeline to hold data for 24h and then release 10s worth of data every 24 hour. few q:
a) confirm my interpretation of every() property. specially when every >> period. does it hold data for every() time length ? If yes, in future are you looking to make it optimized to not store for elongated time but hold it for last period alone, as that is what would happen at the end of every().
b) how to write a tick in another fashion (if possible) to achieve the same effect but not cause memory bloat.
c) Is there any protection, limit we can set where a given pipeline (programmed intentionally or accidentally with very large window) doesnt starve out other pipeline or complete kapacitorD.