I am trying to create tasks that are down-sampling raw data into certain timeframes aggregations (1,5,15,30min), tag them and put into one bucket.
Creating the 1 min aggregated data works fine using every:1, offset:1.
For each other timeframe (5, 15, 30, 1h,…) I decided to use previously aggregated data ( 5x 1min for 5min, 3x5min for 15min) and created new tasks for each aggregation.
My question is:
Is it necessary to assign incremental offsets to each of the consequent tasks (1min = 1s, 5min = 2s, 15min = 3s, … ) in order for them to work correctly?? (Only by doing that, the data is complete, otherwise the aggregations are incorrect - missing datapoints i suppose)
Is there for example a way of assigning hierarchy to the tasks (1min->5min>15min task) / use the concurrency parameter to chain them in the right order without using the offset?
They take fractions of seconds, but whenever task executions overlap (etc. at 12:15 : 1min + 5min +15 min are fired) the data is incomplete unless I assign the incrementing offsets to each of them initially.
I just spent a lot of time trying to figure out a way to do it and learn to what extent influx can deal with such scenario.
Thank you in advance for your help!