Task Execution Hierarchy


I am trying to create tasks that are down-sampling raw data into certain timeframes aggregations (1,5,15,30min), tag them and put into one bucket.
Creating the 1 min aggregated data works fine using every:1, offset:1.
For each other timeframe (5, 15, 30, 1h,…) I decided to use previously aggregated data ( 5x 1min for 5min, 3x5min for 15min) and created new tasks for each aggregation.

My question is:
Is it necessary to assign incremental offsets to each of the consequent tasks (1min = 1s, 5min = 2s, 15min = 3s, … ) in order for them to work correctly?? (Only by doing that, the data is complete, otherwise the aggregations are incorrect - missing datapoints i suppose)

Is there for example a way of assigning hierarchy to the tasks (1min->5min>15min task) / use the concurrency parameter to chain them in the right order without using the offset?

They take fractions of seconds, but whenever task executions overlap (etc. at 12:15 : 1min + 5min +15 min are fired) the data is incomplete unless I assign the incrementing offsets to each of them initially.

I just spent a lot of time trying to figure out a way to do it and learn to what extent influx can deal with such scenario.

Thank you in advance for your help!

Hello @cruder,
Ooh very cool question let me ask around.

1 Like

Hello @cruder,
I don’t think there’s anyway to force tasks dependency with Flulx.
But yes you can force it with offsets like you’re doing.
But I encourage you to make a feature request.

Hello @Anaisdg,

Thanks for the response!
I will try to find a different way of executing the tasks in a desired order / right time :slight_smile: