Flux - data aggregation in local time zone

InfluxDB 2
21

The feed is a year old now, I’m wondering if anything has changed.
I want my tasks to run every day at 00:00 local time to create a daily total, but instead they run at 00:00 UTC which now it +02:00 hours. Very annoying.

If there is no standard option, would this be possible to solve without using external code?

22

@MarcoB

I just bumped into this a couple of weeks ago:

Seem’s that the aggregateWindow has got at location option, but I’m still using 2.0, so have not been trying it out yet. Very little attention to that “little” update :slight_smile:

23

Thanks for the tip!

Yes, now the aggregate function works properly and calculates aggregates daily at 00:00 local time.

However, the task that I use to do this every day still starts at 02:00 local time (00:00 UTC), so this basically gives me aggregate results at both 00:00 and 02:00 …

So, what I’m now looking for is a way to make sure that “daily” tasks also run at 00:00 local time. Any tips how to do that?

24

I’m not sure if this will fix it, but the cron option specifies, that it will use the system time for scheduling.

25

Indeed. I was excited to see it a few weeks ago :smile: and this was the trigger to migrate to 2.2 which I just did and it really seems to work :face_with_open_eyes_and_hand_over_mouth:

There is some inconsistency in the documentation and the way I got this working was with timezone and option location

import "timezone"
option location = timezone.location(name: "Europe/XXX")
from(bucket: "bucket")
...

Do you use it differently?

I’m still working on the CQ/tasks migration, good to know there may be some issues.

26

Yes, that looks what I need. Going to try that!

27

I did it like this:

import "timezone"    
|> aggregateWindow(
        every: 1d,
        fn: last,
        createEmpty: false,
        location: timezone.location(name: "Europe/XXX"),
    )
1 Like
28

Just curious - if the “option location” is used, will the script work as expected using local times? As in there will be no need to set the location parameter when running e.g. aggregateWindow?

29

Unfortunately changing the task to use Cron did not work.

The task, as configured for example as below, still executed at 00:00 UTC.

option task = {
    name: "WaterDaily",
    cron: "0 0 * * *",
    offset: 2m,
}

I will try to see what happens if I add this to the beginning of the task, but I doubt it:

option location = timezone.location(name: “Europe/XXX”)

30

Mmm, I just noticed that the Container that runs Influx did not have set the Time Zone so it was using UTC. Just updated that, and I guess that may solve the Cronjob starting at 00:00 local time.
W’ll see tomorrow :wink:

31

Ok, that didn’t work. Still having the task starting at 00:00 UTC with the cron job, even with the TZ of the host set correctly.
Also adding the option location = … to the beginning of the task did not do anything.

Any more suggestions?

32

Maybe try to make a normal cron task on the container doing e.g. touch - to see if that follows the timezone.

33

My system cron is running in local time, but task only in UTC. However, I don’t care much about when it runs, I can shift my cron expression. I actually more care about the start and stop dates to be properly aligned in local time so I do date.truncate additionally, which works just fine:

import "timezone"
import "date"
option location = timezone.location(name: "Europe/XXX")

from(bucket: bucket)
    |> range(start: date.truncate(t: -2d, unit: 1d), stop: date.truncate(t: 1d, unit: 1d))
34

Ok, so after 3 nights, my task using the cron schedule option are now “suddenly” is working properly starting at 00:00 Local Time. Really strange, but I’m happy that this now works.

Now I just tried your script, but I think using the date.truncate like this does not work. It selects the range using UTC times, so the range is offset with the timezone difference.

Also by using a stop with a “1d” offset the end date is actually 00:00 UTC tomorrow, which I think is not what you’d want.

So in my case (I’m in UTC +2 timezone), the range with your script is from 02:00 yesterday until 02:00 until tomorrow, while it should be from 00:00 yesterday to 00:00 today, using local time.

I did find a way to work around this:

import "timezone"
import "experimental"

startoffset = 26h
tzoffset = 2h

start = experimental.subDuration(d: startoffset, from: today())
stop = experimental.subDuration(d: tzoffset, from: today())

data =
    from(bucket: "bucket")
        |> range(start: start, stop: stop)
data
    |> aggregateWindow(every: 1d, fn: last, createEmpty: false, location: timezone.location(name: "Europe/XXXX"))

This is of course tedious as you’d have to change the script with each daylight saving date to change the “tzoffset” variable.

35

You are right, date.truncate does not work yet with local time, we need to wait for Flux v0.165 or better v0.167 (more time related fixes). It was good enough for me as I was taking only one specific value from that day and I was more interested in aligning the_time value properly (which suprisingly works together with aggregateWindow).

36

@MarcoB i am also facing the same problem. I have noticed that, after you write a task, the first time it runs in UTC. And then subsequent tasks run in local time as per the set schedule in cron.

If you edit the task later, then again it will run in UTC for the first time and then the subsequent runs are in local time. !

37

Exactly what I also found out, that is definitely a bug.
Anywhere where this can be reported to the developers?

38

@Anaisdg
note that this is probably a bug. everytime I reboot my pc, first time the task runs at UTC. then it becomes normal.

39

So, any new progress or solution to this problem?
I encounter the same situation…