Tasks failing 99% of the time

[heixum]

Hi,
I am facing issues with influxdb 2.0 Tasks. I am trying to downsample my data with the following task:
"option task = {name: \"cq-raw-to-fine-mean\", every: 1m}\n\ndata = from(bucket: \"ts_raw\")\n\t|> range(start: -duration(v: int(v: task.every) * 2))\n\t|> filter(fn: (r) =>\n\t\t(r._measurement == \"hotel\"))\n\t|> filter(fn: (r) =>\n\t\t(r._field == \"p\"))\n\ndata\n\t|> aggregateWindow(fn: mean, every: 1m)\n\t|> to(bucket: \"ts_fine\", org: \"gordianet\")"

However, in most cases the run log reports the following error (with many unreadable characters):
could not execute task run; Err: unknown field type for p: <�Ds�����u�A�a,=hotel,room=103,site=27901803,�=p p= 1583784900000000000: unknown field type for p: <�Ds�����u�A�a,

Strangely enough, in some cases (once every several minutes) the task runs successfully. Any ideas?

Many thanks in advance.

[vladbragoi]

I think it happens because the data type of your computed values changes every time the task is executed. Probably you need to cast them to the same data type before to invoke the to function.

[vladbragoi]

I found the problem. This happens because the task runs indefinitely, even if data arrives or not to the bucket. Moreover, the aggregateWindow fill records with null values when data does not arrive, instead of 0 for example. So, when the to function is invoked with null values, it causes type error you get.

A solution is to pipe the aggregateWindow with fill function, such as in this example:

    from(bucket: "your-bucket")
	    |> range(start: -task.every)
	    |> filter(fn: (r) =>
		    (r._measurement == "your-measurement"))
        |> aggregateWindow(every: 5m, fn: mean)
	    |> fill(column: "_value", value: 0.0)
	    |> yield(name: "mean")

Edit:
Maybe better will be:

    from(bucket: "your-bucket")
	    |> range(start: -task.every)
	    |> filter(fn: (r) =>
		    (r._measurement == "your-measurement"))
	    |> aggregateWindow(every: 5m, fn: mean)
	    |> filter(fn: (r) => exists r._value)
	    |> yield(name: "mean")

[heixum]

Hi, thank you very much for your response.

I am not sure I understand the last part (yield). What I would like to do is to copy the mean values on a different bucket e.g.

data
    |> aggregateWindow(fn: mean, every: 1m)
    |> to(bucket: "ts_fine", org: "gordianet")

How can I achieve this with your method?

Thanks again

[vladbragoi]

Just filtering empty records:

data
    |> aggregateWindow(fn: mean, every: 1m)
    |> filter(fn: (r) => exists r._value)
    |> to(bucket: "ts_fine", org: "gordianet")

[heixum]

Many thanks, this worked!