Trying to join multiple measurements

I wanna query STATUS values only if TYPE value is equal 4 on same “time window”.

When I query separate data, I got this:

from(bucket: "real-data")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "STATUS" or r["_measurement"] == "TYPE")
  |> yield(name: "last")
result table _start _stop _time _value _field _measurement ID
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T15:45:40.7Z 108 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T15:46:22.864Z 108 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T15:47:04.028Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T15:47:46.192Z 104 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T15:49:52.684Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T17:08:18.856Z 104 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T17:33:36.76Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T17:34:18.924Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T17:44:09.22Z 104 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T17:49:46.532Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T18:03:49.812Z 104 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T18:08:02.796Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T18:38:15.848Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T18:38:58.012Z 104 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T18:40:22.34Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T19:06:22.408Z 102 value STATUS 187
0 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T19:44:19.264Z 104 value STATUS 187
1 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T15:45:40.7Z 3 value TYPE 187
1 2023-08-06T15:41:12Z 2023-08-06T19:54:11Z 2023-08-06T17:33:36.76Z 4 value TYPE 187

My query has to ignore all data before 2023-08-06T17:33:36.76Z (when STATUS is equal 4).

I tried to join by time (Grafana can do this, but I can’t reproduce this join by time on flux language), tried a full join, outer join, but stucked on join.

Anyone can help me advance on this topic?

I wanna create a state timeline for STATUS values if TYPE is 4.

Yes I can Help You :man_superhero:

there are 2 ways to accomplish this, the performance may vary significatively.

  1. Pivot
from(bucket: "real-data")
 |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
 |> filter(fn: (r) => r["_measurement"] == "STATUS" or r["_measurement"] == "TYPE")
 |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
// I suggest not forgetting the windows to make your query efficient. choose the every parameter you preffer I.E. 10s
 |> pivot(rowKey:["_time"], columnKey: ["_measurement"], valueColumn: "_value")  
 |> filter(fn: (r) => r["TYPE"] == 4)  
 |> yield(name: "last")

  1. Join

The Key to make Join work is that you should have 2 different queries and the grouping should be the same.

import "join"

 Status =
 from(bucket: "real-data")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false) 
// I suggest not forgetting the windows to make your query efficient. choose the every parameter you preffer I.E. 10s
  |> filter(fn: (r) => r["_measurement"] == "STATUS" )
  |> group()
Type =
 from(bucket: "real-data")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)  // I suggest not forgetting the windows to make your query efficient. choose the every parameter you preffer I.E. 10s
  |> filter(fn: (r) => r["_measurement"] == "TYPE" )
  |> filter(fn: (r) => r["_value"] == 4  )  
  |> group()
Result =
  join.left(
    left: Type,
    right: Status,
  on: (l, r) => l._time == r._time,
    as: (l, r) => ({l with _valule2: r._value}),
)
  |> yield(name: "last")

Hope it works at the first try

Thank you @fercasjr but I got this results, unfortnely, wrong:

For 1. Pivot:

_start _stop _time _field ID _measurement STATUS TYPE
2023-08-06 06:27:05Z 2023-08-06 10:33:04Z 2023-08-06 08:27:22Z value 99 STATUS 108 3

For 2. Join:

_start _stop _time _field ID _measurement _value TYPE
2023-08-06 06:27:05Z 2023-08-06 10:33:04Z 2023-08-06 08:27:22Z value 99 STATUS 108 3

Join is closer.

Remembering, I had to go from this:

Original query:

from(bucket: "real-data")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "TYPE" or r["_measurement"] == "STATUS")
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
  |> yield(name: "last")

Original result:

group _measurement _field _value _time ID
0 STATUS value 3 2023-08-04T12:38:13.065Z 88
0 STATUS value 9 2023-08-04T12:38:54.062Z 88
0 STATUS value 3 2023-08-04T12:39:35.059Z 88
0 STATUS value 3 2023-08-04T12:40:16.056Z 88
0 STATUS value 5 2023-08-04T12:40:57.053Z 88
0 STATUS value 3 2023-08-04T12:42:19.047Z 88
0 STATUS value 5 2023-08-04T14:09:05.666Z 88
0 STATUS value 3 2023-08-04T14:34:22.555Z 88
0 STATUS value 3 2023-08-04T14:35:03.552Z 88
0 STATUS value 5 2023-08-04T14:45:18.507Z 88
0 STATUS value 3 2023-08-04T14:50:05.486Z 88
0 STATUS value 5 2023-08-04T15:05:07.420Z 88
0 STATUS value 3 2023-08-04T15:09:13.402Z 88
0 STATUS value 5 2023-08-04T15:39:17.270Z 88
0 STATUS value 3 2023-08-04T15:41:20.261Z 88
0 STATUS value 3 2023-08-04T16:07:18.147Z 88
1 TYPE value 2 2023-08-04T12:38:13.065Z 88
1 TYPE value 4 2023-08-04T14:34:22.555Z 88

Expected result:

_time STATUS TYPE
2023-08-04T12:38:13.065Z 3 2
2023-08-04T12:38:54.062Z 9 2
2023-08-04T12:39:35.059Z 3 2
2023-08-04T12:40:16.056Z 3 2
2023-08-04T12:40:57.053Z 5 2
2023-08-04T12:42:19.047Z 3 2
2023-08-04T14:09:05.666Z 5 2
2023-08-04T14:34:22.555Z 3 4
2023-08-04T14:35:03.552Z 3 4
2023-08-04T14:45:18.507Z 5 4
2023-08-04T14:50:05.486Z 3 4
2023-08-04T15:05:07.420Z 5 4
2023-08-04T15:09:13.402Z 3 4
2023-08-04T15:39:17.270Z 5 4
2023-08-04T15:41:20.261Z 3 4
2023-08-04T16:07:18.147Z 3 4

I was expecting for type measurement fill each status reading (originally, type appear only 2 times on this time)

Join and pivot looks promising, but I can’t make they work properly, I tried fill and experimental.fill too but can’t make work together.

I just solved:

from(bucket: "real-data")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "STATUS" or r["_measurement"] == "TYPE")
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
  |> pivot(rowKey:["_time"], columnKey: ["_measurement"], valueColumn: "_value")  
  |> fill(column: "STATUS", usePrevious: true)
  |> fill(column: "TYPE", usePrevious: true)
  |> yield(name: "last")

I was not using “COLUMN” on fill.

Thank you for your help @fercasjr

Hello @bash Thank you for your question! and solution.

@fercasjr Thank you so much for your contribution!

Your welcome, I am glad to help.