Column type panic when rewriting a collection of query statements as function block

Fluxlang
1

I’m slowly learning Flux and have bumped into an error I can’t resolve. I’m unsure if I’m misunderstanding the language itself, or if I’ve found a bug. My environment is InfluxCloud v2 Beta.

In short, an otherwise successful query produces a panic when I attempt to rewrite it as a function block. The raw, successful query is immediately below followed by my attempt at a function block version of it.

The function block produces this error: panic: column _value_reading:string is not of type float.

Have I structured the function incorrectly? I can provide snippets of the relevant tables produced by the various statements if it’s helpful.

A handful of searches in the open issues for the Flux project in Github did not result in any obviously related bugs. Then again, I’m still learning the language and am quite unfamiliar with its inner workings.

Query

readings = from(bucket: "<bucket>")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "Air Temperature" and r._field == "Value")
  |> filter(fn: (r) => r.Location == "<location>")
  
validities = from(bucket: "<bucket>")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "Air Temperature" and r._field == "Validity")
  |> filter(fn: (r) => r.Location == "<location>")
  |> group(columns: ["_value"])
  |> filter(fn: (r) => r._value == "Good")

join(tables: {reading:readings, validity:validities},
     on: ["_time", "_stop", "_start", "Location", "Source", "_measurement"])

Function Block

validReadings = (tables=<-, measurement, location, validity) => {
  source = tables
    |> filter(fn: (r) => r._measurement == measurement)
    |> filter(fn: (r) => r.Location == location)
  
  readings = source
    |> filter(fn: (r) => r._field == "Value")

  validities = source
    |> filter(fn: (r) => r._field == "Validity")
    |> group(columns: ["_value”])
    |> filter(fn: (r) => r._value == validity)

  return join(tables: {reading:readings, validity:validities},
              on: ["_time", "_stop", "_start", "Location", "Source", "_measurement"])
}

from(bucket: "<bucket>")
  |> range(start: -1h)
  |> validReadings(measurement:"Air Temperature", location:"<location>", validity:"Good")
2

@mkarlesky You’re definitely headed down the right path. It just looks like you’re running into some type conflicts in your readings column. I’m not sure what data type this column currently is, but you could cast it to a float. Try this:

validReadings = (tables=<-, measurement, location, validity) => {
  source = tables
    |> filter(fn: (r) => r._measurement == measurement)
    |> filter(fn: (r) => r.Location == location)
  
  readings = source
    |> filter(fn: (r) => r._field == "Value") |> toFloat()

  validities = source
    |> filter(fn: (r) => r._field == "Validity")
    |> group(columns: ["_value"])
    |> filter(fn: (r) => r._value == validity)

  return join(tables: {reading:readings, validity:validities},
              on: ["_time", "_stop", "_start", "Location", "Source", "_measurement"])
}
3

Thank you, @scott. Your suggestion was helpful. It did resolve the column panic, or, rather — since a recent update to Influx Cloud v2 — the generic “internal error.”

However, now that the function executes to completion, it returns no results. Like my original question, the most curious aspect of this is that a nearly identical piecemeal set of statements runs successfully and as expected. As soon as those statements are repackaged as a function I get no results.

What is the difference between the following (slight updates to my original posting)? I think the former is logically identical to the latter. Yet, the former executes as expected while the latter does not.

Further, the first set of statements below does not require the use |> toFloat() in order to run successfully while the function version of the same does require this.

I’m either missing a key piece of syntax / Flux usage, or I’m bumping up into some kind of Flux bug.

Piecemeal Statements

readings = from(bucket: "<bucket>")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "Air Temperature" and r._field == "Value")
  |> filter(fn: (r) => r.Location == "<location>")
  |> drop(columns: ["host"])
  
validities = from(bucket: "<bucket>")
  |> range(start: -1h)
  |> filter(fn: (r) => r._measurement == "Air Temperature" and r._field == "Validity")
  |> filter(fn: (r) => r.Location == "<location>")
  |> drop(columns: ["host"])
  |> group(columns: ["_value"])
  |> filter(fn: (r) => r._value == "Good")

join(tables: {reading:readings, validity:validities},
     on: ["_time", "_stop", "_start", "Location", "Source", "_measurement"])

Function Block

validReadings = (tables=<-, measurement, location, validity) => {
  readings = tables
    |> filter(fn: (r) => r._measurement == measurement and r._field == "Value")
    |> filter(fn: (r) => r.Location == location)
    |> drop(columns: ["host"])
    |> toFloat() // Note: Only needed here, preceding works without

  validities = tables
    |> filter(fn: (r) => r._measurement == measurement and r._field == "Validity")
    |> filter(fn: (r) => r.Location == location)
    |> drop(columns: ["host"])
    |> group(columns: ["_value"])
    |> filter(fn: (r) => r._value == validity)

  return join(tables: {r:readings, v:validities},
              on: ["_time", "_stop", "_start", "Location", "Source", "_measurement"])
}

from(bucket: "<bucket>")
  |> range(start: -1h)
  |> validReadings(measurement:"Air Temperature", location:"<location>", validity:"Good")
4

This may very well be a bug. I’m not 100% sure. Do you have any sample data I could test against?

6

@scott Happy to provide that. In what format and how do I get it to you?

7

CSV would be fine. You could upload a sample CSV file here.

8

Influx Snapshot.csv.txt (14.6 KB)

@scott Attached is a 10 minute snapshot of real data that’s been edited slightly to anonymize it as well as ensure it is representative. Location and Source are tags. Value and Validity are fields. While Validity should be a tag, we haven’t fixed this in our schema yet. As such, in the meantime, it makes for an interesting Flux learning challenge to extract only Values of a given Validity.

Thanks again for your help.

(The forum upload tool forced me to add a .txt extension to the filename.)

9

Thanks @mkarlesky. If you’re trying to extract rows with only values of a certain validity, this can be simplified by using pivot().

validReadings = (tables=<-, measurement, location, validity) =>
  tables
    |> filter(fn: (r) => r._measurement == measurement and r.Location == location)
    |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value")
    |> filter(fn: (r) => r.Validity == validity)
    // This maps the 'Value' column the '_value' column, but preserve existing columns
    |> map(fn: (r) => ({ r with _value: r.Value }))

from(bucket: "<bucket>")
  |> range(start: -1h)
  |> validReadings(measurement:"Air Temperature", location:"<location>", validity:"Good")

Try this ^

10

@scott Thank you! That worked (after fixing a syntax thing)! Clearly, I have some more learning to do on the right ways to slice and dice tables. Now I know the direction to take on that.

The proper approach you’ve provided notwithstanding, why in my previous approach did the raw statements succeed while the function block version did not? Was my errant approach revealed to be wrong by way of the function? Or, is there an actual Flux bug in there somewhere?

Many thanks again.

11

Sorry, I found my syntax error and fixed it in the sample above. I realized you were going down a path that I’ve been down before – using a join to align multiple fields to common times. This is my primary use case for pivot() :smiley:.

I’ll have to do a little more testing with your function block. I’m not sure if it’s a bug or if there’s something in the function that is causing it to not output any data.