InfluxDB return different amounts of data each time (Node.js)

[katinas]

Hello, I have an issue where InfluxDB return not the same number of items each query.

    let fluxQuery =

    'from(bucket:"' + bucket + '")'

    + '|> range(start: -7d) ' 

    + '|> filter(fn: (r) => r["type"] == "5")'

It’s a simple query, but every single time I get a different amount of data (sometimes 160 entries, sometimes 200 etc.). What could cause this problem ? Is it because I have a free tier of InfluxDB cloud ? (Although the queries are fine in the cloud query builder)

Thank you

[scott]

@katinas How quickly are new points written? When you use a relative duration as a start time, it is relative to now() (the current UTC time). So each time you execute the query using a relative duration, you’re actually querying a different time window. What happens when you use absolute start and stop times?

let fluxQuery =

'from(bucket:"' + bucket + '")'
+ '|> range(start: 2020-07-15T15:00:00Z, stop: 2020-07-22T15:00:00Z) ' 
+ '|> filter(fn: (r) => r["type"] == "5")'

[katinas]

Thanks for the reply.

Points are written only when I write them myself (still in testing phase), I haven’t written new points for a few days.

I tried the query you added:

I printed the number of items returned from the query, still inconsistent… Strange

I think I will try hosting influxDB locally, maybe it’s something with the cloud DB.

Thanks again

[scott]

How are you counting the returned items? In Node? I’m just curious how Node is interpreting the result set and if there’s something in the response that differs each time. When you comes to the actual data, try counting the number of rows in Flux rather than Node.

from(bucket: "example-bucket")
  |> range(start: -7d)
  |> filter(fn: (r) => r.type == "5")
  |> group()
  |> count()

[katinas]

Ok, some progress . Yes using count() in Flux does produce a consistent result (your query below), while in Node produces inconsistent results.

I am using this code to put the results into an array (similar to THIS example):

  const queryApi = new InfluxDB({url, token}).getQueryApi(org)
  let fluxQuery =
  'from(bucket:"' + bucket + '")'
  + '|> range(start: -' + days + 'd) ' 
  + '|> group(columns: ["_measurement"])'
  + '|> filter(fn: (r) => r["type"] == "' + type + '")'
  + '|> filter(fn: (r) => r["postDate"] == "' + postDate + '")'
  let data = []
  queryApi.queryRows(fluxQuery, {
    next(row, tableMeta) {
      const o = tableMeta.toObject(row)
      data.push(o)
    },
    error(error) {
      reject(utils.buildErrObject(404, error))
    },
    complete() {
      resolve(data)
    },
  })

Thanks again for the reply

[katinas]

OK I found out what was the problem. Rather than writing “start: -7d”:

from(bucket: "example-bucket")
  |> range(start: -7d)
  |> filter(fn: (r) => r.type == "5")
  |> group()
  |> count()

but writing with a specific date:

from(bucket: "example-bucket")
  |> range(start: 2020-07-25, stop: now())
  |> filter(fn: (r) => r.type == "5")
  |> group()
  |> count()

fixed the problem! I noticed that if I write in start: -15d, -20d or any other, it returned the same results (even though actual data is different), but setting a date fixed it anyways.