Handling counter / value wrap

[Jens]

My electricity, gas an water meters occasionally get exchanged, and thus the counter gets reset.
This causes a counter reset in my data, like below.

How can I create InfluxDB (with Flux) queries which simply ignore this reset, e.g. when generating differences and/or derivatives? I know about derivative and “nonNegative: true”, but that still returns a delta of 0 if it is > 0. I want to pretend this reset did not occur at all - and if possible, also without manual value hackery because I want to use this feature in many datasets.

Thank you!

[Anaisdg]

Hello @Jens,
You can filter out for the 0 values and then use cumulative sum like:

import "array"

array.from(rows: [{_value: 1},{_value: 2},{_value: 3},{_value: 0},{_value: 1},{_value: 0}, {_value: 1}])
|> filter(fn: (r) => r._value != 0)
|> cumulativeSum(columns: ["_value"])
|> yield(name:"ahhh") 

[Jens]

This yields

500 Internal Server Error: type error 1:1-1:15: unknown import path: "array"

in my setup. (Influx 1.8.10)
Where is the “array” module located?

Also, AFAIunderstand, cumulativeSum() only works if I have a measurement which counts deltas. In that case I wouldn’t need it at all because I could just exclude negative values. But I have absolute values (see screenshot above) in my measurement, so cumulativeSum() doesn’t work correctly here, it returns result values far too high.

[Anaisdg]

Hello @Jens,
I filtered out the 0 values first.
I added timestamps to help visualize it better. What I understood was that you wanted to take the data you shared in your screenshot and have the counter continue in upward form the original data instead of resetting to 0.

Do you know what version of flux you’re using? You need: Flux 0.103.0+

import "array"

array.from(rows: [{_time: 2020-01-01T00:00:00Z , _value: 1},{_time: 2020-02-01T00:00:00Z, _value: 2},{_time: 2020-03-01T00:00:00Z, _value: 3},{ _time: 2020-04-01T00:00:00Z, _value: 0},{_time: 2020-05-01T00:00:00Z, _value: 1},{_time: 2020-06-01T00:00:00Z , _value: 0}, {_time: 2020-07-01T00:00:00Z ,_value: 1}])
|> yield(name:"before")
|> filter(fn: (r) => r._value != 0)
|> cumulativeSum(columns: ["_value"])
|> yield(name:"after") 

Is there anyway you could draw on your screenshot what you’re expecting or create some dummy data input and expected output? Thanks

[Jens]

I’m sorry I did not reply to this topic earlier. Influx on my Raspberry Pi broke down FUBAR because of the 32bit > 2GB issue in Influx 1.8 and I had to reinstall and setup everything, and I haven’t imported my old data yet.

I’ll get back to this once I have my old setup running again. Thank you!

[scott]

@Jens this is exactly what increase() was designed for.

data
    |> increase()

[Jens]

@scott,

I’m actually relieved you say this. I thought this cannot have been missed, counter wrap is such a typical time series issue … I’ll get my old data online again and try this ASAP.

Thank you!