I’m using a starlark script in my telegraf config-file.
It basically calculates the difference between the previous amount of rainfall from a sensor and the current amount of rainfall and writes it into a measurement in InfluxDB (V2).
The sensor only reports the total amount of rainfall (and overflows at a value of 255), but I finally need the amount of rainfall per hour, day, …
To keep the previous amount of rainfall I use the state-dictionary. All works fine, but there is one important drawback.Whenever I restart the telegraf service (e.g. to enable a new configuration) the values in state[ ] are reset. The previous value becomes 0, the current, absolute amount of rainfall becomes the difference. Hence, there is always a peak in recorded rainfall, when I restart telegraf.
Is there a better way to store previous values in the starlark script? Or is there a way to prevent the reset of the state [ ] content?
Since I write the previous value to the InfluxDB anyhow: is there a way to read from an InfluxDb in telegraf? ChatGPT told me it is not, but it is not always reliable …
Here is an excerpt of the code:
def apply(metric):
# if metric that contains field 'Rain' is processed
if 'Rain' in metric.fields:
current_raw = int(metric.fields['Rain'])
#check if previous values are already initialised in the state-dictonary
if not "prev_raw" in state:
state["prev_raw"] = 0
if not "prev_corr" in state:
state["prev_corr"] = 0
previous_raw = state["prev_raw"]
previous_corr = state["prev_corr"]
diff_raw = current_raw - previous_raw
#if there was an overflow (255 --> 0) in the rain-bucket counter
if diff_raw < 0:
# overflow, add 255
diff_raw += 255
current_corr = previous_corr + diff_raw
#write current values to previous values for next script execution
state["prev_raw"] = current_raw
state["prev_corr"] = current_corr
...
HI @srebhan , thank your for your reply and your questions. All very valid question, I will try some answers.
My first question here would be why you don’t compute the rate in the query or using a database task rather than computing it in Telegraf?
The simple answer is, that I’m not able to do it
Actually I tried for another use case already to compute the time difference between the last and last but one value and didn’t succeed.
It may (besides my limited knowledge) has to do with the fact, that I try to use InfluxQL as query language.
Since I’m using InfluxDB V2 SQL is not available.
Since Flux is kind of depreciated in the coming V3 I don’t want to invest in learning it.
Though I’m not completely consistent with it, since I’m using already a task, where, afaik, Flux is currently the only option. My friend ChatGPT helped me out.
That’s kind of the 2nd part of the answer. The Flux task I’m using has two issues: I don’t fully understand it (and don’t want to invest to change that fact) and it has technical issues in one function (if you are interested in the details: Tasks runs quite long and uses a lot of memory - #6 by scott).
But maybe that is still a good approach. Do you have an advice how to achieve my goal with a query or a task? Using the query I see the issue that the rain counter overflows at 255.
The easier answer is: I was not aware of it.
Now, that you have pointed it out, I have to dig in it to understand it. The overflow situation comes to my mind also here.
Actually I found that example and was under the impression that I followed it, with the exception that I did not store the complete measurement but only the value of interest.
But this example, by using the state dictionary, leads to the problems I described in my initial post (state dictionary reset when telegraf is restarted).
When Telegraf is restarted, the state is lost, this is true. You might want to add a feature request to add state persistence to the starlark aggregator…
Anyway, I really suggest to look into InfluxQL functions (especially the derivative function) as this will solve the telegraf restart part…
