I have measurements on a MQTT topic that are updated every 10 seconds.
I use Telegraf to get this data and write it to my Influx2 db. That works fine, but generates a lot of overhead writes to Influx as I only need to store the data at a 1 minute level.
What I would like to achieve is to throttle this, so to only write the last measurement every 1 minute interval.
I tested the principle with the MinMax Aggregator Plugin and that works fine, but that it is not having the “last” aggregation function.
So question is, is there another way/plugin to achieve this? Thanks in advance for any tips!
Is the Final Aggregator plugin something you’re looking for. This should report out the “last” value in your time period.
Also wanted to verify that you did change your
interval = "1m" to reduce the cadence of collecting and hence the amount of data written to InfluxDB.
Hi, thanks for thinking along!
I doubt however if the Final Aggregator plugin will work. As I understand from the description it only writes the data to the output when there is a time out after a series of measurements:
When a series has not been updated within the time defined in series_timeout
, the last metric is emitted with the_final
In my case however the data is an uninterrupted stream, with a measurement every 10 seconds. So I would say it never times-out?
Reducing the interval for collecting will indeed help in the write frequency, but not in the amount of data written to InfluxDB as MQTT/Mosquito will queue the messages until they are read.
Ok, to follow up, I did experiment with the Final Aggregator, but as expected this did not deliver the desired result.
Tried with this setting:
## The period on which to flush & clear the aggregator.
period = "1m"
## If true, the original metric will be dropped by the
## aggregator and will not get sent to the output plugins.
## drop_original = false
## The time that a series is not updated until considering it final.
series_timeout = "9s"
But that just gives every measurement as it basically times out after every measurement. If I increase the timeout I don’t get any measurement…
So again, the goal is to just pick the last measurement every minute out of a continuous series that is has updates every 10 seconds. Anybody a suggestion for another way to achieve this? Thanks!
Bump: anybody an idea how to do this?