Use Telegraf to only write reduce writes to Influx

Telegraf
,
1

I have measurements on a MQTT topic that are updated every 10 seconds.
I use Telegraf to get this data and write it to my Influx2 db. That works fine, but generates a lot of overhead writes to Influx as I only need to store the data at a 1 minute level.

What I would like to achieve is to throttle this, so to only write the last measurement every 1 minute interval.
I tested the principle with the MinMax Aggregator Plugin and that works fine, but that it is not having the “last” aggregation function.

So question is, is there another way/plugin to achieve this? Thanks in advance for any tips!

2

Hi @MarcoB,

Is the Final Aggregator plugin something you’re looking for. This should report out the “last” value in your time period.

Also wanted to verify that you did change your interval = "1m" to reduce the cadence of collecting and hence the amount of data written to InfluxDB.

3

Hi, thanks for thinking along!

I doubt however if the Final Aggregator plugin will work. As I understand from the description it only writes the data to the output when there is a time out after a series of measurements:
When a series has not been updated within the time defined in series_timeout, the last metric is emitted with the_final appended.

In my case however the data is an uninterrupted stream, with a measurement every 10 seconds. So I would say it never times-out?

Reducing the interval for collecting will indeed help in the write frequency, but not in the amount of data written to InfluxDB as MQTT/Mosquito will queue the messages until they are read.

4

Ok, to follow up, I did experiment with the Final Aggregator, but as expected this did not deliver the desired result.

Tried with this setting:

[[aggregators.final]]
  ## The period on which to flush & clear the aggregator.
  period = "1m"
  ## If true, the original metric will be dropped by the
  ## aggregator and will not get sent to the output plugins.
  ## drop_original = false

  ## The time that a series is not updated until considering it final.
  series_timeout = "9s"

But that just gives every measurement as it basically times out after every measurement. If I increase the timeout I don’t get any measurement…

So again, the goal is to just pick the last measurement every minute out of a continuous series that is has updates every 10 seconds. Anybody a suggestion for another way to achieve this? Thanks!

5

Bump: anybody an idea how to do this?