Override retention_policy in aggregator

Telegraf
1

Hey,
not sure how extreme is what I wanted to set up, but I was thinking to use the aggregator in telegraf to have a downsampling (basicstats maybe mean only). It would simplify a lot the whole downsampling+retention policy configs in my opinion. The problem is the aggregator does not seem to support a retention_policy override.
Any suggestions? Is it really a missing feature or I am misusing this.

Thanks

2

The basic strategy would be to use the metric filtering options to route the aggregate metrics to a second influxdb output. The tag can then be removed before sending.

[[outputs.influxdb]]
  urls = ["http://influxdb.example.com"]
  retention_policy = "default"
  [outputs.influxdb.tagdrop]
    rp = ["aggregate"]

[[outputs.influxdb]]
  urls = ["http://influxdb.example.com"]
  retention_policy = "aggregate"
  tagexclude = ["rp"]
  [outputs.influxdb.tagpass]
    rp = ["aggregate"]

[[aggregator.basicstats]]
  [aggregator.basicstats.tags]
    rp = "aggregate"
3

Thanks Daniel,
this is great, thanks for the help! I will go with this!

4

@daniel I tried it out, and probably this is the expected behaviour but it is against what I wanted to do.
I have this config:

[agent]
  interval = "10s"

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  retention_policy = "default"
  [[outputs.influxdb.tagdrop]]
    rp = ["aggregate"]

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  retention_policy = "aggregate"
  tagexclude = ["rp"]
  [[outputs.influxdb.tagpass]]
    rp = ["aggregate"]

[[aggregators.basicstats]]
  period = "1m"
  stats = ["mean"]
  [[aggregators.basicstats.tags]]
    rp = "aggregate"

I see two unexpected things:

  1. the aggregator fields are present in the default retention policy, with value in every minute
  2. in the aggregate retention policy I see all the data what is in default + in every minute the aggregation. This is the bigger problem, I only want to see the aggregated data otherwise I do not gain much with this setup. I cannot use drop_original because then I will not have any data in the default retention policy.

At this point I was thinking to use fieldpass and fielddrop in the two outputs:

[agent]
  interval = "10s"

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  retention_policy = "default"
  [[outputs.influxdb.tagdrop]]
    rp = ["aggregate"]
  fielddrop = ["*_mean"]


[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  retention_policy = "aggregate"
  tagexclude = ["rp"]
  [[outputs.influxdb.tagpass]]
    rp = ["aggregate"]
  fieldpass = ["*_mean"]

[[aggregators.basicstats]]
  period = "1m"
  stats = ["mean"]
  [[aggregators.basicstats.tags]]
    rp = "aggregate"

but it does not seem to have any effect

5

@daniel sorry, it seems I cannot read, I used double where I should not, this config (for someone who is interested) works fine:

[agent]
  interval = "10s"
  omit_hostname = true

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  retention_policy = "default"
  [outputs.influxdb.tagdrop]
    rp = ["aggregate"]

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  retention_policy = "aggregate"
  tagexclude = ["rp"]
  [outputs.influxdb.tagpass]
    rp = ["aggregate"]

[[aggregators.basicstats]]
  period = "1m"
  stats = ["mean"]
  [aggregators.basicstats.tags]
    rp = "aggregate"