Duplicate data getting pulled from Prometheus

, ,
1

I currently have prometheus set as an input, and im using influx for long term storage of my metrics.

When i query in influx the result is 2x of that in prometheus…

my telegraph config is as below, but i cant work out if this is due to the intervals or not… anyone had anything similar?

Configuration for telegraf agent

[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "3s"
flush_interval = "10s"
flush_jitter = "5s"
precision = ""
debug = false
quiet = true
logfile = ""
hostname = ""
omit_hostname = true

###############################################################################

OUTPUT PLUGINS

###############################################################################

[[outputs.influxdb]]
urls = ["{{ env “INFLUXDB_ENDPOINT” }}"]
database = "prometheus"
retention_policy = ""
write_consistency = "any"
timeout = “10s”

2

I’m not sure what could be going on, by 2x of that do you mean that a value of 2 from the source would be stored as a 4 in InfluxDB?

3

yes so if prometheus is display a value of 2 at time x then influx is showing 4 at time x… not sure if its pulling in the data twice… before i upgraded influx it was 5 times the amount… seems to grow over time…

4

Are you using an aggregation function when you query the data from InfluxDB? Perhaps multiple sources are being summed?