Scraping and storing prometheus app metrics into influx

Telegraf
,
1

Hi,

We have Telegraf running as a pod scraping prometheus metrics from our application Pods. We have our inputs configured as

[[inputs.prometheus]]
name_prefix = “apps_”
urls =
metric_version = 2
monitor_kubernetes_pods = true
[[outputs.influxdb]]
timeout = “15s”
namepass = [“apps_*”]
database = “apps”
retention_policy = “14d”
urls = [“{influxdb-url}:8086”]
username = “username”
password = “password”
When the data gets into the influx, we end up with only one single measurement, no tags and all the labels end up as fields…

Is it how it is supposed to be interpreted ?

The series cardinality is 285484…

thanks,

-Sreeni

2

With metric_version = 2 set all metrics will be reported to one measurement by default, all labels are reported as tags, and the Prometheus metric name is the field key.

3

Thanks a lot @daniel !! So, is the high series cardinality to be expected and normal ?

-Sreeni

4

The cardinality is going to depend on the number of unique tag/label combinations, so I can’t say for sure if what you are seeing is normal for your usage. If you feel like this is more than expected you will probably have to dive in to the actual values. As a bare minimum you are going to see one series per metric name.