Hello, I’m facing an issue with my Docker setup that includes a Blackbox Exporter, which exports metrics to Prometheus. From there, the data is supposed to be written to InfluxDB using Telegraf. However, I’ve noticed that the
probe_success metric is nowhere to be found in InfluxDB.
To be honest, I can find many metrics in the
prometheus measurement but not any of those that are being scraped and that I can see in the Prometheus web interface.
I can find metrics like
prometheus_http_requests_total but I can’t find that in the Prometheus web interface…
If any of you have encountered a similar issue or have some insights to share, I’d appreciate your help.
Maybe someone with specific Blackbox Exporter experience will chime in. However, from the Telegraf side here are some thoughts:
- Try using the
[[outputs.file]] output plugin to see what metrics are getting collected and ensure it is somehow not getting filtered by your config. The file output will print the metrics to stdout.
- If you navigate to the prometheus endpoint itself do you see the metric? What does it look like? Is it a string?
- Logs and configs are really nice to see to ensure we are understanding the flow of your config. It also good to include the entire config, so we can see if any processors or other filtering is involved as well.
Hey, thanks for your response!
I just checked if the metrics displayed at
http://localhost:9090/metrics match those in the Telegraf output, and they do. That’s good news on one hand, but on the other hand, I’m now puzzled as to why the metrics I see through the graphical web interface at
http://localhost:9090 are not being displayed there. Do you have any idea about this?
To be honest, I don’t need any of the metrics I can see at the
/metrics endpoint, I only need the
prometheus.yml looks like this:
- job_name: "ha-control-targets"
- source_labels: [__address__]
- source_labels: [__param_target]
- target_label: __address__
My input plugin in Telegraf looks like this:
urls = ["http://localhost:9090/metrics"]
metric_version = 2
Okay, I think I got it… Wow, sometimes I am amazed by my own stupidity…
As you can see in the in the
[[inputs.prometheus]] configuration, I’m scraping the Prometheus metrics endpoint that offers metrics about Prometheus… But I needed to scrape the endpoint of the blackbox exporter (
But still, is it possible to keep the
urls array small and organized? I got many targets and many modules with whom I’m scraping and the list would become really really long, if I entered them like this:
urls = [
Glad you got this working better!
As far as the URLs, I briefly looked at the config options. Would the
static_configs option provide you with a list of targets that you could set in your config and then you get all the results on your metrics endpoint?