The basic prometheus.scrape to bucket example in the documentation does not work

Fluxlang
1

I’m trying to set up a custom scraper query, based on the info in: Scrape Prometheus metrics | Flux 0.x Documentation

A simple query of the form:

import "experimental/prometheus"

prometheus.scrape(url: "http://example.com/metrics")
    |> to(bucket: "example-bucket")

fails with the error:

 error @3:1-3:61: expected stream[#A] but found [B] (array) (argument tables)

The error is on the to line - commenting that line out results in a successful query, returning multiple tables scraped from the metrics endpoint.

I couldn’t figure out any combination of array.map or the standard map functions that seemed to work here. Am I missing something obvious?

Running the latest version: 2.2.0

Thanks,
- John

2

Hi @johnspurlock,
Welcome to the community! Can you give me an idea of the metrics you are trying to scrape? I have tested the scraping against the InfluxDB endpoint and this was successful:

import "experimental/prometheus"

prometheus.scrape(url: "http://localhost:8086/metrics")
  |> to(bucket: "TestBucket", org: "TestOrg")
3

Hi Jay! Instead of pasting my entire metrics endpoint, here is minimal example metrics text that should reproduce the issue:

# HELP metric1 description for metric 1
# TYPE metric1 counter
metric1{source="source1"} 4 1649410029674
metric1{source="source2"} 4646 1649432552071

# HELP metric2 description for metric 2
# TYPE metric2 gauge
metric2 265 1649432584080

This can be parsed successfully by the prometheus.scrape function (returns an array of tables?), and can be imported into a bucket successfully with the default basic “scraper” UI, but trying to pipe the results of this query to the to function fails.

And also weirdly, array.map does not seem to work on the results either, I thought maybe I could try calling the to function on every array item, but it blows up even on a simple identity transform:

import "experimental/prometheus"
import "experimental/array"

prometheus.scrape(url: "http://example.com/metrics")
|> array.map(fn: (x) => x)

…results in: An internal error has occurred - check server logs

Server logs include: panic: length of stream not supported

4

@johnspurlock What version of InfluxDB/Flux are you using?

5

InfluxDB v2.2.0 (git: a2f8538) build_date: 2022-04-06T17:36:40Z

6

Ok, this is a bug with current function signature of prometheus.scrape(). I just submitted an issue. Hopefully we can get it resolved soon.

Some background: Flux uses function type signatures to describe parameter, input, and output types for each function. We recently updated how streams of tables are represented in function signatures. This specific function looks like it was missed as part of that update.

7

Great, thanks! I’ll keep an eye out for a fix in the next version.

8

@johnspurlock Flux 0.164.0 just shipped with a fix for this issue. I don’t know exactly when it will make its way into the InfluxDB repo, but it should be in the next day or two. Once it’s in there, you can either build InfluxDB from source or you can use the nightly version.