histogramQuantile over scraped Prometheus histogram

InfluxDB 2
1

In the latest release of InfluxDB2, I am able to create a prometheus-like histogram over a Telegraf metric (i.e. cpu system usage) by running the histogram function with a set of bins over it. By default, the upper bound column will be called “le”, and will be of type double. I can then calculate quantiles over this histogram by using the histogramQuantile function.

I want to do the same with metrics scraped by the in-built Prometheus scraper, that scrapes it’s data from an endpoint created with the official Python Prometheus client. The histograms created by that scraper have a _field column with the upper bound bin values, however, this column is of type string. This means using the _field column as upper bound column in the histogramQuantile function does not work because of incompatible types.

The toFloat function can only be used to map a type conversion over the _value column of a record set, so that’s no option either. So my question is, is it currently possible to use histogramQuantiles over scraped Prometheus metrics and how ?

2

@NielsK From what I understand, Flux’s histogram and related functions will undergo a bit of an overhaul soon.

For type-conversion, rather than using the toFloat function, which only allows converting the _value column, create a custom function using toFloat's definition:

toFloat function definition

toFloat = (tables=<-) =>
  tables
    |> map(fn:(r) => float(v: r._value))

Customized toFloat function

customToFloat = (tables=<-) =>
  tables
    |> map(fn:(r) => float(v: r._field))

Note that the custom function is using the float fucntion (needs to be documented) to map over the _field column instead of _value.

3

Thanks Scott, I had already tried something like this.

However, this custom function maps over the table, and gives the _value field the value of float(_field).

If this is the way this is intended to be used we need a function that maps over the table, and gives the_field field the value of float(_field).

4

We are going to be overhauling how the scraper works in InfluxDB2 and also the histogram functionality to make things more friendly with how Prometheus metrics works. I’d expect this to land in the next 4 weeks, but no promises since we have a bunch under development currently.

5

Thanks for the answer Paul. I had a conference call with Russ yesterday evening and he also brought this to my attention. He also said feedback was welcome on the proposed changes on Github, so I’ll read up on the proposals in detail later today.