Unable to use the built-in _monitoring bucket

InfluxDB 2
1

Hello,

I’ve just setup my first InfluxDB 2 system to support some customer issues using Influx Cloud through load testing and performance optimisation.

However despite my continued efforts, I have been unable to get basic DB internal monitoring working using the _monitoring bucket as exists with InfluxDB 1 in the _metrics database.

I found this useful link pointing out that a scraper needs to be created to collect the Prometheus metrics from the /metrics endpoint and stuff them into the _monitoring bucket; however when I attempt to do this with the admin user, I do not have the possibility to select the _monitoring bucket as the destination bucket.

Further more, the _monitoring bucket just shows up as a system bucket where I am unable to change/set anything on it.

It might be suggested to just create another bucket, but the InfluxDB 2 OSS monitoring solution coming with InfluxDB 2 uses the _monitoring bucket.

I’d appreciate some help as it is quite frustrating to get stuck on such basic system monitoring.

Regards,

Greg

2

Hello @geva,
Yes the montioring bucket is mostly reserved for metrics to be collected about influxdb tasks and the UI uses that bucket to populate elements.

You can create any bucket you want though and use that instead. I would try doing that. I would refer to our docs instead of other random documentation.

And then put that Flux in a task.

and write the prometheus metrics ifyou need them to any bucket of your choice with a the to() function:

If you need to store these metrics long term

4

Hello @Anaisdg

Thanks for your responses. I have of course been using the InfluxData documentation, however had to cast a wider net when this didn’t work. For example the “Monitoring” page of the official InfluxDB v2 doesn’t make any sense as it talks about requiring InfluxDB Cloud account, installing Telegraf on a machine, as well as having an InfluxDB v2 OSS version running.

We are a significant customer of yours and I am concerned at how difficult it is to get standard database monitoring for your hosted offering which we use for our customers.

If a technical support ticket would be a better way to approach this, please let me know - but we cannot continue down a path with a database which we cannot monitor effectively.

Regards,
Greg

5

Hello @geva,
Yes correct. It is considered best practice to use an additional instance of InfluxDB to monitor your Cloud instance or vice versa this is because if your instance goes down and you’re using the same one to monitor it, then you won’t be able to continue to monitor it or receive alerts.
I believe getting started monitoring via that guide doesn’t actually take that long.
Yes of course if you’re a paying customer you’re welcome to open a support ticket.
What problems have you encountered following that guide?
In a best case scenario how would you expect to perform the monitoring?
If you intend to use the same instance of influxdb to monitor the same instance, what is your plan if you encounter connectivity issues or downtime?
Why are you interested in performign this monitoring?
What metrics are important to you?

Thanks

6

HI Anais,

Wow! Lots of questions… here are answers:

I believe getting started monitoring via that guide doesn’t actually take that long.

I’m sure that it doesn’t - but I want make sure I am following the appropriate path to internal monitoring.

Yes of course if you’re a paying customer you’re welcome to open a support ticket.
OK

What problems have you encountered following that guide?

Requiring a self-hosted OSS DB in order to monitor an InfluxDB Cloud DB isn’t what I had in mind.

In a best case scenario how would you expect to perform the monitoring?

I expect that the database has some internal metrics/monitoring capabilities, or that if these come through the /metrics endpoint, that the InfluxDB Cloud offering wires up a scrape job to provide collection and dashboards so that database performance can be monitored over time.

I also expect more clarity/coherence in the explanations. It seems that the “database metrics are enabled by default” in version 1 versus version 2 actually mean two different things; whereas in version 2 they are available externally and no longer persisted. I wasn’t able to confirm nor deny this point however.

If you intend to use the same instance of influxdb to monitor the same instance, what is your plan if you encounter connectivity issues or downtime?

That might make sense longer-term; but as with any database I would expect a certain amount of relevant performance monitoring dashboards. Think pgadmin4, or a cloud database like Azure PostgreSQL – you don’t have to install or setup anything special to see generic database metrics over time.

If the database is down then there won’t be any performance metrics to collect.

Why are you interested in performing this monitoring?

Lots of reasons; troubleshooting, application performance optimisation, ensuring customer SLAs, understanding infrastructure health.

What metrics are important to you?

Standard REDS metrics (Requests, Errors, Duration, Saturation), also USE (Utilisation, Saturation, Errors). Metrics on things like ingestion, queries, compaction, garbage collection - the things that you’ve had in InfluxDB v1.

I hope that helps clarify.