Queue length exceeded

InfluxDB 2
1

We are running a set of Hyper parameter training steps in parallel against an influx instance. Each of these steps spins up 16 concurrent threads to query the data base, and we have found since a recent upgrade of influx to Version 2.0.0 (41a2324) if we try and run parallel instances of these hyper parameter training steps, then the process fails with the following error message:

queue length exceeded

This seems to come from the following file (line 370)

Is there documentation about what I should do when this error occurs? Should I back off and retry the query?

2

So to work around this issue, we are now handling the error and just waiting 1 second before retrying:

tries = 0
    while tries <= 50:
        try:
            tables = self._query_api.query(
                query,
                org=self._org
            )
            break
        except:
            tries += 1
            time.sleep(1)
    print("TRIES FOR QUERYING DATA: {}".format(tries))

It retries up to 50 times, with a sleep of 1 seconds between tries. This way, everything works fine, and I’ve seen up to 9 tries sometimes in the actual run.

3

This topic was automatically closed 60 minutes after the last reply. New replies are no longer allowed.

4

@ivanscattergood This is currently under investigation. Thank you for the details.