Write to InfluxDB cloud fails with ApiException: (503) Reason: Service Unavailable; upstream connect error or disconnect/reset before headers. reset reason: connection failure

Interesting. I’ll check that out.

Hi @Anaisdg . How are things looking with:

  • the resolution of the 503 errors,
  • a description of what happened, and
  • a close of the issue so that we can all sleep easy knowing it’s fixed and won’t happen (at least not the same) again.

Do we have any feedback on whether the problem has been fixed? Or are we still relying on the mitigation that was applied.

Hi @Anaisdg ,
I think it would be good to update us on what’s happening with the mitigation, the fix and a brief explanation of what happened. As you know this was pretty serious and among colleagues has caused the reputation for InfluxDB robustness to take a dip.

Hello @asmith,
I apologize for the delay I thought I responded already. After upgrading some 3rd party libraries, our write request api is more stable because of decreased memory use. That was the underlying problem.

Thanks @Anaisdg for completing the thread here. I’ll mark the issue as solved.

I’m still seeing errors I’m afraid!

Removing the “solved” status then.
What are you seeing @dabeeeenster ?

https://sentry.io/share/issue/8f95b4c24dc54713b43d0723e87f38bc/
https://sentry.io/share/issue/20c71e9aa6cb4e37b7209815c9d6f29a/

What do the stats look like day to day. That may help someone from InfluxDB to take a look.