I deployed 1.3.1 yesterday on a HA setup and ran twice into the same issue that caused a halt on both instances at different times.
It looks like a race condition when retrieving tag values and writing new tags. These are ephemeral time series, in the order of 20000 new unique tags per day.
The error I am seeing is: fatal error: concurrent map iteration and map write.
I switched back to the inmem setting for the index version. I never saw this issue with the inmem setting and version 1.2.1.
If you’re familiar with Github, that would be the best place to continue the investigation into this. Of course if it’s easier for you to add more details to this community thread that’s fine too.
Questions I have are:
Do you have your influxdb logs that contain the panic and the stack trace? They would be helpful. The more log history leading upto the panics the better. If you don’t want to post them on Github/here then you can email them to me. My address is my Community username @influxdb.com
How many series and measurements do you have in your problematic database?
