We are seeing a strange phenomenon with our self-hosted InfluxDB 1.7.2 instance. The symptoms are:
- Data stops “being written” to a particular measurement with a particular tag value after ~5 minutes of restarting the InfluxDB service. That is: when we query for data the query doesn’t return data after a certain point in time.
- After restarting the service, all the historical data seems to be restored somehow. That is: we execute the same query we did before and suddenly the data is there. After 5 minutes we stop seeing data.
- We use the REST api to write data; this seems to fo flawlessly. All calls get a 204 response, as we expect.
- This only seems to happen for data being written with a particular tag. Other data in the same measurement with a different tag value seem to be unaffected.
It looks like somehow WAL is not being processed correctly, but we’re completely stumped on what this issue might be.
This started when we started writing data to the measurement in question from another source and introduced a new tag value. The old data does not have any of these tag values; maybe this is a problem? Does anyone have any clues for us? Thanks!