Disk usage grows rapidly after a few days, cleans up with restart

1

Hi,

I have a InfluxDB 1.8.2 instances collecting data from telegraf on host, it works well but recently it has begun to consume lots of disk space, a restart frees lots of space and its fine for a few days and then it begins to grow again. Any suggestions on what to look at?

Thanks

2 Likes
2

Sounds like it could be a log file maybe?

3

I don’t think it’s log, the space used space is in the data directory, I do get some messages to syslog but I have not enabled debug level yet. And it happends after a few days of running, before the data grows as normal.

4

Oh, hey! This looks exactly like my problem that i posted a while back Strange disk space usage leak after InfluxDB 1.8.x upgrade

It seems to have something to do with shard compaction causing tsm.tmp files to be left behind, which eat up space, but are cleaned up at a restart.

Still co clue why it happens, or how to fix it. Seriously considering a downgrade to 1.7.9 just to see if that will avoid the problem

1 Like
5

Ah, thank you! I will see if can locate these large tsm.tmp files the next time it happens, you have found no solution to the problem. I saw there was a 1.8.3 ongoing in GitHub, but I could not find anything that looked related to this.

6

Hello!
I’m currently asking for help. I appreciate your patience while I look for an answer.
Thank you.

2 Likes
7

I’m also seeing this problem, it not good, I have to restart a few time a week now to have the tmp-files removed before (or after) the disk has run out of space, after a restart I’m back to 50% used again.