Data in influxdb is increasing yet size of /var/lib/influxdb/data folder keeps decresing/fluctuating

I am pushing data into influxdb using telegraf with retention policy set to autogen.
I am monitoring the size of folder /var/lib/influxdb/data. I can see new entries(checked using select count(*)) being made in the influxdb database, whereas the size of /var/lib/influxdb/data folder increases a bit and then decreases (e.g. increases from 1GB to 2.2GB and then comes back to 1GB) all this happens within a minute.

My flush interval and input interval are set to 10s. Does any one has idea around it?
(influxdb version 1.7.6)

Hi @gourav1905 ,
does that happen after a restart of the database ? what is the size now ?

Hey @MarcV Can’t restart database, as data is being continuously written to influxdb through telegraf.
I am suspecting a possible reason could be because of compaction.
My hypothesis is following.:- Data from cache is being moved to data folder after certain threshold is reached. This data is not compressed. After some time or data size the compaction kicks in.
Most of the data points are similar, except the timestamp and one tag attached, which is possibly causing high level of compression.

What do you think of this hypothesis?

Hi @gourav1905,

That could be it , i am surprised this all happens within 1 minute …

@MarcV Probably because of the amount of data going into it. I have 10 telegraf instances running pushing around 850 metrics data at flush interval of 10s.

But despite this amount of data I am surprised how well data is being compressed. This setup has been running for a couple of days and the /data folder size grew only to around 3GB. (All data points are different, had added global tags to them to ensure that.)