Choose retention policy

influxdb
#1

Hello!

I have influx db (v.1.6.0) with about 25 million records and retention policy DURATION 30w REPLICATION 1 SHARD DURATION 4h. Some services write data (not too much - about 100 per sec), some services read these data.
And I noticed that influx server (VDS, 8 CPU, 16G RAM) sometimes (randomly) starts to serve queries very long time (~1min and above). It causes to go down all services :frowning: Queries aren’t complicated, with simple WHERE conditions

Is it because of wrong retention policy? May be I should change it?
I mean if WHERE clause in query has time period more 4h - is it can be the reason of overhead?

#2

Have you had a chance to review the best practices for shard group durations? If your typical query is 5h you might want to consider changing your shard duration to 10h for example. If you’re SGD is only 4hs and you’re querying for a longer period, that could definitely be a contribution to long serve time. You might want to take a look at the default shard group durations to get a feel for the relationship between SGD and RPD. Also, your RPD could be too long if you’re not frequently querying data in all of that period. Finally, have you investigated enabling TSI? It could be worth looking into depending on your cardinality.