Our use case is a lot of devices sending us very few data but very frequently. We make a prototype of data collect using influxdb, but our use case lead us to a lot of insert into influxdb with very small batch size (through java client).
Thanks to the technical paper, we know that the performance of small batch size is very poor compared to a larger batch size. Howerver, we are able to saturate our influxdb node with some very low input.
Only 3000 field values per second on a 8 CPU, 16 GB ram server are able to saturate our node.
Is there a specific approach to manage our use case ?
Also when influxdb is saturate it only returns a 500, is there not any queue process available ?
As a solution we think about using a queue mechanism in entrance that will bufferise timeseries entries before to send them to influx with bigger batch size but we want to be sure that influxdb does not have any feature that could help us better.