Influx DB GO client - too many open files

Store
,
1

We have installed tick stack using the following link -
How To Monitor System Metrics with the TICK Stack on Ubuntu 16.04 | DigitalOcean .

The specs of the computer in which TICK stack is installed is given below
RAM :- 32GiB
cores : -4
Processor :- Intel(R) Core™ i5-6500 CPU @ 3.20GHz

We have developed a Go Client which is running on a machine with following configuration
RAM :- 32GiB
cores : -4
Processor :- Intel(R) Core™ i5-6500 CPU @ 3.20GHz

The Go client utilizes the 4 cores and starts up 5000 concurrent go routines which writes to a InfluxDB database with string tag and a integer values. When run this app , we loose a lot of data because of following error - too many open files

Can any one help us on how to fix this

Regards
Sabarish

2

@sabarishs It sounds like there is some inefficient code in the client you wrote. Can you share the repo with that client in it? Without more information I can’t help you further.

3

Change the maximum number of open files ( file descriptors) using ulimit -a . Also ensure that Influxdb connections are gracefully closed at the end.

4

You are right, the code was ineffecient, we refactored it

5

Hi

also you can edit /etc/security/limit.conf
and add at the end of the file the following lines

  • soft nofile 300000
  • hard nofile 300000

This will set maximum open files to 300000 so you wont get any issues…
be careful with the number cause after 1M a lot of issues can pop up. not always but happens.
Also you can improve linux kernel on /etc/sysctl.conf …check on google.com best prectices.