We are running a single node instance of InfluxDB on a computer with the following configuration
Memory :- 32 GB
Disk :- 2 TB SSD
Core : 4
OS - Ubuntu 16.06
Influx version :- 1.3.2
Every second we are writing 8000 data points . Each data point has one tag set and 200 field set.We are using the GoLang client to insert data in to database . During insertion, after running for a while , the write operation fails with the following error - “httpClient.Write{“error”:“timeout”}\n”
Is there a limit o the number of data points that can be written in one second
BTW this generally means that the requests are not getting processed as quicker as you have the data available. In your case may be consider using a UDP client. Also you should check the schema, 200 fields seems pretty big number.
Thanks… we extended the time out… seems to be stable now… Is there a typical recommendation on the number of fields in our scenario. The following is our scenario
Memory :- 32 GB
Disk :- 2 TB SSD
Core : 4
OS - Ubuntu 16.06
Influx version :- 1.3.2
Tags per data point :- 1
Fields / data point :- between 100
Number of datapoints per second : 8000
The number of fields is high but not too high. Just keep in mind that writing 8000 points with 200 fields per batch means approximately 1.6M values are being written per second. This number of points in a request will take some time to process. The more important consideration is InfluxDB’s response times for write requests remain constant over time, which means InfluxDB is keeping up with the load.
To reduce the response time, I’d recommend identifying which resource is currently the bottleneck. I would guess CPU is probably limiting factor for your current setup. If your CPU usage is very high, I’d recommend doubling the number of CPU cores and see if that improves the request response times.
