Hello,
I am using Logstash 5.6.1-1 on Centos 7. I have my input configured as an HTTP input, some filters and an output writing to InfluxDB, which is on another Linux server. My output configuration is as follows:
output {
influxdb {
flush_size => 5000
host => [“x.x.x.x”]
db => "testinflux"
port => "8086"
measurement => "xyz"
allow_time_override => true
use_event_fields_for_data_points => true
coerce_values => {
"type " => “string”
“unit” => “string”
}
send_as_tags => [
"type "
“unit”
]
}
stdout {
codec => rubydebug
}
When I send my data via single process to Logstash via HTTP, I seem to only write 25 points to InfluxDB.
When I send via 2 processes, my InfluxDB seems to get 50 points on average.
When I send via 3 and 4 processes, its 75 points and 100 points on average.
After if I add more processes, it just stays on 100 points written on average (doesnt go up to 125, 150 etc). The averages seem very consistent.
I configured flush_size and changed it from default 100 to 5000. The write averages do not change at all after configuration change.
I changed the following in logstash.yml:
pipeline.workers: 4
pipeline.workers: 2
pipeline.output.workers: 4
pipeline.output.workers: 1
pipeline.batch.size: 2000
pipeline.batch.size: 125
I have 4 CPU and 8GB RAM.
Is there another configuration I need to tweak in order to write more than 100 points to InfluxDB via Logstash?I want to achieve 5000 points on average.
