Influxdb rw via httpd issue when default systemd service file writes to stderr aka /var/log/messsages

influxdb
#1

Has any one investigated if influxdb httpd r/w facing scaling issues even when using “batching” with slow down writing to /var/log/messages for the writes.

We start influxdb on centos with std systemd service file and by default it goes to /var/log/messages

Need help to scale the htttp rw and how to move the STD err and STD log to different files in the standard influxdb service files? will that solve the scaling problem . its still a single file for stdout and single file stderr (max)

Appreciate any guidance in this matter.

Do I need to look to the params to scale instead

cache-max-memory-size = 1048576000
cache-snapshot-memory-size = 26214400

I am pretty sure we can scale influxdb but not sure

Attaching the gbd backtrace showing hung in httpd

gdb.txt (28.0 KB)

#2

well just by turning off http logging
by
log-enabled = false

we can scale pretty much exponentially.

But still I cannot redirect the stdout to null and std err to file !

it seems Influxdb by defualt puts everything in stderr and that goes to journal by default!

Any other tuning help to scale is helpful.