I’m using influx_stress v1 to do stress test. As my business develops, the series will be more than 10 million. So I use stress test to simulate online writing.
One of my config as follows(I run 20 processes, difference among the processes is only one the tag servername’s prefix):
[write]
[write.point_generator]
# The basic point generator will generate points of the form
# cpu,host=server-%v,location=us-west value=234 123456
[write.point_generator.basic]
# number of points that will be written for each of the series
point_count = 151200
# number of series
series_count = 50000
# How much time between each timestamp
tick = “1m”
# Randomize timestamp a bit (not functional)
jitter = true
# Precision of points that are being written
precision = “s”
# name of the measurement that will be written
measurement = “cpu”
# The date for the first point that is written into influx
start_date = “2018-Jan-01”
# Defines a tag for a series
[[write.point_generator.basic.tag]]
key = “hostname”
value = “serverten”
[[write.point_generator.basic.tag]]
key = “location”
value = “us-west”
[[write.point_generator.basic.tag]]
key = “vip”
value = “1.1.1.1”
# Defines a field for a series
[[write.point_generator.basic.field]]
key = “bandwidth”
value = “float64” # supported types: float64, int, bool
[[write.point_generator.basic.field]]
key = “first_byte_num”
value = “float64” # supported types: float64, int, bool
[[write.point_generator.basic.field]]
key = “hit_num”
value = “float64” # supported types: float64, int, bool
[[write.point_generator.basic.field]]
key = “reqs”
value = “float64” # supported types: float64, int, bool
[[write.point_generator.basic.field]]
key = “speed_1-2M”
value = “float64” # supported types: float64, int, bool
[[write.point_generator.basic.field]]
key = “status_code_200”
value = “int” # supported types: float64, int, bool
[[write.point_generator.basic.field]]
key = “status_code_206”
[write.influx_client]
[write.influx_client.basic]
# If enabled the writer will actually write
enabled = true
# Addresses is an array of the Influxdb instances
addresses = [“localhost:8086”] # stress_test_server runs on port 1234
# Database that is being written to
database = “stress3”
# Precision of points that are being written
precision = “s”
# Size of batches that are sent to db
batch_size = 1000
# Interval between each batch
batch_interval = “20s”
# How many concurrent writers to the db
concurrency = 10
# ssl enabled?
ssl = false
# format of points that are written to influxdb
format = “line_http” # line_udp (not supported yet), graphite_tcp (not supported yet), graphite_udp (not supported yet)
As time goes, RES goes. As next two lines show: shared memory takes a lot (> 50% of RES).
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31662 influxdb 20 0 21.6g 16g 9.4g S 110.9 13.3 127:54.15 influxd
I analysis /proc/31662/smaps and find that, tsm take muck shared memory, like this:
VmFlags: rd wr mr mw me ac
7f584bb5c000-7f58cc12f000 r–s 00000000 08:03 72221230 /home/work/influx/data/stress3/autogen/30/000000256-000000004.tsm
Size: 2103116 kB
Rss: 1131788 kB
Pss: 1131788 kB
But, I don’t why, tsm are mmap-files, they should take virtual memory instead of physical memory except they are read. Moreover, who’s share memory between influxd?
