Timeout error while data input

Zsolt1 · February 27, 2019, 2:35pm

Dear members!

We have made some tests on InfluxDB and we run in problems, when inserting data while 3 continous queries are active.

These exceptions are not dropped and everything goes well, when we do the same data input so that there are not continuous queries in the database.

We would like to ask for your help.

InfluxDB runs on this virtual machine:

CentOS release 6.8 (Final)

Kernel 2.6.32-642.11.1.e16.centos.plus.x86_64 on an x86_64

Memory: 2GB

Processors: 2 core ( Intel® Core™ i5-8250U CPU @ 1.60GHz 1.80 GHz )

Hard disk: SSD 45 GB

InfluxDB on virtual machine:

InfluxDB v1.7.4

Config: Default

UI management for InfluxDB: InfluxDB Studio

100.000 Measurement with Retention policy 1RP_1h

5 fields / Measurement

Batch active: (5000, 200, TimeUnit. MILLISECONDS )

Retention policies in the DB:

name,duration,shardGroupDuration,replicaN,default

autogen,0s,168h0m0s,1,True

1RP_1h,1h0m0s,1h0m0s,1,False

2RP_1H15M,1h15m0s,1h0m0s,1,False

3RP_1H30M,1h30m0s,1h0m0s,1,False

4RP_1H45M,1h45m0s,1h0m0s,1,False

Continuous queries in the DB:

CREATE CONTINUOUS QUERY CQ1perc_RP2 ON ATALLASTESZT_REAL
BEGIN
SELECT mean() INTO “2RP_1H15M”.:MEASUREMENT FROM “1RP_1h”././ GROUP BY time(1m)
END

CREATE CONTINUOUS QUERY CQ2perc_RP3 ON ATALLASTESZT_REAL
BEGIN
SELECT mean() INTO “3RP_1H30M”.:MEASUREMENT FROM “1RP_1h”././ GROUP BY time(2m)
END

CREATE CONTINUOUS QUERY CQ4perc_RP4 ON ATALLASTESZT_REAL
BEGIN
SELECT mean() INTO “4RP_1H45M”.:MEASUREMENT FROM “1RP_1h”././ GROUP BY time(4m)
END

Continuous data input (as fast as possible, around 30000 data points /sec) with real time timestamps to InfluxDB using Java Api 2.15.

We insert a data point for each of the 100000 measurements and repeat it several times.

Values are random integers in all of the 5 fields.

After 12 minutes from the start of the data input, exceptions appeared on Java console and arised periodically:

febr. 27, 2019 1:00:19 DU org.influxdb.impl.BatchProcessor write

SEVERE: Batch could not be sent. Data will be lost

org.influxdb.InfluxDBIOException: java.net.SocketTimeoutException: Read timed out

at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:812)

at org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:463)

at org.influxdb.impl.OneShotBatchWriter.write(OneShotBatchWriter.java:22)

at org.influxdb.impl.BatchProcessor.write(BatchProcessor.java:340)

at org.influxdb.impl.BatchProcessor$2.run(BatchProcessor.java:370)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.net.SocketTimeoutException: Read timed out

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.socketRead(Unknown Source)

at java.net.SocketInputStream.read(Unknown Source)

at okio.Okio$2.read(Okio.java:140)

at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)

at okio.RealBufferedSource.indexOf(RealBufferedSource.java:358)

at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230)

at okhttp3.internal.http1.Http1Codec.readHeaderLine(Http1Codec.java:226)

at okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:202)

at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:101)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)

at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)

at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)

at org.influxdb.impl.BasicAuthInterceptor.intercept(BasicAuthInterceptor.java:22)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)

at org.influxdb.impl.GzipRequestInterceptor.intercept(GzipRequestInterceptor.java:42)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)

at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:152)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)

at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)

at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:264)

at okhttp3.RealCall.execute(RealCall.java:93)

at retrofit2.OkHttpCall.execute(OkHttpCall.java:186)

at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:800)

… 11 more

s

febr. 27, 2019 1:00:52 DU org.influxdb.impl.BatchProcessor write

SEVERE: Batch could not be sent. Data will be lost

org.influxdb.InfluxDBIOException: java.net.SocketTimeoutException: timeout

at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:812)

at org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:463)

at org.influxdb.impl.OneShotBatchWriter.write(OneShotBatchWriter.java:22)

at org.influxdb.impl.BatchProcessor.write(BatchProcessor.java:340)

at org.influxdb.impl.BatchProcessor$2.run(BatchProcessor.java:370)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: java.net.SocketTimeoutException: timeout

at okio.Okio$4.newTimeoutException(Okio.java:232)

at okio.AsyncTimeout.exit(AsyncTimeout.java:286)

at okio.AsyncTimeout$2.read(AsyncTimeout.java:241)