Hello @bednar ,
I had written a code to read 8 files of line protocol data (nearly 1MB each, 5760 rows per file) → obtain a connection to the InfluxDB instance → perform writes in parallel using a thread-pool of size 8. For both v1 and v2, one connection is created and that same connection is passed on to the 8 threads. As of now, I am attaching snippets from the code that perform the write to InfluxDB.
Influx v1 client
Creating a connection with the v1.8 Influxdb instance.
InfluxDB influxDB = InfluxDBFactory.connect("http://localhost:8086", "root", "root");
Each data file is read in advance and stored in the data structure List<String> as accepted by Influxdb write APIs. The connection and the file data is then passed to the task given below.
class TaskV1 implements Runnable {
public InfluxDB influxDB;
public List<String> filedata;
public TaskV1(InfluxDB influxDB, List<String> filedata) {
this.influxDB = influxDB;
this.filedata = filedata;
}
public void run() {
long ingest_start = System.currentTimeMillis();
influxDB.write("temp", "autogen", InfluxDB.ConsistencyLevel.ONE, TimeUnit.MILLISECONDS, filedata);
long ingest_end = System.currentTimeMillis();
System.out.println("Data ingestion time: " + (ingest_end - ingest_start));
}
}
Similarly, for InfluxDB v2 client
Creating a connection
InfluxDBClient influxDBClient1 = InfluxDBClientFactory.create("http://localhost:8086", token, org, bucket);
Passing the connection and file data to the parallel task
class TaskV2 implements Runnable {
public InfluxDBClient influxDBClient;
public List<String> filedata;
public TaskV2(InfluxDBClient influxDBClient, List<String> filedata) {
this.influxDBClient = influxDBClient;
this.filedata = filedata;
}
public void run() {
long ingest_start = System.currentTimeMillis();
try (WriteApi writeApi = influxDBClient.getWriteApi()) {
writeApi.writeRecords("temp", "org", WritePrecision.MS, filedata);
}
long ingest_end = System.currentTimeMillis();
System.out.println("Data ingestion time: " + (ingest_end - ingest_start));
}
}
Please let me know if this is suffice or more information is needed. I had followed the influxdb-client-java page for the data-write example. One difference between v1 and v2 write is the try part in the v2 client. I am not sure if that is causing some wait leading to extended write time. I have also tried creating 8 separate connections and passing it to the individual threads. However, I got the same result. I would be happy to try out any changes that you suggest to make the v2 write comparable or better than v1 write.
Thanks in advance!
