Insert Performance Bulk Data

InfluxDB 2
1

Need to preface this with I’m still new to InfluxDB.

I’m inserting a lot of data into my influx and i seem to be having some performance issues

 $point = Point::measurement($data->server_id)
    ->addTag('lat', $ipAddr->city->lat)
    ->addTag('lon', $ipAddr->city->long)
    ->addTag('country', $ipAddr->city->country->display_name)
    ->addTag('city', $ipAddr->city->display_name)
    ->addTag('protocol', $data->protocol)
    ->addTag('state', $data->state)
    ->addTag('net_in', $data->netIn)
    ->addTag('net_out', $data->netOut)
    ->addField('ip', $data->ip)
    ->time(Carbon::parse($data->date)->setTimezone($data->timezone)->timestamp);

$writeApi->write($point);

This might not be the best way to insert data, is it possible I can create an array of these points and write them all at once instead of individually? Also I think those data points need to be Tags and not Fields but unsure what performance affect that has.

Whilst this is running I seem to be getting some very strange behaviour from InfluxDB where it’ll spike then drop off and hang for a bit then continue it’s processing

image
image1843×783 238 KB

Above is a screenshot from the influxdb metrics I enabled, The Inputs drop off every few mins for no apparent reason when theres still data attempting to be inserted, The API appears to just hang during this time and do nothing

The drops on the IO line up with the memory dips also. It’s like it’s clearing down it’s memory every few mins causing massive performance issues for me. Any idea or reccomendations.

2

Hi @Dectom,

thanks for using our client.

This might not be the best way to insert data, is it possible I can create an array of these points and write them all at once instead of individually?

You can create a bulk of points by:

$point1 = Point::measurement('h2o')
            ->addTag('location', 'europe')
            ->addField('level', 2);

$point2 = Point::measurement('h2o')
           ->addTag('location', 'west')
           ->addField('level', 5);

$writeApi->write(array($point1, $point2));

Regards