InfluxDB Performance when handling duplicate entries

InfluxDB 2
, , , ,
1

Dear Influx Community member,

I am currently implementing a project, where we collect data from machines via MQTT and a Unified Name Space. Influx will be connected via a data bridge to the MQTT broker where the UNS data model lives, from where it will ingest timeseries data.

In this particular case, data that should be loaded as fields into a single influx measurement (row) could come in over a period of a couple of minutes. My question is as follows:

  1. Could I just send the individual fields into Influx with their timestamp and tags, letting Influx taking care of adding the new fields to the already existing measurement.

  2. Or, should I organize this data in measurements myself, before inserting them into InfluxDB to minimize the need for Influx to merge the duplicate measurements?

I am mainly asking this in relation to InfluxDB performance related to merging duplicate measurements.

Some googling did get me answers about how influx handles such duplicates, but nothing about the performance impact.

I look forward to your insights!

Robbert

2

Hello @Robbert,
You can’t merge measurements. You can use tasks to transform data and rewrite it to InfluxDB. But I don’t recommend this.
InfluxDB is schema on write.
I recommend organizing the data in measurements before inserting them into InfluxDB.