Troubles with cardinality

influxdb

#1

Hello,

First of all i’,m going to explain the scenario.
We are working in a Industry 4.0 project that need to collect all the signals of a machine (we insert every 100ms, new data to our mesuarement)
In the begining ours mesuarement structure was the next:

Tags:

  • Location: (Part of the machine, example: Robot1, Robot2 etc…)
  • Line: (Line where is located the machine)
  • Status: (This tag is for the quality of the Signal)

Fields:
-Signals (more than 400)

With this configuration we has a good performance (when querying)

The troubles appear when we need to store a traceability Code (it’s a unique string to identify the piece that it’s in the location in the moment of get the data of the signal)

After change the structure of the measurement and added a new Tag for the TraceabilityCode we collect data during 12h and we has 59 000 differents series and our performance became extremely poor (Time of queries change from 1s to 1m)

In this moment we have 3 posible solutions but i don’t know what is the best option.

Option 1: Enabling the new time series index
pros: small and fast change
cons: With our volume of data i’m scare what happen in the next months (we have a yearly policy)

Option 2: Change measurement structure and insert the traceability code as a field
pros: Recover performance
cons: We need to create an aditional field for each signal (example: Temperature_TraceabilityCode) and this duplicate our volumen of data. And another against is difficult to do the queries (Exmaple: get all the signals for a determinate traceabilityCode)

Option 3: Create another measurement only for the traceability

Tags:

  • Field (The name of the signal)
  • Location (Part of the machine, same that the other measurement)

Fields:

  • TraceabilityCode

pros: Recover performance and more easy to query
cons: More volume of data and another measurement.

What do you think would be the best option?

Thanks in advance,
Mario


#2

My preference would be to enable TSI. This shouldn’t put your data at risk (you can always do a full backup beforehand. There’s full instructions here for backing up and using TSI).

Changing your schema at this point isn’t ideal because you would have to rewrite all of your data, and as you mentioned, adding another measurement only increases the cardinality.