Sensor Data - Series Cardinality

#1

We are evaluating Time Series databases for our sensor data. We have a use case to capture sensor data from up to 6 million sensors. Each sensor has the following information that we need to capture:
“deviceid, location, units, status, value, readTimestamp”.

DeviceID - Upto 6 million
Location - a million different locations
units - 10 - 20 units
Status - 5-10 status codes

Queries will be grouped by deviceId’s or deviceId+units or deviceId+units+status - All grouped by timestamps. One of the requirements is to be able to create graphs and charts.

I am aware of the series cardinality in influx. I love everything about InfluxDb but is this the right solution for my problem? Appreciate any insight so i can decide if i should pursue this line of thought.

Thanks
Srini

#2

@srini-raju This usecase is exactly what InfluxDB was meant to solve!

A single instance of InfluxDB can handle between 5-10 Million series. More than that and you will need a cluster. One thing to note about those tags is that some are dependent on others. For example Location will not add series cardinality because DeviceID has more values and does not change Location.

Status sounds like it varies with DeviceID so that will add additional series. Can you give me some more information about units? Is that tag dependent on DeviceID?

Jack

#3

@jackzampolin, thanks for the response. Like you mentioned a device will be in one location , so device cardinality comes into play here. Each device though may have a few statuses at any given point. So that might add to cardinality. Units is tied to deviceId. A deviceID will have one unit at any given time, so might not add to cardinality.

So, 5-10 milion is fine for the latest version of influx? I remember it was around 100k in version 0.9.
Another question: Is cardinality limitation of 5-10 million within a measurement or across DB?

Thanks
Srini

#5

@srini-raju It is across an instance

#6

Hi @jackzampolin,
Does the retention policy have an effect on series cardinality, i.e. if series are dropped due to the rp in effect, is the series cardinality reduced as well? Thus, the series cardinality impact could be controlled by the retention policy?

Thanks,
Flavio

#7

@monfla00 Yes it can! Thats how RPs are designed to work.