Influx data design and performance issue

1

Hello,

We have a distributed system that runs on high load therefore we have experienced some data lost due to influx overriding points that had arrived in the same time exactly. Tried to move to nano seconds precision but since we work with Java, we could not find the real nano seconds in clock way. So we did something else, I fear, that might causing us performance issues.

We added a tag called distinctor, which we random a integer value between 1-1000. This insures all points are inserted an nothing got overwritten. But Influx started working slowly after sometime. Restart fix it and it started working fast again.

Can this happen because of the extra tag? it is a low cardinality value.

We are on testing phase so we are running on AWS micro machine.

Regards,
Ido

2

I would have “randomized” the nanosecon part of the timestamp, to make a nanosecond timestamp you can do: (real second timestamp, or microsecond) * 10^x + random nanosecond (0000-1000 the same you are doing but without an extra tag)