InfluxDB Integer Storing

1

Hello,

among FAQ I found the following info: “InfluxDB stores all integers as signed int64 data types.”
Does it mean that if I save a 1 byte variable, it will be take 8 bytes (before the compression)?

Regards

2

Hello @AleksandarN-ee,
Thanks for your question. I’m not sure, I’m asking around. Why do you ask? What’s your higher level problem/question?

3

Hello Anaisdg,

I just need to know how much memory the InfluxDB engine takes up.

Regards

4

@AleksandarN-ee,
Nothing is compressed in the in-memory cache.
That being said, if you write the one byte text representation of an integer field " temp=3 that it will be stored as an 8-byte integer if you only write one single value for that block. We use the following compression approaches for all integers: simple8b, run-length encoding or no compression at all. Finally, the compression actually occurs on the deltas between integers, not the integers themselves.

5

As I understood, several compression types are used until the data is eventually written into the .TSM file.

A couple of sentences from the InfluxDB website:

  • "When a write comes in the new points are serialized, compressed using Snappy, and written to a WAL file. "
  • “Integer encoding uses two different strategies depending on the range of values in the uncompressed data. Encoded values are first encoded using ZigZag encoding.”
  • “If all ZigZag encoded values are less than (1 << 60) - 1, they are compressed using simple8b encoding. If any values are larger than the maximum then all values are stored uncompressed in the block. If all values are identical, run-length encoding is used.
    This works very well for values that are frequently constant.”

As I understand, the data is compressed both at the .WAL and .TSM level? Am I understanding something wrong here?

6

Yes it is compressed at the WAL level. All writes in the WAL are stored using protobufs and we leave it to the library we use and the PB spec to make those decisions.