# How is data internally stored? How to consider Line Protocol?

**URL:** <https://community.influxdata.com/t/how-is-data-internally-stored-how-to-consider-line-protocol/7685>\
**Category:** Systems\
**Tags:** influxdb, influxdata, schema\
**Created:** [November 30, 2018, 9:03am UTC](https://community.influxdata.com/t/how-is-data-internally-stored-how-to-consider-line-protocol/7685 "2018-11-30T09:03:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![luca15f](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/luca15f/32/2496_2.png) [@luca15f](https://community.influxdata.com/u/luca15f)\
**Post date:** [November 30, 2018, 9:03am UTC](https://community.influxdata.com/t/how-is-data-internally-stored-how-to-consider-line-protocol/7685/1 "2018-11-30T09:03:09Z")

</div>

Hi all,

I am a new InfluxDB user, and, especially, a new InfluxDB features student.  
I am interested in how data is actually stored:

- I know that data is _physically_ stored in TSM files; what can I find in their data blocks?? Is data still in Line protocol, or in some relative compressed form?
- When writes come to Influx, what gets actually written into WAL files? just log information or actual data? If the latter, what role do caches have?
- How do TSM files indexes work? I mean, the indexes in the index block present in each TSM file.
- For each record in line protocol, timestamp is optional. If I omit it, how can i display time series data in a correct order?

By the way: my series data is stored in tabular CSV files. How can I easily convert them to line protocol? Do I have to create my own script, or there is some other way? I installed the whole TICK stack on my machine.

Thanks,  
Luca.

---

<div class="post-metadata">

**Author:** ![katy](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/katy/32/855_2.png) [@katy](https://community.influxdata.com/u/katy)\
**Post date:** [December 4, 2018, 7:15pm UTC](https://community.influxdata.com/t/how-is-data-internally-stored-how-to-consider-line-protocol/7685/2 "2018-12-04T19:15:42Z")

</div>

Hi!

I would start by watching [this video](https://youtu.be/2SUBRE6wGiA) explaining some of the internals. I’m not sure if it covers all of your questions, but I think it’s a good start. 🙂

---

<div class="post-metadata">

**Author:** ![gunnar](https://avatars.discourse-cdn.com/v4/letter/g/ed655f/32.png) [@gunnar](https://community.influxdata.com/u/gunnar)\
**Post date:** [December 4, 2018, 7:24pm UTC](https://community.influxdata.com/t/how-is-data-internally-stored-how-to-consider-line-protocol/7685/3 "2018-12-04T19:24:59Z")

</div>

This [series of blog posts on InfluxDB internals](https://www.influxdata.com/blog/influxdb-internals-101-part-one/) may also be interesting and probably summarizes what’s in the video linked by Katy.

If you’d like to go even deeper, check out the [TSM design document](https://github.com/influxdata/influxdb/blob/master/tsdb/engine/tsm1/DESIGN.md) and the [code](https://github.com/influxdata/influxdb/tree/master/tsdb/engine/tsm1). I think there is also a TSI design document floating around somewhere, but I can’t find it right now. The code for TSI is [here](https://github.com/influxdata/influxdb/tree/master/tsdb/index/tsi1).

---

<div class="post-metadata">

**Author:** ![luca15f](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/luca15f/32/2496_2.png) [@luca15f](https://community.influxdata.com/u/luca15f)\
**Post date:** [December 6, 2018, 9:23am UTC](https://community.influxdata.com/t/how-is-data-internally-stored-how-to-consider-line-protocol/7685/4 "2018-12-06T09:23:21Z")

</div>

Thanks guys, shared links have been very helpful to me.