Time Keys in the TSM1 database (InfluxDB Code)

1

I’m trying to solve a problem I’ve had with Influx since the early days ( see Influx Query Boundary Data (When Recording By Exception) ) for details ).

The following shows my current understanding of how things work. I might be very, very wrong and would welcome correction.

I’m looking into how time-series are looked up from the database in 1.7.9, and it seems that one of the critical steps is to go through the shards to find those files (called Locations, I believe) from file_store.go that are guaranteed to contain those timeseries entries within the minimum and maximum date range of the query. These shards may contain any number of series, but will be restricted to a certain minimum/maximum time range.

My question is, is there a per-series index on the time range within the shard, that works between shards, and allows me to travel backwards? I’m assuming it’s out there so that you can perform descending queries, but I can’t seem to find it referenced in code. Noting though that I’m not a GO developer, so I could just be missing something.

My objective is to get one record before and one record after the time range for every measurement point in the query. To do that, I’d need to…

  1. Look up the first matching record in my chosen series within the shard found using the existing algorithm.
  2. Find the previous record on that series, given that the record may be located inside a different shard. That shard needs to be returned as one of the valid Locations.
  3. 1 & 2, but this time for the last point in my time period, and ensure that shard containing the next point is included with the list of locations.

I was hoping that someone could point me in the direction of the code that (in InfluxQL ideally, or Flux if required)

  1. Determines which shards are used and
  2. Filters out result values based on the timestamp

I know that’s a lot to ask - if anyone out there has a good grasp of the Influx internals and thinks they might be able to help get this working (in 1.7.9 or 2), we might be interested in offering a contract to do so.