Query all tag changes over time

, ,
1

Hi there,

I’m wondering if there is any way to query all tag changes over time?

I know a bit about the internal storage format of Influx and see that this will be quite an expensive operation. I don’t really mind about it, as I need to perform this operation rather seldom. I also know that I can query all different tag values by SHOW TAG KEYS and SHOW TAG VALUES, but this doesn’t fit my usecase.

In general, my question is: “At which time did the previous row had other tags then the current one?” In order to get the information, I somehow need to order all data over all time, not only per series. (My sensors can change one tag for only one day and change it back to its previous tagset afterwards. And I don’t have any control over writing data to Influx, but only reading).
Once I have the ordered data over time, I then need to iterate over all rows and detect for which row, the previous row had another tagset and thus receive my desired row “where a previous tag was different to the current one”.

Do I have to do this entire operation on my own, meaning the merging and iterating over all rows, or can I get some help from Influx in terms of writing a better query then just SELECT * FROM myMeasurement?

Once this is done, can I do the same kind of operation in Kapacitor? So for each row, I don’t really need a window of fixed time but only the previous datarow. And can this operation react on time series changes back in time accordingly? i.e. one sensor didn’t send data due to a lost internet connection => I add this data a few days later => Kapacitor script detects if and when there were tag changes compared to the previous row (Which is for one of the newly added rows a comparison to an “old” row in Influx).

I know, I speak a lot in terms of rows which is more a SQL storage then a columnar storage like Influx is, but I think, for this one problem, the problem can be described in an easier to understand way.

Best regards

2

@theo As you mention late in your post the row metaphor is more applicable to SQL than to InfluxQL. Also what do you mean by tag changes? Are you adding new Tag Keys, or just new Tag Values? Because we organize data based on the series key (measurement + tag keys + tag values) it makes this kind of analysis difficult.

3

@jackzampolin Yeah I know this is rather difficult as I mentioned in “I know a bit about the data storage layout”.

What I want: I don’t want speficially to track new tag keys or new tag values, but each “tag change” over time. I might have something like “status” where status is almost always “ok”, but sometimes “invalid”. So I want to know at which point in time, I got my first “invalid” status and at which point, it was “ok” again. But this is not related only to status but if any tag changes (e.g. did the unit change?). I don’t care about the values itself, but only the “meta data” in form of tags. And currently, I don’t see a way to query this except querying all data, sort it by myself over time and then iterate over it.

I hope that there is at least some kind of index in InfluxDB such that I can get an ordered list over time, not only over different series. I think, it is quite common that one wants to have an ordered list over all time, not just per series (See my use case with status tag).

4

Sorry to revive this really old post, but did you find a solution to this?