Continuous query couldn't create MEASUREMENTS

Store
1

Hello, everyone.

I’m newbie on influx db.
I try to monitoring my gitlab instance using influxdb and I need help :disappointed_relieved:

First, I use influxdb 1.2.4 version on docker container.

I have some problem on continuous query.

My cq exmaple is

CREATE CONTINUOUS QUERY rails_allocated_memory_per_action ON gitlab
BEGIN
    SELECT percentile(allocated_memory, 95) AS allocated_memory_95th,
           percentile(allocated_memory, 99) AS allocated_memory_99th,
           mean(allocated_memory) AS allocated_memory_mean,
           max(allocated_memory) AS allocated_memory_max
    INTO downsampled.rails_allocated_memory_per_action
    FROM one_hour.rails_transactions
    WHERE action =~ /.+/
    AND action !~ /^Grape#/
    GROUP BY time(1m), action
END

From log I found this.

[I] 2017-06-21T06:17:00Z executing continuous query rails_allocated_memory_per_action (2017-06-21 06:16:00 +0000 UTC to 2017-06-21 06:17:00 +0000 UTC) service=continuous_querier
[I] 2017-06-21T06:17:00Z SELECT percentile(allocated_memory, 95) AS allocated_memory_95th, percentile(allocated_memory, 99) AS allocated_memory_99th, mean(allocated_memory) AS allocated_memory_mean, max(allocated_memory) AS allocated_mem
ory_max INTO gitlab.downsampled.rails_allocated_memory_per_action FROM gitlab.one_hour.rails_transactions WHERE action =~ /.+/ AND action !~ /^Grape#/ AND time >= '2017-06-21T06:16:00Z' AND time < '2017-06-21T06:17:00Z' GROUP BY time(1m)
, action service=query
[I] 2017-06-21T06:17:00Z finished continuous query rails_allocated_memory_per_action (2017-06-21 06:16:00 +0000 UTC to 2017-06-21 06:17:00 +0000 UTC) in 5.363964ms service=continuous_querier

I guess CQ executed well however when I execute SHOW MEASUREMENTS, there isn’t rails_allocated_memory_per_action.

If I execute below query

SELECT percentile(allocated_memory, 95) AS allocated_memory_95th, percentile(allocated_memory, 99) AS allocated_memory_99th, mean(allocated_memory) AS allocated_memory_mean, max(allocated_memory) AS allocated_memory_max 
FROM gitlab.one_hour.rails_transactions 
WHERE action =~ /.+/ AND action !~ /^Grape#/ AND time >= '2017-06-21T06:16:00Z' AND time < '2017-06-21T06:17:00Z'
GROUP BY time(1m), action

it has data something like this

...

time                allocated_memory_95th allocated_memory_99th allocated_memory_mean  allocated_memory_max
----                --------------------- --------------------- ---------------------  --------------------
1498025760000000000 4.759552e+06          5.607424e+06          1.3684254117647058e+06 5.607424e+06

name: rails_transactions
tags: action=Projects::GitHttpController#info_refs.txt
time                allocated_memory_95th allocated_memory_99th allocated_memory_mean allocated_memory_max
----                --------------------- --------------------- --------------------- --------------------
1498025760000000000                                             718131.2              3.514368e+06

...

And if I execute INTO query manually,

SELECT percentile(allocated_memory, 95) AS allocated_memory_95th, percentile(allocated_memory, 99) AS allocated_memory_99th, mean(allocated_memory) AS allocated_memory_mean, max(allocated_memory) AS allocated_memory_max 
INTO gitlab.downsampled.rails_allocated_memory_per_action 
FROM gitlab.one_hour.rails_transactions 
WHERE action =~ /.+/ AND action !~ /^Grape#/ AND time >= '2017-06-21T06:16:00Z' AND time < '2017-06-21T06:17:00Z'
GROUP BY time(1m), action

rails_allocated_memory_per_action MEASUREMENTS is created however no more data stored.

I think the CQs doesn’t work well so they can’t store data to MEASUREMENTS.

If anyone has hints why this happen, I’d really appreciate it!

2

@darkrasid I think you are running into issues with Retention Policies. Can you share the output of SHOW RETENTION POLICIES ON gitlab?

3

Sure :slight_smile:

> SHOW RETENTION POLICIES ON gitlab
name        duration  shardGroupDuration replicaN default
----        --------  ------------------ -------- -------
autogen     0s        168h0m0s           1        false
one_hour    1h0m0s    1h0m0s             1        true
downsampled 168h0m0s  24h0m0s            1        false
year        8736h0m0s 168h0m0s           1        false
forever     0s        168h0m0s           1        false

Thank you

4

@darkrasid It seems like maybe if there is no data in that measurement when the CQ runs then it won’t calculate properly. Have you tried adding an EVERY ... FOR ... clause to your CQ? Another thing that might help here is adding fill() to your query as well.

5

@jackzampolin

Thank you for your hint. But when I use EVERY…FOR it wasn’t work.
And I think there are data within 1 minute.

I found something strange behavior about my CQs.

I tried to execute CQs changing group by time(1m) to group by time(2m) and then the measurements were created and everything worked well. And group by time(100s) also works but under 100s (eg 90s , 60s …) aren’t worked.

I don’t have any idea why this situation happens. When I try test it on my local machine, setting group by time(1m) is also worked well (also use docker )

Could you please provide me any clue about this issue that can help me to find the way to solve it?

6

@darkrasid From the information I have here that would seem to indicate there is some latency in metric delivery and there is no data in the measurements when your CQ is running.

7

Yes, there was latency and time difference between servers was also existed.

Using Every For , as you suggest, is solving my issue.

I’d really appreciate for you help!

Thank you

1 Like
8

@darkrasid Glad you were able to get that working!