I’m having an issue where the I can run a downsample query and it works but it does not seem to work in a continuous query. Here is my steps:
# I create my retention policies
CREATE RETENTION POLICY "a_month" ON "mydb" DURATION 30d REPLICATION 1 DEFAULT
CREATE RETENTION POLICY "a_year" ON "mydb" DURATION 365d REPLICATION 1
# Create the continous query
CREATE CONTINUOUS QUERY downsamp ON mydb BEGIN SELECT mean(temp) AS "mean_temp" INTO "a_year"."downsampled_data" FROM "mydata" WHERE uuid='xyz' GROUP BY time(1m) END
# Insert data into mydata 480 sequential 1 second intervals (query not shown) for today - 1 day, and today + 1 day (I understand historical data is not handled?). Telegraf is used for batching.
# Confirm retention policies (show retention policies), confirm continuous query (show continuous queries), and that mydata has been inserted with show series mydata and in Grafana
The above all works but no downsample data is created, however it does get created if I manually run the same downsample query:
SELECT mean(temp) AS "mean_temp" INTO "a_year"."downsampled_data" FROM "mydata" WHERE uuid='xyz' GROUP BY time(1m)
Good tips, I had not changed my continuous query settings in my influx config file, so I set them as follows:
[continuous_queries]
enabled = true
log-enabled = true
query-stats-enabled = true
# interval for how often continuous queries will be checked if they need to run
run-interval = "10s"
The link you referenced was also my issue. I ended up running a full day of data, and if I understand correctly you need to pass the current run time of the query, in my case the 10 seconds that pass the current time on the server. After testing this my data did show up. How reliable are continuous queries to not lose data points? If data is delivered outside of the correct interval it seems it will not be processed. I’m thinking how I would handle device reporting lag / etc. If I am doing time(1m) with run-interval = “10s” is this adding redundancy? My goal is to downsample all my data to 1m intervals from the second data, for instance by using a mean.
You have come across a key limitation of Continuous Queries in InfluxDB: if the time interval implied by the CQ definition has passed, the SELECT statement specified in the CQ will not be applied to that data.
In other words, CQs only operate on data points which are “recently” uploaded; in general a CQ will not process data points which are uploaded much later than the timestamps in the data points.
If your data is uploaded once per day your CQ would need to specify a RESAMPLE clause to tell it how far back in time to look for “new” data points.
