Can a CQ be triggered or run manually for a given time range?

1

We are in the process of migrating our measurement data storage from PostgreSQL to InfluxDB, for all sorts of reasons which I won’t go in to here. InfluxDB’s “Continuous Query” (CQ) mechanism looks very useful but it poses some problems for us.

Our “myWatt” monitoring device collects measurement data every minute and stores it internally. Periodically it uploads batches of measurements to our server. Ideally the uploads will occur every few minutes (typically at 5-7 minute intervals) but sometimes the myWatt device will be off-line for a longer period and so the measurement batch when it arrives on our server will contain measurement timestamps that are long in the past. (myWatt can store a couple of weeks’ data internally so the measurement timestamps could be that old.)

I have defined a CQ which calculates aggregated mean, max and min for selected raw data fields. I would like to calculate the same mean/max/min fields for raw data containing timestamps which are outside the range of timestamps which the CQ would see. One way to do this would be to run the same SELECT INTO query in our application logic when it detects that the measurement batch contains “old” data. I’m sure this would work but it means that if we want to redefine how we aggregate the data we will have to update our application twice: once to update the CQ definition, and again in our application logic.

So it would be very useful to be able to trigger a pre-defined CQ for a specific time range and measurement series. I have done a bit of research and it appears this has been requested before but the InfluxDB developers have resisted implementing it. Or have I missed something?

Thanks,
Jeremy Begg

2

@JeremySTX We see this usecase as handled by SELECT INTO queries. Are the CQs missing data or is some of your data coming in out of order?

3

Hi Jeremy,

There used to be an endpoint for triggering CQs but it was removed in the 1.1 release. We think it’s a useful feature and I have opened a feature request, which I’m in the process of fleshing out. Your input would be welcomed / appreciated! Allow manual triggering of CQs · Issue #8115 · influxdata/influxdb · GitHub

Thanks,
David

1 Like
4

Thanks Jack, I noticed that comment in the discussion on Github and I understand why you might think that’s OK.

My problem is that the SELECT INTO which I want to write could end up being rather complex and I don’t want to have to maintain multiple copies of it - one for the CQ, one in my application code, and possibly others in any ad-hoc scripts we write for miscellaneous measurement storing. (For example, we have had customers who have been collecting the same data using other methods and then hand us a CSV containing a couple of years of measurements and want us to take over.)

Regards,
Jeremy Begg