I collect time-series data through Telegraf (v1.38) and write it to both a local InfluxDB 2.7 instance and InfluxDB Cloud Serverless. On the local instance I use Flux Tasks to pre-aggregate data, but Cloud Serverless does not support a task scheduler so I am looking for an alternative.
To simplify what I want to do:
time-series data → aggregate daily / monthly / yearly totals → write those totals back as time-series data into InfluxDB
This aggregation should run every night automatically. What is the recommended way to achieve this when targeting Cloud Serverless?
Does Telegraf have a solution for this? (maybe a plugin or something)
I think the best approach in Cloud Serverless would be:
- Write a script in your language of choice that has an InfluxDB client (it’s going to take some manual work to code it for exactly your use case, but coding agents can help, as none of this is truly novel)
- Have your script run some aggregation queries on your source data, and write the results back to a different downsampled/aggregated table for easy reading later on
- Schedule this script to run as necessary with cron
- Have Grafana read from the new table your client has written to, which should be quite performant
There’s a guide in the documentation that covers a similar, but not identical, pattern.
Yeah, I already implemented something similar, but I was wondering if there’s a more native approach. Once you start adding custom code for this kind of setup, it becomes messy pretty quickly.
In my case, the project is dockerized and gets deployed multiple times with some custom configuration each time, so I’m trying to keep the repository as clean and maintainable as possible. Right now, the amount of glue code and configuration needed is starting to hurt readability and stucture.
