Hello,
I have a question regarding the synchronization between the following:
- Telegraf - Kapacitor
- Telegraf - CQs
- CQs - Kapacitor
Imagine the following scenario:
Metrics are collected every 1m through Telegraf by calling a Python script (which calls a public HTTP API).
Lets assume the average HTTP call delay is 1s and the round parameter of telegraf is set to true, then the new data is inserted into InfluxDB every hh:mm:01
I want to process arrived data as immediate as possible and write it back to the database but the way to go is currently not so obvious for me.
I think the recommended way for postprocessing is Kapacitor. So I would build a TICKScript which is called every 1m. However at the time the TICKScript is executed, the data from the same tick is not inserted into the database yet. Of course I could set the call interval of the TICKScript very low but then most of the call would be unnecessary and it hits performance.
The same dilemma occurs with Continuous Queries. How can I be sure that a CQ already finished when a TICKScript is called at the same interval?
Currently I have to set the EVERY clause in my CQs very low to avoid the one timestep delay due to my delayed inputs. My server already suffers and it smells like an anti-pattern.
Another way would be to abuse Telegraf for postprocessing by calling external scripts which sleep for a while, process and return data.
Any recommendations?
