I’m looking for an architecture feedback.
For a home project, I’m considering to store measurements into InfluxDB on a RaspberryPi. I’m considering a retention policy of a few weeks (2-3 months worst case). However I’d like to keep the full detailed history (call me data hoarder :)) on a somewhat more beefy server. Which, let’s say for sake of electricity consumption optimisation, is not up all the time. And worse than that, it is booted quite irregularly whenever I want it up.
What I’d like to reach is that have that server to replicate (synchronize) the data of the influxdb on RaspberryPi since the last time it connected (or at least what is still available due to the retention policy setup).
How would you do it?
My first idea is to keep the original data on the RaspberryPi, synchronize the original data whenever the server is brought up and handle an automatic injection in the full history influxdb on the server.
But that is not fun. Is there a clever use of Telegraf, Kapacitor, Influx subscriptions (or something else I missed) that would allow some kind of replication “whenever the slave is available”.
I’d say that the only constraint would be not to loose too many measurements between 2 synchronizations. For example buffering data in memory in the RaspberryPi might exceed the available RAM or I would loose everything since last sync if the power is cut.
Thanks for any pointer you’d have. There is no wrong answer here so just shoot!