Local to remote DB replication

Welcome & Getting Started
1

hi
currently, my on-site Telegraf agent writes to a remote InfluxDB instance in AWS.
i want to add a local DB on my on-site Telegraf and have it sync the remote (centralized) influxDB
after recovering from site internet outage. just sending the delta.
could it be done and maybe you guys could direct me to an online guide.

thanks

1 Like
2

Hello @ahiyaz,
I recommend checking out this video. Can you please clarify what you mean by just sending the delta?
Thanks.

3

hi
thank you for replaying
i ment deltas in data
the data that was collected by the local DB while the connection to remote was down.

4

after watching the video i see that Telegraf have a builtin output resiliency mechanism to cope with downtime, output buffer that can be configure “metric_buffer_limit” but i assume it suite for really short downtime. do you know where i could find a better version of this video? become the sound is very low.
ot it is me getting old :wink:

5

I am using a rabbitMQ queue for this. My infrastructure is like this:

  • Telegraf agent on monitored server
    ->
  • Telegraf agent with influx input
    1. output to influxdb on-prem -> influxdb server on-prem
    2. output to rabbitmq on-prem
  • Telegraf agent with rabbitmq input (on-prem) and influxdb output (aws)

So I have 2 different flows:
on-prem:
Monitored server -> telegraf agent (used as proxy) -> influxdb server
to aws:
Monitored server -> telegraf agent (used as proxy) -> rabbitmq <- telegraf agent (used as proxy) -> influxdb server

Hope this helps.

6

hi
thanks for answering.
for now ive decided to use the local buffer and see for how long it could be as a backup.
it a very simple architecture before i go to more complex solution.
thanks again