Sync data from/to an InfluxDB that is not always reachable

I’ve got an unexpected requirement for a project that is evaluating to use InfluxDB.

There are 2 separate environments (InfluxDB databases + whatever they build upon it) let’s call them A and B

  • A is a standard environment on a server inside the company
  • B instead moves around (the world) and is not guaranteed to have connectivity

Environment A and B must be able to work independently, but when possible (when environment B is available on the network, it might happen even after some weeks) the data between them must be synced (on both A and B).

I had a look around and looks like there are a bunch of github repos with mirroring or syncing solutions but I’d like to hear if someone out there had similar requirements.

So far I’ìve only used InfluxDB v1 but using v2 is not a problem.

Is there a built-in way to sync InfluxDB databases or should I use something “external”?

2 Likes

Hello!

Did u find any solution for your problem? I am facing with the exactly same situation and looking for answers.

If u have any please share.

@Giovanni_Luisotto Does data from A really need to sync to B? Or could it be one way? If it could be one-way, you could use the new replication feature that shipped with InfluxDB OSS 2.2: InfluxDB Edge Data Replication | InfluxDB OSS 2.2 Documentation

This replicates all writes to a bucket to a persistent cache in your OSS instance which regularly flushes to a remote InfluxDB instance. If it can’t replicate the writes, it will continue to buffer new writes in the cache until it can.

2 Likes

@d60 In the end I didn’t solve this but got a good enough workaround (for my use case).

I’m doing DB monitoring with Telegraf + InfluxDB, the initial idea was to have 2 synced, standalone environments but we didn’t need it that much (those data are not vital), since it was not supported out of the box the idea has been dropped.

To “solve” the problem we installed a Telegraf agent with a big buffer (to keep several hours of data) on the environment that goes around, so it can gather and keep the data even without connectivity and once internet is available it will send them out.

Scott’s proposal about InflxDB 2 replication might work, as what I care about is to have all the data in the main site DB, therefore a one-way replication is enough (from the “moving” remote to the main site), but as I explained above in my case is not strictly needed.

Thank you for your reply. @scott solution may work. I am going to try in the next few weeks.