InfluxDB Relay not buffering failed writes

1

Sorry if this question was asked already.

I use InfluxDB relay to forward new metrics to 3 InfluxDB instances.
Since the servers hosting those InfluxDB instances are rebooted on regular basis (after each other, so not all at the same time) it is clear that some sort of inconsistency will occur.

This is why I thought: Hey, let’s activate buffering in InfluxDB Relay, so the relay will store failed write attempts and pushes them to the InfluxDB instance when it is up again after 2 or 3 minutes.

I thought I configured the relay the right way, but at least within my tests, no failed write attempts are buffered. Instead, all metrics being sent to InfluxDB relay are lost for the short time period when InfluxDB is down.

I realize that InfluxDB Relay wasn’t written for buffering metrics/write attempts; howerver, the way I see it, it should work and I guess I might be doing something wrong.

Can you please have a look at my configuration and tell me what I am missing?


[[http]]
name = "demo-influxrelay-http"
bind-addr = "127.0.0.1:9096"
output = [
    { name="local1", location = "http://127.0.0.1:8086/write", timeout="600s", buffer-size-mb = 100, max-batch-kb = 50 },
]

(I used the configuration from my demo system which has only one InfluxDB node.)

I am using the latest version from Git Hub.

2

According to me, you need to give more outputs in the output list.

output = [
{ name="local1", location = "http://127.0.0.1:8086/write", timeout="600s", buffer-size-mb = 100, max-batch-kb = 50 },
{ name="local2", location = "http://127.0.0.1:8076/write", timeout="600s", buffer-size-mb = 100, max-batch-kb = 50 },
]

You cannot use it using a Single node. You need multiple nodes to use it.

1 Like
3

Thanks @Luv for pointing out this feature works with at least two nodes enabled.
I just spawned another InfluxDB instance on localhost and included it into the relay config - and yes, it works like it should.

Thanks !

I don’t get why buffering only is active when having more than one node configured, but that is another question for another time…

4

Funny. When both of my test nodes are down, the Relay will not buffer failed writes. Is this again a wrong test of mine or expected?

5

check out my question