We have a number of windows systems running telegraf that send data to a single remote influx instance every 10s. We are running into a problem in that sometimes our telegraf instances stop connecting via UDP to send data. Restarting the telegraf service seems to resolve the issue (until it randomly occurs again).
Logs show repeated:
2018-03-29T22:34:24Z E! Failed to connect to output influxdb, retrying in 15s, error was ‘Error creating UDP Client [udp://172.16.22.30:8089]: Error dialing UDP address [172.16.22.30:8089]: dial udp 172.16.22.30:8089: connect: A socket operation was attempted to an unreachable network.’
I’m looking for any hints, clues, things to check into, to try and resolve or at least narrow down the cause of this issue.