If snmp custom agent is restarted have to restart telegraf too

We have our custom SNMP agent. And we are using telegraf to get data from it. There is this strange issue whenever the agent is a restart we have to restart the telegraf too. Otherwise, it won’t fetch the data and keep giving a timeout issue.
2021-07-14T17:42:20Z E! [inputs.snmp] Error in plugin: agent udp:// : gathering table errorEvent: performing bulk walk for field errorEvent-program: Request timeout (after 3 retries) If we use snmptable or walk for that particular agent it works and shows data, its only telegraf which doesn’t work until it is restarted.
Did anyone experience the same issue? and what could be the possible fix for this? (edited)