Telegraf_SNMP_Trap

Telegraf
44

Hi,
Today I have captured wireshark at destination server where telegraf plugin is installed and found the following output.
It clearly provide some additional details that telegraf is not receiving any snmp_trap data, but no issues either in server/network/firewall end.

  1. In the wireshark logs it clearly received snmp_trap data from the source device
  2. But whilst running the telegraf --test to get snmp_trap data nothing has been received.

Telegraf logs
[root@tsdcgbddwmdk01 telegraf.d]# telegraf --config telegraf_snmp_trap.conf --input-filter snmp_trap --output-filter influxdb_v2
2023-08-17T13:27:00Z I! Loading config file: telegraf_snmp_trap.conf
2023-08-17T13:27:00Z W! DeprecationWarning: Option “timeout” of plugin “inputs.snmp_trap” deprecated since version 1.20.0 and will be removed in 2.0.0: unused option
2023-08-17T13:27:00Z I! Starting Telegraf 1.26.0
2023-08-17T13:27:00Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-08-17T13:27:00Z I! Loaded inputs: snmp_trap
2023-08-17T13:27:00Z I! Loaded aggregators:
2023-08-17T13:27:00Z I! Loaded processors:
2023-08-17T13:27:00Z I! Loaded secretstores:
2023-08-17T13:27:00Z I! Loaded outputs: influxdb_v2
2023-08-17T13:27:00Z I! Tags enabled: host=tsdcgbddwmdk01
2023-08-17T13:27:00Z W! Deprecated inputs: 0 and 1 options
2023-08-17T13:27:00Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:“hvnpldwmdk01”, Flush Interval:10s
2023-08-17T13:27:00Z I! [inputs.snmp_trap] Listening on udp://:162

//wireshark logs captured at destination server
[root@tsdcgbddwmdk01 ~]# sudo tshark -i ens192 host 10.11.10.10
Running as user “root” and group “root”. This could be dangerous.
Capturing on ‘ens192’
1 0.000000000 10.11.10.10 → 10.12.12.10 SNMP 146 snmpV2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.4.3.0
2 0.000021142 10.12.12.10 → 10.11.10.10 ICMP 174 Destination unreachable (Host administratively prohibited)
3 5.322235114 10.12.12.10 → 10.11.10.10 SNMP 92 getBulkRequest 1.3.6.1.2.1.2.2.1.1
4 5.407301221 10.11.10.10 → 10.12.12.10 SNMP 146 snmpV2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.4.3.0
5 5.407326321 10.12.12.10 → 10.11.10.10 ICMP 174 Destination unreachable (Host administratively prohibited)
6 12.837581063 10.11.10.10 → 10.12.12.10 SNMP 146 snmpV2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.4.3.0
7 12.837600268 10.12.12.10 → 10.11.10.10 ICMP 174 Destination unreachable (Host administratively prohibited)
8 12.852588261 10.11.10.10 → 10.12.12.10 SNMP 146 snmpV2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.4.3.0
9 12.852597390 10.12.12.10 → 10.11.10.10 ICMP 174 Destination unreachable (Host administratively prohibited)
10 25.096900877 10.11.10.10 → 10.12.12.10 SNMP 321 snmpV2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.4.1.6889.2.73.9.1.1.1 1.3.6.1.4.1.6889.2.73.9.1.1.2 1.3.6.1.4.1.6889.2.73.9.1.1.3 1.3.6.1.4.1.6889.2.73.9.1.1.10 1.3.6.1.4.1.6889.2.73.9.1.1.11 1.3.6.1.4.1.6889.2.73.9.1.1.34
11 25.096924337 10.12.12.10 → 10.11.10.10 ICMP 349 Destination unreachable (Host administratively prohibited)
12 30.322280956 10.12.12.10 → 10.11.10.10 SNMP 92 getBulkRequest 1.3.6.1.2.1.2.2.1.1
13 30.407429869 10.11.10.10 → 10.12.12.10 SNMP 146 snmpV2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.4.3.0
14 30.407454353 10.12.12.10 → 10.11.10.10 ICMP 174 Destination unreachable (Host administratively prohibited)

45

Hi Hipska,

Once again I run the same command and here is the outcome. In the wireshark trace this server is receiving the trap OID data from the remote agent. At least it should send the output to influxDB which is running in same server, but nothing has been received.

2023-08-21T12:47:16Z I! Loading config file: telegraf_snmp_trap.conf
2023-08-21T12:47:16Z W! DeprecationWarning: Option “timeout” of plugin “inputs.snmp_trap” deprecated since version 1.20.0 and will be removed in 2.0.0: unused option
2023-08-21T12:47:16Z I! Starting Telegraf 1.26.0
2023-08-21T12:47:16Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-08-21T12:47:16Z I! Loaded inputs: snmp snmp_trap
2023-08-21T12:47:16Z I! Loaded aggregators:
2023-08-21T12:47:16Z I! Loaded processors:
2023-08-21T12:47:16Z I! Loaded secretstores:
2023-08-21T12:47:16Z W! Outputs are not used in testing mode!
2023-08-21T12:47:16Z I! Tags enabled: host=tsdcgbddwmdk01
2023-08-21T12:47:16Z W! Deprecated inputs: 0 and 1 options
2023-08-21T12:47:17Z I! [inputs.snmp_trap] Listening on udp://:162
2023-08-21T12:48:57Z E! [inputs.snmp] Error in plugin: agent udp://10.11.10.10:161: performing get on field uptime: request timeout (after 3 retries)
2023-08-21T12:49:57Z E! [telegraf] Error running agent: input plugins recorded 1 errors

46

Hi @Hipska and @jpowers, please let me know the next plan of action.
Thanks,
Manju

47

As requested multiple times before, run telegraf in test mode and then send traps. I didn’t see that yet. Either you run telegraf in regular mode, or you run it in test mode while another telegraf is still listening on port 162. Both are not helpful in any way.

48

Sorry I forgot to add the entire output including command in previous note. Here is the details.

If I kill the telegraf process and run this --test command, I am not getting error: udp :162: bind: address already in use. However even after kill the process of Telegraf associated to 162 port no output.

[root@tsdcgbddwmdk01 telegraf.d]$ telegraf --config telegraf_snmp_trap.conf --test-wait 120
2023-08-21T12:47:16Z I! Loading config file: telegraf_snmp_trap.conf
2023-08-21T12:47:16Z W! DeprecationWarning: Option “timeout” of plugin “inputs.snmp_trap” deprecated since version 1.20.0 and will be removed in 2.0.0: unused option
2023-08-21T12:47:16Z I! Starting Telegraf 1.26.0
2023-08-21T12:47:16Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-08-21T12:47:16Z I! Loaded inputs: snmp snmp_trap
2023-08-21T12:47:16Z I! Loaded aggregators:
2023-08-21T12:47:16Z I! Loaded processors:
2023-08-21T12:47:16Z I! Loaded secretstores:
2023-08-21T12:47:16Z W! Outputs are not used in testing mode!
2023-08-21T12:47:16Z I! Tags enabled: host=tsdcgbddwmdk01
2023-08-21T12:47:16Z W! Deprecated inputs: 0 and 1 options
2023-08-21T12:47:17Z I! [inputs.snmp_trap] Listening on udp://:162
2023-08-21T12:48:57Z E! [inputs.snmp] Error in plugin: agent udp://10.11.10.10:161: performing get on field uptime: request timeout (after 3 retries)
2023-08-21T12:49:57Z E! [telegraf] Error running agent: input plugins recorded 1 errors

49

@Hipska, I am able to fix the UDP issue, and now we are not getting udp port in use error.

However I am still not getting any test traps from the remote agent, but no issues for the port 162 as we are seeing in the TCP dump, whilst executing the below command no output received for input snmp_trap.

[root@tsdcgbddwmdk01 telegraf.d]# telegraf --config telegraf_snmp_trap.conf --test-wait 60
2023-08-31T10:19:52Z I! Loading config file: telegraf_snmp_trap.conf
2023-08-31T10:19:52Z W! DeprecationWarning: Option “timeout” of plugin “inputs.snmp_trap” deprecated since version 1.20.0 and will be removed in 2.0.0: unused option
2023-08-31T10:19:52Z I! Starting Telegraf 1.26.0
2023-08-31T10:19:52Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-08-31T10:19:52Z I! Loaded inputs: snmp snmp_trap
2023-08-31T10:19:52Z I! Loaded aggregators:
2023-08-31T10:19:52Z I! Loaded processors:
2023-08-31T10:19:52Z I! Loaded secretstores:
2023-08-31T10:19:52Z W! Outputs are not used in testing mode!
2023-08-31T10:19:52Z I! Tags enabled: host=tsdcgbddwmdk01
2023-08-31T10:19:52Z W! Deprecated inputs: 0 and 1 options
2023-08-31T10:19:52Z I! [inputs.snmp_trap] Listening on udp://:162
[root@tsdcgbddwmdk01 telegraf.d]#