Telegraf inputs.snmp collecting troubles

Telegraf
,
1

I’m trying to configure the snmp input for telegraf to pull metrics from my F5 Big IP load balancers. I am able to see all the OID utilizing snmpwalk, but for whatever reason i am not having the same success when configuring the snmp input.

# snmpwalk -v2c -c <community string> <big ip - ip> RFC1213-MIB::sysName.0
SNMPv2-MIB::sysName.0 = STRING: <big ip hostname>

example snmpwalk command/output - i ran each and every oid i configured to verify it was returning a value without error

initial config

[[inputs.snmp]]
  agents = ["<big ip - ip>"]
  version = 2
  community = "<community string>"
  interval = "60s"
  timeout = "10s"
  retries = 3
  name = "snmp"

[[inputs.snmp.field]]
  name = "hostname"
  oid = "RFC1213-MIB::sysName.0"
  is_tag = true

  [[inputs.snmp.field]]
    name = "F5_httpRequests"
    oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.56"

  [[inputs.snmp.field]]
    name = "F5_client_connections"
    oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.8"

  [[inputs.snmp.field]]
    name = "F5_client_bytes_in"
    oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.60"

initial config output

# telegraf --test | grep snmp
2018/06/02 00:54:56 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.snmp, Collection 1

i modified the config to and i would get an output of some kind, but it does not contain all of the data in the config and the format seems off. it was the best i was able to achieve though…

adjusted config

[[inputs.snmp]]
  agents = ["<big ip - ip>"]
  version = 2
  community = "<community string>"
  interval = "60s"
  timeout = "10s"
  retries = 3
  name = "snmp"

 [[inputs.snmp.field]]
  name = "hostname"
  oid = "RFC1213-MIB::sysName.0"

 [[inputs.snmp.field]]
  name = "F5_httpRequests"
  oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.56"

 [[inputs.snmp.field]]
  name = "F5_client_connections"
  oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.8"

 [[inputs.snmp.field]]
  name = "F5_client_bytes_in"
  oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.60"

adjusted config output

# telegraf --test | grep snmp
2018/06/02 00:58:15 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.snmp, Collection 1
> snmp,agent_host=172.22.15.71,host=ops-worker-001.prod.dc3 hostname="lb01.prod.dc3" 1527901098000000000

I see my first mistake in the config not being nested under the main [[inputs.snmp]]… what i don’t get is why can’t i tag anything or get all of the outputs? i tried making tables and things of that nature… i even used the example config (verifying all the oids returned values) and that didn’t even work.

example config

[[inputs.snmp]]
  agents = [ "host.example.com" ]
  version = 2
  community = "public"

  [[inputs.snmp.field]]
    name = "hostname"
    oid = "RFC1213-MIB::sysName.0"
    is_tag = true

  [[inputs.snmp.field]]
    name = "uptime"
    oid = "F5-BIGIP-SYSTEM-MIB::sysSystem.6"

  # IF-MIB::ifTable contains counters on input and output traffic as well as errors and discards.
  [[inputs.snmp.table]]
    name = "interface"
    inherit_tags = [ "hostname" ]
    oid = "IF-MIB::ifTable"

    # Interface tag - used to identify interface in metrics database
    [[inputs.snmp.table.field]]
      name = "ifDescr"
      oid = "IF-MIB::ifDescr"
      is_tag = true

  # IF-MIB::ifXTable contains newer High Capacity (HC) counters that do not overflow as fast for a few of the ifTable counters
  [[inputs.snmp.table]]
    name = "interface"
    inherit_tags = [ "hostname" ]
    oid = "IF-MIB::ifXTable"

    # Interface tag - used to identify interface in metrics database
    [[inputs.snmp.table.field]]
      name = "ifDescr"
      oid = "IF-MIB::ifDescr"
      is_tag = true

  # EtherLike-MIB::dot3StatsTable contains detailed ethernet-level information about what kind of errors have been logged on an interface (such as FCS error, frame too long, etc)
  [[inputs.snmp.table]]
    name = "interface"
    inherit_tags = [ "hostname" ]
    oid = "EtherLike-MIB::dot3StatsTable"

    # Interface tag - used to identify interface in metrics database
    [[inputs.snmp.table.field]]
      name = "ifDescr"
      oid = "IF-MIB::ifDescr"
      is_tag = true

I’m quite certain i am missing something small and silly, but i need more eyes on this to help me find that.

full telegraf config

[global_tags]

# Configuration for telegraf agent
[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  debug = true
  quiet = false
  logfile = ""
  hostname = ""
  omit_hostname = false

# Configuration for sending metrics to InfluxDB
[[outputs.influxdb]]
   urls = ["http://influxdb-server:8086"]
   database = "telegraf"

# Read metrics about cpu usage
[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false


# Read metrics about disk usage by mount point
[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs"]


# Read metrics about disk IO by device
[[inputs.diskio]]
  # no configuration

# Get kernel statistics from /proc/stat
[[inputs.kernel]]
  # no configuration


# Read metrics about memory usage
[[inputs.mem]]
  # no configuration


# Get the number of processes and group them by status
[[inputs.processes]]
  # no configuration


# Read metrics about swap memory usage
[[inputs.swap]]
  # no configuration

# Read metrics about system load & uptime
[[inputs.system]]
  # no configuration

 # Collect bond interface status, slaves statuses and failures count
 [[inputs.bond]]

# Read metrics about docker containers
 [[inputs.docker]]
   endpoint = "unix:///var/run/docker.sock"
   gather_services = false
   container_names = []
   container_name_include = []
   container_name_exclude = []
   timeout = "5s"
   perdevice = true
   total = false
   docker_label_include = []
   docker_label_exclude = []

# Read stats from one or more Elasticsearch servers or clusters
 [[inputs.elasticsearch]]
   servers = ["http://es-cluster-nodes:9200"]
   http_timeout = "5s"
   local = false
   cluster_health = false
   cluster_stats = false

# Monitor disks' temperatures using hddtemp
 [[inputs.hddtemp]]

# Read InfluxDB-formatted JSON metrics from one or more HTTP endpoints
 [[inputs.influxdb]]
   urls = [
     "http://influxdb-server:8086/debug/vars"
   ]
   timeout = "5s"

# Monitor sensors, requires lm-sensors package
 [[inputs.sensors]]


 # Read metrics from storage devices supporting S.M.A.R.T.
 [[inputs.smart]]
    use_sudo = true

[[inputs.snmp]]
  agents = ["<big ip - ip>"]
  version = 2
  community = "<community string>"
  interval = "60s"
  timeout = "10s"
  retries = 3
  name = "snmp"

 [[inputs.snmp.field]]
  name = "hostname"
  oid = "RFC1213-MIB::sysName.0"

 [[inputs.snmp.field]]
  name = "F5_httpRequests"
  oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.56"

 [[inputs.snmp.field]]
  name = "F5_client_connections"
  oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.8"

 [[inputs.snmp.field]]
  name = "F5_client_bytes_in"
  oid = "F5-BIGIP-SYSTEM-MIB::sysGlobalStat.60"

if there is something else impacting the snmp config i don’t know… i removed all of that stuff except the base config and got the same results.

2

The [[inputs.snmp.field]] table works using a SNMPGET, while the [[inputs.snmp.table]] uses a WALK, so you may want to create a [[inputs.snmp.table]] to capture your OIDs. Make sure you capture at least one item not as a tag as this is required by line protocol.

If you just want a few items inside the table only specify the OID for the field like:

  [[inputs.snmp.table]]
    name = "interface"
    inherit_tags = [ "hostname" ]

    [[inputs.snmp.table.field]]
      name = "ifDescr"
      oid = "IF-MIB::ifDescr"
3

thanks for the info! i did not realize that and the documentation wasn’t really clear on that. i guess that’s more of a general snmp knowledge thing. i appreciate it.

any recommendations on general snmp info that might help with future config tasks?

4

I struggle with SNMP as well, so I don’t have much in the way of advice. Were you able to get your configuration working as desired?

5

yes… i appear to be able to pull in metrics… i have only been able to get a few in right now. so we will see if i continue to have success…

so if i’m putting all my oid in a table.field under the table i should be fine?

6

actually the one thing i’m having an issue with is that some of my oid require quotes and those metrics don’t appear to be loading.

[[inputs.snmp.table.field]]
       name = "F5_vs-80_client_bytes_in"
       oid = "F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientBytesIn.\"/VS-80\""

this is an example of the metric i’m trying to grab.

7

i thought i made some progress…

     [[inputs.snmp.table.field]]
       name = "vs_client_connections"
       oid = 'F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns.\"/Common/vs_client\"'

     [[inputs.snmp.table.field]]
       name = "vs_client_bytes_in"
       oid = 'F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientBytesIn.\"/Common/vs_client\"'

 Error in plugin [inputs.snmp]: initializing table f5-snmp: initializing field vs_client_connections: translating: exit status 2: F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns.\"/Common/vs_client\": Unknown Object Identifier (Index out of range: \"/Common/vs_client\" (ltmVirtualServStatName))

removing the escapes resolves the error, but i don’t get any metrics.

tried adding it as a table and as just a field and those didn’t work either.

8

The version without the escapes (that shows no error or metrics) is the right one. I don’t have this vendor MIB, but I’ve been playing around with a few things. I think my earlier advice to only set the field OID might be a problem. Could you run this:

snmptranslate -Td -Ob 'F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientBytesIn."/Common/vs_client"'