Net_response Issues and inconsistencies

Telegraf
1

I am attempting to use the net_response input plugin to monitor a Squid proxy server but I do not seem to be getting the expected results. I have even reviewed the golang code and duplicated the section that seems to not work in Go Play and I get the expected results for the regexp portion but not from the plugin. Just wondering if the plugin code has been included in the latest build.

What I am trying to do is connect to one of my Squid proxies at 10.10.2.5:8080 and send a “HEAD http://www.google.com” string and get back a “HTTP/1.0 200 OK” string. See below …

[[inputs.net_response]]
  protocol = "tcp"
  address = "10.10.2.5:8080"
  timeout = "1s"
  send = "HEAD http://www.google.com"
  expect = "HTTP/1.0 200 OK"
  tagexclude = ["host","role","rack","rack-name","dc","network"]

If I run a test I get a string_found=false and no result_type at all. See below …

# telegraf --config squid.conf --test
* Plugin: inputs.net_response, Collection 1
> net_response,server=10.10.2.5,port=8080,protocol=tcp string_found=false,response_time=1.000848022 1502923238000000000

The actual string returned when I do this through telnet is this …

# telnet 10.10.2.5 8080
Trying 10.10.2.5...
Connected to 10.10.2.5.
Escape character is '^]'.
HEAD http://www.google.com
HTTP/1.0 200 OK
Date: Wed, 16 Aug 2017 20:06:21 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: NID=110=WwV22ciCaAo2Lr8gdnh250iFQN-D9A_R-DzbJ5BpL-_TtX3_Pxc-lX5G8Vz4SmwMMLZWatSLXEA0JwLjGB-BxHzxc2qgul2HTUFzfr2Al8QL8Yq8LVUykQY9LqOdCjgf; expires=Thu, 15-Feb-2018 20:06:21 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
X-Cache: MISS from ninja
X-Cache-Lookup: MISS from ninja:8080
Via: 1.0 ninja (squid)
Connection: close
Connection closed by foreign host.

Looking closely at the code I should get a match in regexp if my expect string is contained within the response. The fact that the code tests Ok in Go Play (regexp section only) and that I never see “result_type” in the output leads me to believe that the code I see at GitHub is not really the code that is running.

Running Telegraf v1.3.5 (git: release-1.3 7192e68b2423997177692834f53cdf171aee1a88).

Because of this I have had to write my own code and use the Exec input to do things like this. Have others had better success with this plugin?

Not a GoLang guy but including my Go Play code for posterity …

package main

import (
	"fmt"
	"regexp"
)

func main() {
	var data = `HTTP/1.0 200 OK
Date: Wed, 16 Aug 2017 20:06:21 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: NID=110=WwV22ciCaAo2Lr8gdnh250iFQN-D9A_R-DzbJ5BpL-_TtX3_Pxc-lX5G8Vz4SmwMMLZWatSLXEA0JwLjGB-BxHzxc2qgul2HTUFzfr2Al8QL8Yq8LVUykQY9LqOdCjgf; expires=Thu, 15-Feb-2018 20:06:21 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
X-Cache: MISS from ninja
X-Cache-Lookup: MISS from ninja:8080
Via: 1.0 ninja (squid)`

	var Expect = "HTTP/1.0 200 OK"
	re := regexp.MustCompile(`.*` + Expect + `.*`)

	find := re.FindString(string(data))
	if find != "" {
		fmt.Println("result_type = success")
		fmt.Println("string_found = true")
	} else {
		fmt.Println("result_type = string_mismatch")
		fmt.Println("string_found = false")
	}
}

If I run this I get …

result_type = success
string_found = true

Code I am viewing is at telegraf/net_response.go at master · influxdata/telegraf · GitHub

2

The result_type field is new and will be in the upcoming 1.4 release.

3

Thanks for that @daniel. So any idea on the string_found issue. Note that I did not replicate the actual Readline from response code so my big ole multi-line string may not be what’s returned from the plugin but I found no easy way to see those results during testing. I did go back to the 1.3 branch and see that you are right! Not a GitHub guy either but learning.

4

It looks like only the first line of the response is checked for the expect string based on this. I’m not sure why this behavior was chosen, perhaps to avoid hanging if the server doesn’t close the connection?

5

My string should have been in the 1st line so I think it must have thrown an error before checking the string…

// Handle error
if err != nil {
	fields["string_found"] = false

I have noticed that the plugin seems to wait for the entire timeout before returning. For example, I have tested with a 5 second timeout and that delays that collection and the response time would be 5 seconds and some change. This really should have taken only milliseconds since it’s all internal IP addresses on the same subnet.

6

For now I have decided to do this with python and inputs.exec since I can get the correct response time and read the entire response (actually read_until) looking for my “expect” string. I just generate an output similar to what I would get with inputs.net_response formatted for influxdb. I have used this process for other services I need to remotely monitor (like smtp, dns, etc).

I’ll keep watching input.net_response as it matures and may eventually go native when it’s ready. Thanks for the clarifications and help.