Built a new telegraf output plugin but need to log out to telegraf log file

Hi all,

I built a new Telegraf output plugin but for debug reasons I need to be able to write data to the log file.

Example:
My plugin is supposed to reach out to google to perform a handshake and return a token to make an http request.
I’ve added the plugin to the source code, compiled it, and can see it available telegraf, but it looks like I’m not getting a token and want to log that the handshake is successful and/or what the token is.

2020-08-03T17:03:06Z E! [agent] Error writing to outputs.cloudrun: Post https://RESTURL: net/http: invalid header field name "Authorization ="

How would I add this functionality to my code?

Thanks!!

Can you add a few snippets from your code showing the logic you have currently?

Is it possible you’re logging the events you need with debug level, but telegraf is running with only info level or higher severity enabled?

Hi!
Thanks for the reply!

This plugin is being used to support sending metrics to a Wavefront proxy container in the GCP CloudRun cloud space. In order to send metrics into the proxy in this space you need to send a HTTP REST POST request with a token that has a maximum expiration time of one hour. In order to account for that I have a package that I added to Telegraf that is referenced in the plugin:

Some of that code is below

func GetToken(secret string, email string, url string) string {
	token, err := generateJWT(secret, email, url, 3000)

	if err != nil {
		println(err.Error())
	}
	//fmt.Printf("Token: %s\n", token)

	accessToken, err := getGoogleID(token)
	if err != nil {
		println(err.Error())
	}
	return accessToken
}

On the plugin side, I basically took most of the code from the outputs.http package but added a few extra fields to the http struct to account for the additional fields required in order to refresh the token in the above code:

type HTTP struct {
	URL             string            `toml:"url"`
	Timeout         internal.Duration `toml:"timeout"`
	Method          string            `toml:"method"`
	Username        string            `toml:"username"`
	Password        string            `toml:"password"`
	Headers         map[string]string `toml:"headers"`
	ClientID        string            `toml:"client_id"`
	ClientSecret    string            `toml:"client_secret"`
	TokenURL        string            `toml:"token_url"`
	Scopes          []string          `toml:"scopes"`
	ContentEncoding string            `toml:"content_encoding"`
	JSONSecret      string            `toml:"json_file_location"`
	GCPEmailAddress string            `toml:"cloudrun_email"`
	tls.ClientConfig

	client     *http.Client
	serializer serializers.Serializer
}

I added some constants (not shown here) to create some defaults for the headers and then added the return of the token code (the first code block) into a variable called token and set it as a header.

	token := gcp.GetToken(h.JSONSecret, h.GCPEmailAddress, h.URL)
	req.Header.Set("User-Agent", internal.ProductToken())
	req.Header.Set("Content-Type", defaultContentType)
	req.Header.Set("Accept", defaultAccept)
	req.Header.Set("Authorization = Bearer", token)
	if h.ContentEncoding == "gzip" {
		req.Header.Set("Content-Encoding", "gzip")

It looks like it all works:

2020-08-04T14:47:05Z D! [agent] Connecting outputs
2020-08-04T14:47:05Z D! [agent] Attempting connection to [outputs.cloudrun]
2020-08-04T14:47:05Z D! [agent] Successfully connected to outputs.cloudrun
2020-08-04T14:47:05Z D! [agent] Starting service inputs
2020-08-04T14:47:18Z D! [outputs.cloudrun] Buffer fullness: 0 / 10000 metrics

But then errors when it tries to send to the proxy because what looks like it not using a token:

2020-08-04T14:49:18Z D! [outputs.cloudrun] Buffer fullness: 27 / 10000 metrics
2020-08-04T14:49:18Z E! [agent] Error writing to outputs.cloudrun: Post https://wavefront-proxy-.run.app: net/http: invalid header field name "Authorization = Bearer"

I’m trying to figure out what might be going on by adding some logging.
Thoughts?

EDIT:
ssorka over at github gave me this information:
This is a great question, thanks!

Add this to your plugin struct, and a logger will be set automatically for it when it’s registered:

type MyPlugin {
  ...
  Log telegraf.Logger `toml:"-"`
}

in your tests, you’ll have to set this yourself to the test logger, eg:

myplugin := &MyPlugin{}
myplugin.Log = testutil.Logger{}

Using the logger is easy, it implements the telegraf.Logger interface
For example, using it in an input’s gather function would look like this:

 func (p *MyPlugin) Gather(acc telegraf.Accumulator) error {
  p.Log.Debug("Starting the gather function!")
  ...
}

Now that that’s said, we should get that into the docs somewhere.

Hi,
I was able to get it working by fixing the code:

	handshake := gcp.GetToken(h.JSONSecret, h.GCPEmailAddress, h.URL)
	token := fmt.Sprintf("Bearer %s", handshake)
	req.Header.Set("User-Agent", internal.ProductToken())
	req.Header.Set("Content-Type", defaultContentType)
	req.Header.Set("Accept", defaultAccept)
	req.Header.Set("Authorization", token)
	if h.ContentEncoding == "gzip" {
		req.Header.Set("Content-Encoding", "gzip")
	}
1 Like

This topic was automatically closed 60 minutes after the last reply. New replies are no longer allowed.