Capturing both body and query with telegraf's httplistenerv2 plugin

Telegraf
,
1

Hey, I’m testing the capture of headers sent with curl, however it seems like you can either choose to have data source as “body” or “query”. I need to capture headers and json body separately, and i need both. Headers would be for data validation in a processor script and the rest would go on to influx if the data is indeed valid.

example curl: curl -v -H “X-customvalidation: token” -d @file.json localhost:8080/telegraf

What i’ve tried so far:
i’ve specified the headers in telegraf conf (telegraf agent won’t run)
various other things, resulting in no outcome of headers.

2

I do not believe data_source option is what you want. Instead I would look at the http_header_tags option. You might be able to do:

http_header_tags = {"X-customvalidation" = "header"}

to capture the header as a tag and use it during processing. However, note that you would be adding that token as a tag, so be aware of the potential cardinality issues.

i’ve specified the headers in telegraf conf (telegraf agent won’t run)

It is not clear to me what header you set. We cannot help if you do not provide additional logs, config, etc.

Here is an example:

[agent]
   debug = true

[[inputs.http_listener_v2]]
   service_address = ":8080"
   http_header_tags = {"X-customvalidation" = "header"}

[[outputs.file]]

❯ curl -v -H "X-customvalidation: token" --data "metric,tag=test value=42" localhost:8080/telegraf
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /telegraf HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.1.2
> Accept: */*
> X-customvalidation: token
> Content-Length: 24
> Content-Type: application/x-www-form-urlencoded
> 
< HTTP/1.1 204 No Content
< Date: Mon, 05 Jun 2023 13:35:24 GMT
< 
* Connection #0 to host localhost left intact

2023-06-05T13:34:22Z I! [inputs.http_listener_v2] Listening on [::]:8080
2023-06-05T13:34:32Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2023-06-05T13:34:42Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2023-06-05T13:34:52Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2023-06-05T13:35:02Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2023-06-05T13:35:12Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2023-06-05T13:35:22Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
metric,header=token,host=ryzen,tag=test value=42 1685972124002643328
3

Hey, thanks for the reply!

I’ve received some more information about what i need to do with Telegraf. In addition to headers, i need to capture some query parameters as well as json body. Is the data source option flexible enough to be used for this use case? Is Telegraf capable of doing this? The query parameters are required for some sha256 operations.

4

The http listener plugin is set up as you noted in your original comment to build metrics out of the query or the body, but not both. Do you have an example of why you would want to use both when you probably already have control over the data sent to telegraf?