Telegraf HTTP Output GZIP Encoding Not Working For Me

Telegraf
1

I’m having trouble with the content_encoding = “gzip” setting in the telegraf config. It does not appear to work for me. I’m seeing 0 bytes at the target http output webserver.

  • Default encoding works. (uncompressed)
  • GZIP encoding sent manually via curl command works as well. (see below)

I’m using same input for comprressed and decompressed cases from telegraf.

Is there anything I’m missing from my telegraf config file below? My sample python webserver use the following code to decompress and print the output.

            with gzip.GzipFile(fileobj=BytesIO(raw_data)) as gz:
                decompressed_data = gz.read().decode('utf-8')
            print("Decompressed Data:\n", decompressed_data)

Debug Webserver Output - Uncompressed output works fine.

(‘content_length:\n’, 286)
(‘content_encoding:\n’, ‘’)
(‘content_location:\n’, ‘’)
++++++ NOT COMPRESSED ----

(‘Received Data:\n’, u’{“metrics”:[{“fields”:{“@1_portList_tags”:“1:1”,“@2_portList_tags”:“1:7”,“@3_portList_tags”:“1:13”,“ipAddress”:“10.1.2.3”,“macAddr”:“00:73:04:05:06:07”,“vlanId_key”:10},“name”:“TestStats”,“tags”:{“reporterSerialnum”:“\“XYZ-5678\””,“serialnum”:“\“ABCD-1234\””},“timestamp”:1734804378}]}\n’)
127.0.0.1 - - [21/Dec/2024 18:06:18] “POST / HTTP/1.1” 200 -

Debug Webserver Output - Telegref GZIP output does NOT work. I see content_length of 0.

(‘content_length:\n’, 0)
(‘content_encoding:\n’, ‘gzip’)
(‘content_location:\n’, ‘’)
---------- gzip ---------

(‘Decompressed Data:\n’, u’')

Debug WebServer - Manually sending using curl WORKS with GZIP

echo “Hello, World!” | gzip | curl -X POST -H “Content-Encoding: gzip” --data-binary @- http://localhost:8080

Debug WebServer
(‘content_length:\n’, 34)
(‘content_encoding:\n’, ‘gzip’)
(‘content_location:\n’, ‘’)
---------- gzip ---------

(‘Decompressed Data:\n’, u’Hello, World!\n’)
127.0.0.1 - - [21/Dec/2024 18:14:51] “POST / HTTP/1.1” 200 -

alim@calabash devtest $ cat telegraf1.conf
[global_tags]

[agent]
collection_jitter = “0s”
debug = false
flush_interval = “1s”
flush_jitter = “1s”
hostname = “EXOS-5520”
interval = “4s”
logfile = “./telegraf.log”
metric_batch_size = 1000
metric_buffer_limit = 1000
omit_hostname = true
precision = “”
quiet = false
round_interval = true

[[inputs.file]]
files = [“./inputs/input”]
interval = “2s”
data_format = “influx”
metric_batch_size = 50

[[outputs.http]]
url = “http://localhost:8080/”
method = “POST”
tagexclude = [“path”]
data_format = “json”
metric_batch_size = 5
use_batch_format = true
content_encoding = “gzip”
[outputs.http.headers]
Content-Type = “application/json; charset=utf-8”

$ ./telegraf --version
Telegraf 1.33.0 (git: HEAD@67902005)

2

@srebhan Do you have any insight on this possible issue? I re-wrote that section of the http output plugin to compress reqBody first, before adding to reqBodyBuffer and that seems to work. I’m not sure why the original code is having issues.

func MyCompressWithGzip(data []byte) ([]byte, error) {
        var buf bytes.Buffer
        gz := gzip.NewWriter(&buf)
        _, err := gz.Write(data)
        if err != nil {
                return nil, err
        }
        if err := gz.Close(); err != nil {
                return nil, err
        }
        return buf.Bytes(), nil
}

func (h *HTTP) writeMetric(reqBody []byte) error {
	var err error

	if h.ContentEncoding == "gzip" {

		rc, err := MyCompressWithGzip(reqBody)
		if err != nil {
			return err
		}
		reqBody = rc
	}
	var reqBodyBuffer io.Reader = bytes.NewBuffer(reqBody)