Telegraf HTTP Output GZIP Encoding Not Working For Me

Telegraf
1

I’m having trouble with the content_encoding = “gzip” setting in the telegraf config. It does not appear to work for me. I’m seeing 0 bytes at the target http output webserver.

  • Default encoding works. (uncompressed)
  • GZIP encoding sent manually via curl command works as well. (see below)

I’m using same input for comprressed and decompressed cases from telegraf.

Is there anything I’m missing from my telegraf config file below? My sample python webserver use the following code to decompress and print the output.

            with gzip.GzipFile(fileobj=BytesIO(raw_data)) as gz:
                decompressed_data = gz.read().decode('utf-8')
            print("Decompressed Data:\n", decompressed_data)

Debug Webserver Output - Uncompressed output works fine.

(‘content_length:\n’, 286)
(‘content_encoding:\n’, ‘’)
(‘content_location:\n’, ‘’)
++++++ NOT COMPRESSED ----

(‘Received Data:\n’, u’{“metrics”:[{“fields”:{“@1_portList_tags”:“1:1”,“@2_portList_tags”:“1:7”,“@3_portList_tags”:“1:13”,“ipAddress”:“10.1.2.3”,“macAddr”:“00:73:04:05:06:07”,“vlanId_key”:10},“name”:“TestStats”,“tags”:{“reporterSerialnum”:“\“XYZ-5678\””,“serialnum”:“\“ABCD-1234\””},“timestamp”:1734804378}]}\n’)
127.0.0.1 - - [21/Dec/2024 18:06:18] “POST / HTTP/1.1” 200 -

Debug Webserver Output - Telegref GZIP output does NOT work. I see content_length of 0.

(‘content_length:\n’, 0)
(‘content_encoding:\n’, ‘gzip’)
(‘content_location:\n’, ‘’)
---------- gzip ---------

(‘Decompressed Data:\n’, u’')

Debug WebServer - Manually sending using curl WORKS with GZIP

echo “Hello, World!” | gzip | curl -X POST -H “Content-Encoding: gzip” --data-binary @- http://localhost:8080

Debug WebServer
(‘content_length:\n’, 34)
(‘content_encoding:\n’, ‘gzip’)
(‘content_location:\n’, ‘’)
---------- gzip ---------

(‘Decompressed Data:\n’, u’Hello, World!\n’)
127.0.0.1 - - [21/Dec/2024 18:14:51] “POST / HTTP/1.1” 200 -

alim@calabash devtest $ cat telegraf1.conf
[global_tags]

[agent]
collection_jitter = “0s”
debug = false
flush_interval = “1s”
flush_jitter = “1s”
hostname = “EXOS-5520”
interval = “4s”
logfile = “./telegraf.log”
metric_batch_size = 1000
metric_buffer_limit = 1000
omit_hostname = true
precision = “”
quiet = false
round_interval = true

[[inputs.file]]
files = [“./inputs/input”]
interval = “2s”
data_format = “influx”
metric_batch_size = 50

[[outputs.http]]
url = “http://localhost:8080/”
method = “POST”
tagexclude = [“path”]
data_format = “json”
metric_batch_size = 5
use_batch_format = true
content_encoding = “gzip”
[outputs.http.headers]
Content-Type = “application/json; charset=utf-8”

$ ./telegraf --version
Telegraf 1.33.0 (git: HEAD@67902005)

3

Disregard this issue.

This was a problem with my test webserver which was looking at content-length. Telegraf will send GZIP’d data as chunked, so content-length will always be 0. If content-length is 0, I needed to check if payload data is chunked and read payload accordingly. In any case, this is not a telegraf issue.

4

@arnel_lim
thanks for sharing your question and solution! Im glad you figured it out