Can't load http_response in Telegraf to Influx

[Marko]

Hi everybody

Sorry, my first contribution here is a problem… I copy pasted a block of data I put for another web site, and somehow, it is not fhlushing the data to my Influx database (the other script beside, which has a couple of different URLs works fine).

telegraf --debug --config test2.conf
2020-05-10T16:06:33Z I! Starting Telegraf 1.14.2
2020-05-10T16:06:33Z I! Loaded inputs: http_response http_response http_response
2020-05-10T16:06:33Z I! Loaded aggregators:
2020-05-10T16:06:33Z I! Loaded processors:
2020-05-10T16:06:33Z I! Loaded outputs: influxdb
2020-05-10T16:06:33Z I! Tags enabled: host=M1PMONAPP001.mydomain.local
2020-05-10T16:06:33Z I! [agent] Config: Interval:30s, Quiet:false, Hostname:“M1PMONAPP001.mydomain.local”, Flush Interval:30s
2020-05-10T16:06:33Z D! [agent] Initializing plugins
2020-05-10T16:06:33Z D! [agent] Connecting outputs
2020-05-10T16:06:33Z D! [agent] Attempting connection to [outputs.influxdb]
2020-05-10T16:06:33Z D! [agent] Successfully connected to outputs.influxdb
2020-05-10T16:06:33Z D! [agent] Starting service inputs
2020-05-10T16:07:34Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:08:01Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:08:34Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:09:00Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:09:31Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:10:00Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:10:33Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:11:00Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:11:31Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:12:03Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:12:32Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:13:02Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:13:31Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:14:01Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
^C2020-05-10T16:14:19Z D! [agent] Stopping service inputs
2020-05-10T16:14:19Z D! [agent] Input channel closed
2020-05-10T16:14:19Z I! [agent] Hang on, flushing any cached metrics before shutdown
2020-05-10T16:14:19Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-05-10T16:14:19Z D! [agent] Closing outputs
2020-05-10T16:14:19Z D! [agent] Stopped Successfully

Here is the script test3.conf

[agent]
interval = “30s”
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = “5s”
flush_interval = “30s”
flush_jitter = “5s”
precision = “”
hostname = “”
omit_hostname = false

[[outputs.influxdb]]
urls = [“http://127.0.0.1:8086”]
database = “testdb”
skip_database_creation = true
[outputs.influxdb.tagpass]
collector = “test_web”

[[inputs.http_response]]
name_suffix = “test_login”
urls = [“https://test.mydomain.local”]
response_timeout = “10s”
method = “GET”
follow_redirects = false
interval = “300s”
response_string_match = “test page”
[inputs.http_response.tags]
collector = “test_web”

When running it, even in debug, I don’t have any info on what is happening :

And when running a test run of the script, I do see it is pulling the data correctly… So somehow it is not reaching influx :

telegraf --debug --config test3.conf --test
2020-05-10T16:21:16Z I! Starting Telegraf 1.14.2
2020-05-10T16:21:16Z D! [agent] Initializing plugins
http_responsesafetys_login,collector=test_web,host=M1PMONAPP001.mydomain.local,method=GET,result=success,server=https://test.mydomain.local,status_code=200 content_length=3833i,http_response_code=200i,response_string_match=1i,response_time=0.033108951,result_code=0i,result_type=“success” 1589127677000000000

In influx, I do have the safetys database created, however, it is not populated :

InfluxDB shell version: 1.8.0
use testdb
Using database testdb
show measurements

Any guidance on where to look / troubleshoot would be more than welcome !
Thank you very much

[rawkode]

Hi,

Can you add a file output and rerun Telegraf, please? This can help us rule out some class of problems:

[[outputs.file]]
  urls = ["stdout"]

[Marko]

Thanks for the feedback, really appreciated.
I added an output to stdout which helps a lot in investigating. After fiddling a lot with the config file, I ended up finding that my tagpass is not working for some reason… When I comment the line out, everything outputs correctly, meaning it was filtering all my data out

2020-05-12T13:16:35Z I! Starting Telegraf 1.14.2
2020-05-12T13:16:35Z I! Loaded inputs: http_response http_response http_response
2020-05-12T13:16:35Z I! Loaded aggregators:
2020-05-12T13:16:35Z I! Loaded processors:
2020-05-12T13:16:35Z I! Loaded outputs: influxdb
2020-05-12T13:16:35Z I! Tags enabled: host=M1PMONAPP001
2020-05-12T13:16:35Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:“M1PMONAPP001.m4ii.local”, Flush Interval:10s
2020-05-12T13:16:35Z D! [agent] Initializing plugins
2020-05-12T13:16:35Z D! [agent] Connecting outputs
2020-05-12T13:16:35Z D! [agent] Attempting connection to [outputs.influxdb]
2020-05-12T13:16:35Z D! [agent] Successfully connected to outputs.influxdb
2020-05-12T13:16:35Z D! [agent] Starting service inputs
> 2020-05-12T13:16:50Z D! [outputs.influxdb] Wrote batch of 15 metrics in 5.418454ms
2020-05-12T13:16:50Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
^C2020-05-12T13:16:58Z D! [agent] Stopping service inputs
2020-05-12T13:16:58Z D! [agent] Input channel closed

Am I getting the tagpass setting wrongly?

Thanks again for your help & time !

[daniel]

Sorry about this, it is a bug of sorts in Telegraf. With tagpass you need an array of strings and unfortunately this does not get reported as an error in our TOML parsing:

  [outputs.influxdb.tagpass]
-   collector = "test_web"
+   collector = ["test_web"]