Http_response input plugin shows multiple result_codes at a time instead of showing only one

Telegraf
, ,
1

Hello friends,
I am trying to use http_response telegraf input plugin to monitor multiple websites and systems. I am using result_code as field and based on that I am creating Grafana dashboard to show the status of the website.

The problem I am facing is, I am getting multiple result_code at a time in influxdb dashboard when I tested the plugin. But, what I am trying to achieve is just to get one result at a time, and it should be that way too I guess. Here is the screenshot:

image
image988×529 86.8 KB

This is the flux query I am working with

from(bucket: "system_stat")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "http_response")
  |> filter(fn: (r) => r["_field"] == "result_code")
  |> filter(fn: (r) => r["host"] == "fedora")
  |> filter(fn: (r) => r["method"] == "GET")
  |> filter(fn: (r) => r["server"] == "https://liyumfi.com/")
  |> aggregateWindow(every: v.windowPeriod, fn: last, createEmpty: false)
  |> yield(name: "last")
2

From Influxdb

image
image1115×331 19.5 KB

3

The http_response plugin is only capable of returning a single status code at a time, take a look at the code itself here. If you truly think this is an issue with Telegraf, then please provide the output from [[outputs.file]] showing two status codes showing up at the same time.

What is more likely occurring is that your flux query or your chart/graph is setup incorrectly and not grouping sufficiently. For example you are aggregating over your window period. Is it possible that multiple return codes occurred during that period?

4

If you truly think this is an issue with Telegraf, then please provide the output from [[outputs.file]] showing two status codes showing up at the same time

Where do I get that?

What is more likely occurring is that your flux query or your chart/graph is setup incorrectly and not grouping sufficiently. For example you are aggregating over your window period. Is it possible that multiple return codes occurred during that period?

I just used influxdb web to build the query, and I checked last to get the last value.

image
image1360×652 70.2 KB

5

Hello @jpowers
I have checked [[outputs.file]] and there is no duplicate value shown there,

# /tmp/metrics.out

http_response,host=fedora,method=GET,result=success,server=https://liyumfi.com/,status_code=200 response_time=0.68368988,http_response_code=200i,content_length=95354i,result_type="success",result_code=0i 1702558501000000000

What confuses me is it’s working fine on InfluxDB v2.7.1 and Grafana 10.0.1 on my Fedora 37 another PC. I used the same flux query.

6

Are you sure you don’t have two telegrafs running?

7

@jpowers
I have just installed Telegraf using dnf in Fedora, if I try to install Telegraf again it won’t let me after all. Here is telegraf.service daemon file I am using

[Unit]
Description=Telegraf
Documentation=https://github.com/influxdata/telegraf
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=-/etc/default/telegraf
User=telegraf
ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-director>
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartForceExitStatus=SIGPIPE
KillMode=mixed
TimeoutStopSec=5
LimitMEMLOCK=8M:8M
CapabilityBoundingSet=CAP_NET_RAW
AmbientCapabilities=CAP_NET_RAW

[Install]
WantedBy=multi-user.target
8

@jpowers @Anaisdg

Please help, I tried my best, removing telegraf and influxdb and reinstalling it but still I am getting multiple results at a time, and in Grafana it looks really bad since it shows two results at a time.

image
image1111×538 29.8 KB

9

Hello @jpowers
It’s been a while and I couldn’t figure out the problem so far. I have updated telegraf and influxdb multiple times and this specific issue still persists

10

@Amanuel_Elhanan,

It is very likely that you have two telegraf running on two different systems or even the same system. My suggestion is to change one of the tags from fedora to something else and see if you still get duplicates or if the tags remain the same.

I say it is very likely because when you used outptus.file you reported that only a single metric showed up. Meaning telegraf is doing the right thing and only producing a single metric.

1 Like
11

@jpowers I just checked the influxdb telegraf configuration that I have created, I edited some parts of it as you suggested, but I just saw two [[outputs.influxdb_v2]] I think is that could be the reason, I deleted that and it worked.

Sorry for taking your time, and thank you.

1 Like