Basicstats counter not working

Telegraf
1

Hello!

I am trying to count the number of failed login attempts but am failing at getting the aggregator.basicstats plugin to work, i.e. it doesnt output anything.

My telegraf.conf looks like this

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "20s"
  debug = true

[[outputs.file]]
  files = ["stdout"]

[[aggregators.basicstats]]
  period = "10s"
  drop_original = false
  stats = ["count"]

[[inputs.tail]]
  files = ["/test.log"]
  from_beginning = false
  watch_method = "poll"
  data_format = "grok"
  grok_patterns = ['''Failed password.*%{IP:ip}''']

To test the setup I am pasting the following line into the file /test.log and saving it

Feb 2 17:04:10 ubuntu sshd[3234386]: Failed password for invalid user ffwe from 127.0.0.1 port 41790 ssh2

Here is the output that I am getting

root@ubuntu:/# telegraf --config telegraf.conf
2023-02-03T12:44:08Z I! Starting Telegraf 1.25.0
2023-02-03T12:44:08Z I! Available plugins: 228 inputs, 9 aggregators, 26 processors, 21 parsers, 57 outputs, 2 secret-stores
2023-02-03T12:44:08Z I! Loaded inputs: tail
2023-02-03T12:44:08Z I! Loaded aggregators: basicstats
2023-02-03T12:44:08Z I! Loaded processors: 
2023-02-03T12:44:08Z I! Loaded secretstores: 
2023-02-03T12:44:08Z I! Loaded outputs: file
2023-02-03T12:44:08Z I! Tags enabled: host=ubuntu
2023-02-03T12:44:08Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"felipe.physik.uni-ulm.de", Flush Interval:20s
2023-02-03T12:44:08Z D! [agent] Initializing plugins
2023-02-03T12:44:08Z D! [agent] Connecting outputs
2023-02-03T12:44:08Z D! [agent] Attempting connection to [outputs.file]
2023-02-03T12:44:08Z D! [agent] Successfully connected to outputs.file
2023-02-03T12:44:08Z D! [agent] Starting service inputs
2023-02-03T12:44:08Z D! [inputs.tail] Tail added for "/test.log"
2023-02-03T12:44:08Z D! [aggregators.basicstats] Updated aggregation range [2023-02-03 13:44:00 +0100 CET, 2023-02-03 13:44:10 +0100 CET]
2023-02-03T12:44:10Z D! [aggregators.basicstats] Updated aggregation range [2023-02-03 13:44:10 +0100 CET, 2023-02-03 13:44:20 +0100 CET]
2023-02-03T12:44:20Z D! [aggregators.basicstats] Updated aggregation range [2023-02-03 13:44:20 +0100 CET, 2023-02-03 13:44:30 +0100 CET]
tail,host=ubuntu,path=/test.log ip="7.0.0.1" 1675428257907367789
2023-02-03T12:44:28Z D! [outputs.file] Wrote batch of 1 metrics in 47.569µs
2023-02-03T12:44:28Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics

I tried with another input that has integers rather than strings as measurement results and in this case the basicstats plugin worked as expected. Is the basicstats plugin only working for numerical fields? If so how can I adapt my setup to achieve my goal? How do I count the lines containing the “Failed password” string?

Thanks!

PS: I know that the grok pattern that I am using is not ideal. I am actually not interested in saving the IP, I really just want to count the lines. Happy to hear any suggestions for a better pattern. I am very new to this.

2

This is exactly the issue. The basicstats aggregator operates on fields of numeric values. Your example tail metric has no numeric values. There is no “max” or “min” ip address.

How do I count the lines containing the “Failed password” string?

If it were me I would continue to parse the log line and create a metrics, similar to what you are doing now, and in your reporting, graphing, metric collecting tool, there do the aggregation of the the sums in your reporting.

3

Thanks! So I have to make sure to feed numerical data to the basicstats plugin. I managed to do that by changing the grok pattern to
grok_patterns = ['''Failed password.*%{NUMBER:num:int}''']
The converts the content of the num field to an integer and now everything works.

Unfortunately I couldnt find a way count the lines in Grafana, which is why I turned to the basicstats plugin.