# How to parse Snowflake log file?

**URL:** <https://community.influxdata.com/t/how-to-parse-snowflake-log-file/30881>\
**Category:** Telegraf\
**Tags:** influxdb, telegraf, grok, logging\
**Created:** [July 25, 2023, 3:58pm UTC](https://community.influxdata.com/t/how-to-parse-snowflake-log-file/30881 "2023-07-25T15:58:44Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pixelcode](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pixelcode/32/13964_2.png) [@pixelcode](https://community.influxdata.com/u/pixelcode)\
**Post date:** [July 25, 2023, 3:58pm UTC](https://community.influxdata.com/t/how-to-parse-snowflake-log-file/30881/1 "2023-07-25T15:58:44Z")

</div>

I’m trying to have Telegraf parse the log file of the local [Snowflake](https://snowflake.torproject.org) instance. It usually looks like this:

```auto
nohup: Input is ignored
2023/07/24 20:09:24 Proxy starting
2023/07/24 20:10:19 NAT type: restricted
2023/07/24 21:11:24 In the last 1h0m0s, there were 3 connections. Traffic Relayed ↓ 69528 KB, ↑ 2418 KB.
2023/07/24 22:11:24 In the last 1h0m0s, there were 13 connections. Traffic Relayed ↓ 622101 KB, ↑ 46242 KB.
2023/07/24 23:51:55 In the last 1h0m0s, there were 4 connections. Traffic Relayed ↓ 23649 KB, ↑ 5311 KB.
2023/07/25 00:51:55 In the last 1h0m0s, there were 7 connections. Traffic Relayed ↓ 123722 KB, ↑ 11554 KB.
2023/07/25 01:51:55 In the last 1h0m0s, there were 2 connections. Traffic Relayed ↓ 124297 KB, ↑ 5029 KB.

```

Here’s my current config:

```auto
[[inputs.tail]]
  # file(s) to tail:
  files = ["~/snowflake/proxy/snowflake.log"]
  from_beginning = false

  # name of the "Metric" (which I want to see in Grafana eventually)
  name_override = "snowflake_log"
 
  grok_patterns = ["%{CUSTOM_LOG}"]

  grok_custom_patterns = '''
SNOWFLAKEDATE %{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{TIME}
CUSTOM_LOG %{SNOWFLAKEDATE:date} In the last 1h0m0s, there were %{NUMBER:snowflake_connections:int} connections. Traffic Relayed ↓ %{NUMBER:snowflake_downstream:int} KB, ↑ %{NUMBER:snowflake_upstream:int} KB.
'''
  data_format = "grok"

```

How can I tell Telegraf to **ignore** lines that include any of the following keywords?

- `nohup: `
- `Proxy starting`
- `NAT type: `

Sometimes, however, there are also errors that include phrases like `stream not found`. How can I tell Telegraf to **use the value `0`** for `snowflake_connections`, `snowflake_downstream` and `snowflake_upstream` if the log line contains the phrase `stream not found`?

Thank you!

---

<div class="post-metadata">

**Author:** ![srebhan](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/srebhan/32/8572_2.png) [@srebhan](https://community.influxdata.com/u/srebhan)\
**Post date:** [July 26, 2023, 9:02am UTC](https://community.influxdata.com/t/how-to-parse-snowflake-log-file/30881/2 "2023-07-26T09:02:10Z")

</div>

@pixelcode I’m not sure I do understand your target setting correctly… With your grok pattern, the `nohup:`, ` Proxy starting` and `NAT type:` (or any line not matching your pattern) will be ignored as the grok parser will not find a pattern match. You can see this when running Telegraf with `--debug` as this will also show when a line was not matched.

If you want additional information (e.g. the errors you mention), you need to provide additional grok-patterns for those lines. If you want to fill-in missing fields you should use the [default processor](https://github.com/influxdata/telegraf/tree/master/plugins/processors/defaults) or if you need more sophisticated logic you should look into the [starlark processor](https://github.com/influxdata/telegraf/tree/master/plugins/processors/starlark) which can also generate additional metrics etc…

---

<div class="post-metadata">

**Author:** ![pixelcode](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pixelcode/32/13964_2.png) [@pixelcode](https://community.influxdata.com/u/pixelcode)\
**Post date:** [July 26, 2023, 6:27pm UTC](https://community.influxdata.com/t/how-to-parse-snowflake-log-file/30881/3 "2023-07-26T18:27:54Z")

</div>

The following processing seems to have solved it:

```auto
[[processors.defaults]]
  namepass = ["snowflake_metrics"]
  [processors.defaults.fields]
    snowflake_connections = 0
    snowflake_downstream = 0
    snowflake_upstream = 0

```

Also, I noticed that I Telegraf doesn’t recognise `~/snowflake/proxy/snowflake.log`, but only `/home/pi/snowflake/proxy/snowflake.log`.

---

<div class="post-metadata">

**Author:** ![srebhan](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/srebhan/32/8572_2.png) [@srebhan](https://community.influxdata.com/u/srebhan)\
**Post date:** [July 27, 2023, 8:46am UTC](https://community.influxdata.com/t/how-to-parse-snowflake-log-file/30881/4 "2023-07-27T08:46:59Z")

</div>

> Also, I noticed that I Telegraf doesn’t recognise `~/snowflake/proxy/snowflake.log` , but only `/home/pi/snowflake/proxy/snowflake.log` .

Well I guess you start Telegraf via systemd and thus the **user** that starts Telegraf is probably **telegraf**. So `~/snowflake/proxy/snowflake.log` expands to ``/home/telegraf/snowflake/proxy/snowflake.log` or whatever the home-dir of that user is set to… 🙂