Cannot parse multiline exec output using Grok input plugin

Telegraf
1

Hello,

I’m trying to create metrics from the exec output below, where I need to have the client Id and the Connected status.

The JORAMMQ_MQTT_HOME environment variable is not defined.
Use JORAMMQ_MQTT_HOME=/usr/share/jorammq-mqtt
JAVA_HOME is not defined use default
Unable to configure Monolog from file: null
Command: dumpClient

____________________________
Welcome to Apache Felix Gogo

g! dumpClient
MQTTClientContext ClientId=vgerndvud2587.xxx.com CONNECTED
 # Clean: false, Queued QoS0: false, TTL: 0
 # Last connection: Tue Nov 22 16:17:46 CET 2022, Connections since last boot: 1
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 0
MQTTClientContext ClientId=Telegraf-Output-kydL1 CONNECTED
 # Clean: true, Queued QoS0: false, TTL: 0
 # Last connection: -, Connections since last boot: 1
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 0
MQTTClientContext ClientId=telegraf-ingest-vgerndvud2587.xxx.com CONNECTED
 # Clean: false, Queued QoS0: false, TTL: 0
 # Last connection: Mon Dec 12 12:05:30 CET 2022, Connections since last boot: 28
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 6253313
MQTTClientContext ClientId=telegraf-etl-vgerndvud2587.xxx.com-zigbee CONNECTED
 # Clean: true, Queued QoS0: false, TTL: 0
 # Last connection: -, Connections since last boot: 1
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 0
MQTTClientContext ClientId=Telegraf-Output-SL7cM CONNECTED
 # Clean: true, Queued QoS0: false, TTL: 0
 # Last connection: -, Connections since last boot: 1
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 0
MQTTClientContext ClientId=telegraf-etl-vgerndvud2587.xxx.com-zwave CONNECTED
 # Clean: true, Queued QoS0: false, TTL: 0
 # Last connection: -, Connections since last boot: 1
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 0
MQTTClientContext ClientId=Telegraf-Output-fHHWc CONNECTED
 # Clean: true, Queued QoS0: false, TTL: 0
 # Last connection: -, Connections since last boot: 1
 # Delivered: 0, Queued: 0, Waiting: 0, Dropped: 0
g! 
g! gosh: stopping shell

I’m using telegraf with exec plugin + grok input like below

[[inputs.exec]]
  timeout = "10s"
  data_format = "grok"
  command = "/jorammq-mqtt/bin/jorammq-admin -exec \"dumpConnections\""
  grok_patterns = ["MQTTConnection ClientId=%{DATA:clientId} %{WORD:statusString}"]

I’ve tested the pattern with https://grokdebugger.com/ and it seems ok

Problem: only one clientId/statusString metric is created, it’s like grok is stopping the parsing after the first element.

Any idea how to parse the full output content ?

2

What version of telegraf are you currenty using? We recently added support for multiline messages in v1.25.0 via feat(parsers.grok): Add option to allow multiline messages by srebhan · Pull Request #12320 · influxdata/telegraf · GitHub

You would need to enable grok_multiline = true and use the MULTILINEDATA pattern

1 Like
3

Thank you for the information ! I was using an older version of telegraf. Finally I implemented the feature another way, but I know I can do this next time !