My use case is processing logs using inputs.tail. The first stage is to do the basic breakup of the log message which works just fine. The remaining portion is captured by %{GREEDYDATA:message}. I now want a second pass of the newly created “message” field to enrich the metric with a number of tags (I am on telegraf 1.26.x).
What I have tried:
Using processor.regex, but this fails as I have to repeat the same regex over and over for each tag I want to create. Note I am not always capturing data to create the tags.
[[processors.regex.fields]]
# extract state
key = “message”
pattern = ‘.old state Established.’
replacement = “down”
result_key = “state”
[[processors.regex.fields]]
# normalise message
key = “message”
pattern = ‘.old state Established.’
replacement = “BGP Neighbor Down”
result_key = “message_normalised”
[[processors.regex.fields]]
# extract state
key = “message”
pattern = ‘.new state Established.’
replacement = “up”
result_key = “state”
[[processors.regex.fields]]
# normalise message
key = “message”
pattern = ‘.new state Established.’
replacement = “BGP Neighbor Up”
result_key = “message_normalised”
Using processor.override. This allows creating multiple tags as I require, but I cannot trigger the processor on contents of a specific field (something like field contents = glob pattern. I can only do this on field name using fieldpass).
[[processors.override]]
[processors.override.tags]
state = “down”
message_normalised = “Interface Down”
level = “warning”
[[processors.override]]
[processors.override.tags]
state = “up”
message_normalised = “Interface Up”
level = “normal”
Other options:
Using starlark: As per docs its slow
Using the new Common Expression Language (CEL) feature: As per docs its slow
Using the new Allow batch transforms using named groups processor.regex feature (pull #13971): Not going to work for creating new tags or fields as the tag or field value will come from the capture group, not a constant.
So to rephrase, is there a way to do the following:
Trigger processor only if field X contents matches glob Y, then create a number of constant tags.
Processor.override looks like the best fit for this.
I have searched for a fieldpass method to filter on contents of a field, not on field name, but have been unsuccessful. Any suggestions on how to achieve this in the most optimal way?
