Help with multiline input.tail log parser

Telegraf
1

Hi,

I am trying to create a telegraf input from tailing a logfile from one of our systems so that I can alert users upon job completion.

The logfile format is custom and multiline. Below is an example of the last three events.

I have found the [[inputs.tail]] plugin and the [inputs.tail.multiline] configuration but I am a bit stuck with how to actually implement the parsing. I need to be able to separate the events, and include some of the information about the job, what the label is, whether it started or finished, what the duration was etc.

I could build a custom parser but it seems like all the tools are there already if I’m able to configure things correctly. I’d really appreciate some help. Cheers

2021.01.19 12:08:49.376 Info JobDispatcher->Job finished successfully, duration [min]: 98
GreenHouseImage
  greenhouse = NW_SE_GH2
  lanes = 1,2,3,4,5,6,7,8,9
  enableWateringWeighing = True

(Scheduler-0573 W and Im)
2021.01.20 10:30:00.792 Info JobDispatcher->Starting job:
GreenHouseImage
  greenhouse = NW_SE_GH2
  lanes = 1,2,3,4,5,6,7,8,9
  enableWateringWeighing = True

(Scheduler-0573 W and Im)
2021.01.20 12:41:29.729 Info JobDispatcher->Job finished successfully, duration [min]: 131
GreenHouseImage
  greenhouse = NW_SE_GH2
  lanes = 1,2,3,4,5,6,7,8,9
  enableWateringWeighing = True

(Scheduler-0573 W and Im)
2

Hello @gsainsbury86 ,
Welcome!
Have you taken a look at the execd processor plugin? You could execute the parsing in the language of your choice :slight_smile: