Telegraf logparser input specify custom file name with date

Hi all, I am new to this forum and just started using Influx data for some small projects.
We have a requirement to monitor log files that have the date included in the file name and this log file rolls over daily. I have tried searching this forum and the github pages for an example of how to define this with no success so far.

An example of the log file is:

  • filename: 2018_01_23.user-events.csv

How can I define this in the configuration of telegraf under the inputs.logparser to use the current year_month_day as a variable prefix to the log file name?

Thanks for your time and responses

I think you will want to change the filename to be more logparser friendly, instead of appending to a file based on date you would need to always write to a static filename and move the old file at the end of the day.

Hi Daniel, thanks for your reply.
So in essence you are saying its not possible to have this done to have a dynamic file name based on date?

Telegraf needs to know which files are still active so it can follow these files, and this would need to match the glob in the files option. You won’t want it to match too many files either, since it could become expensive from a resource point of view. You could probably stick with the current naming scheme so long as you move older files to another directory in order to remove them from matching the glob, or if you were to update the Telegraf config periodically with a new glob.

Thanks Daniel. I think this would be a useful feature since there are some applications that generate in a way that allows the date to be in the file name. In my case, I dont have control over the naming but the matching file name will always be unique and limited to 1 file per day. If we had a variable way to generate this unique file name it would help (even if it read this name from an external file that has an external cron setting this value). Im really trying to automate the monitoring of this log file since it outputs useful stats that I have graphs/alerts generated on.

If there’s any remote possibility or any workarounds, i would be interested to know.