Telegraf Time Conversion To Epoch

Telegraf
1

Hello,

I am trying to parse a log into syslog format so that it can be read by the Log Viewer in Chronograf. The format of the date and time is static in the logs using 2019-07-16 00:02:41 as an example. Is it possible to use a plugin to convert this into epoch time on the output so that it can properly have the time in Log Viewer?

Thanks.

2

What does your config file look like (input plugin/parser if any)? If you are using the grok parser, there is a way to define a custom timestamp format that will then be used as the metric timestamp. See the docs for more information. If you are using the syslog input plugin, the syslog timestamp in the syslog message gets stored as a field.

3

I am using this pattern for Grok

  • patterns = [’%{TIMESTAMP_ISO8601:timestamp:ts-epoch} %{IP:facility:tag} %{URIPROTO:message:string} %{URIPATH:procid:string} (?:%{NOTSPACE:queryparam:drop}|-) %{NUMBER:facility_code:int} (?:%{WORD:hostname:tag}|-) %{IP:clientip:drop} %{NOTSPACE:appname:tag} (?:%{NOTSPACE:referer:drop}|-) %{NUMBER:severity:tag} %{NUMBER:severity_code:int} %{NUMBER:version:int} %{NUMBER:time_taken:drop}’]

But in Log Viewer the timestamp shows the event at 12/31/1969

4

My guess is that this is not one of the "most" that’s not exactly supported as stated here. There has been issues with iso8601, maybe try manually defining your date as discussed here.

5

I’ve adjusted it so that it is:

  • %{MY_TIME:timestamp:ts-epoch}
  • MY_TIME %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}
    but no luck so far.

My only next guess would be somehow the number at the end is what Log Viewer is trying to pickup the time from?
When I output to a file to see the parsed information there is a long number like 1563828698333335501 that trails.

6

When you output to a file, is in in influx format? (epoch timestamp trails) Can you paste a concrete example of what you are seeing?

7

This is an example from the output file.

  • syslog,appname=Microsoft-WebDAV-MiniRedir/10.0.18362,facility=10.10.40.3,host=Test,severity=warning facility_code=80i,severity_code=0i,version=5i,procid="/SystemResources/server.exe.mun",message=“PROPFIND” 1563235366000000000
    pic
    pic.png1797×393 56.3 KB