Unable to rewrite timestamp using starlark processor

Telegraf
,
1

Setup: Telegraf with the tail plugin and grok data format to parse a local log file that is appended to periodically.

I want to take a timestamp from the log file, convert it to UTC, and write that back into metric.time so that I can query and graph based on the timestamp that is inside the log file that telegraf is consuming. Below is my first attempt, but I get a type error when trying to set metric.time.

What format is metric.time in? Any suggestions on how to convert my UTC time data to one that I can write to metric.time?

year = str(metric.fields["year"])
month = str(metric.fields["month"])
day = str(metric.fields["day"])
hours = str(metric.fields.get("hours", "00"))
minutes = str(metric.fields.get("minutes", "00"))
seconds = str(metric.fields.get("seconds", "00"))

utc_timestamp = year + "-" + month + "-" + day + "T" + hours + ":" + minutes + ":" + seconds + "Z"

utc_parsed_time = time.parse_time(utc_timestamp)
metric.time = utc_parsed_time
2

metric.time is in nanoseconds. the below line is what I used to rewrite the timestamp

utc_parsed_time = time.parse_time(utc_timestamp,“2006-01-02T15:04:05Z”).unix_nano

That unix_nano was the magic I was looking for

3

@adam3ngland
Thanks so much for sharing your solution and welcome! What are you using Influxdb for out of curiosity?

4

Normally just influxdb + telegraf + grafana for system metrics and visibility. This use case came up when we wanted/needed to ingest a 3rd party log file and create some dashboards from it, but query based on when the data was written to the log, not when telegraf learned about it. Bit of a weird use case, but that’s what we get for looking at logs much later :person_shrugging: