Telegraf Json Input unable to parse Date

Telegraf
1

Hi i have a test Json file with timestamp field as below

2019-03-06T11:19:48.0000855Z

I keep on getting below error when starting telegraf to read the json test file

2019-03-06T13:07:10Z E! [inputs.file]: Error in plugin: parsing time “2019-03-06T11:19:48.0000855Z”: month out of range

Any ideas how i can parse the dateformat as causing me a headache at the moment

My config is as below
[[inputs.file]]
files = [“D:/Logs/Apps/TeleGrafWindowsPerfMonAgent/TestJson.log”]

Data format to consume.

Each data format has its own unique set of configuration options, read

more about them here:

telegraf/DATA_FORMATS_INPUT.md at master · influxdata/telegraf · GitHub

data_format = “json”

Query is a GJSON path that specifies a specific chunk of JSON to be

parsed, if not specified the whole document will be parsed.

GJSON query paths are described here:

GitHub - tidwall/gjson: Get JSON values quickly - JSON parser for Go

json_query = “”

Tag keys is an array of keys that should be added as tags.

tag_keys = [
“hostname”
]

String fields is an array of keys that should be added as string fields.

json_string_fields = [“CurrentItemSubscriptions”,“MaxItemSubscriptions”,“TotalSubscribedItems”,“Usernames”]

Name key is the key to use as the measurement name.

json_name_key = “LightStreamerJmxResource”

Time key is the key containing the time that should be used to create the

metric.

json_time_key = “timestamp”

Time format is the time layout that should be used to interprete the json_time_key.

The time must be unix, unix_ms, unix_us, unix_ns, or a time in the

“reference time”. To define a different format, arrange the values from

the “reference time” in the example to match the format you will be

using. For more information on the “reference time”, visit

time package - time - Go Packages

ex: json_time_format = “Mon Jan 2 15:04:05 -0700 MST 2006”

json_time_format = “2006-01-02T15:04:05Z07:00”

json_time_format = “01/02/2006 15:04:05”

json_time_format = “unix”

json_time_format = “unix_ms”

json_time_format = “2019-03-06T11:19:48.0000855Z”

Timezone allows you to provide an override for timestamps that

don’t already include an offset

e.g. 04/06/2016 12:41:45

Default: “” which renders UTC

Options are as follows:

1. Local – interpret based on machine localtime

2. “America/New_York” – Unix TZ values like those found in List of tz database time zones - Wikipedia

3. UTC – or blank/unspecified, will return timestamp in UTC

##json_timezone = “”

2 Likes
2

Make sure to use the so called “reference date” in your json_time_format, this is the exact date Mon Jan 2 15:04:05 -0700 MST 2006. I haven’t tested this but it should be:

json_time_format = "2006-01-02T15:04:05.000000Z"
3

To help clear this up… I had the same issue:

Telegraf.conf

[[inputs.file]]
   files = ["out.json"]
   json_time_key = "time"
   json_time_format = "2010-04-17T17:00:00.000Z"

out.json

{ "foo" : 2.0, "time": "2018-11-17T17:49:52.357Z"}

error

2019-04-19T20:52:20Z E! [inputs.file]: Error in plugin: parsing time "2018-11-17T17:49:52.357Z": month out of range

I selected an arbitrary date for telegraf.conf and the plugin requires that we use the reference date of "2006-01-02T15:04:05Z07:00

So, I needed to change telegraf.conf to

Telegraf.conf

[[inputs.file]]
   files = ["out.json"]
   json_time_key = "time"
   json_time_format = "2006-01-02T15:04:05Z07:00"

Which worked…

2019-04-19T21:05:16Z D! [agent] Starting service inputs
2019-04-19T21:05:30Z D! [outputs.influxdb] wrote batch of 1 metrics in 402.131078ms
2019-04-19T21:05:30Z D! [outputs.influxdb] buffer fullness: 1 / 10000 metrics.
3 Likes
4

2 years late but this helped resolve my issue. thanks @John_Sobanski !!

1 Like
5

JSON itself does not specify how dates should be represented, but JavaScript does. What .NET does is a non-standard hack/extension. The problem with JSON date and really JavaScript in general – is that there’s no equivalent literal representation for dates. In JavaScript following Date constructor straight away converts the milliseconds since 1970 to Date as follows:

var jsonDate = new Date(1297246301973);

Then let’s convert it to js format:

var date = new Date(parseInt(jsonDate.substr(6)));

The substr() function takes out the /Date( part, and the parseInt() function gets the integer and ignores the )/ at the end. The resulting number is passed into the Date constructor .

For ISO-8601 formatted JSON dates, just pass the string into the Date constructor:

var date = new Date(jsonDate);