[inputs.file] (CSV) Error in plugin: parsing date time

Telegraf
1

Hi,
Newbie here trying to parse some input csv log-files without success. It gets stuck on the time stamp with error:
[inputs.file] Error in plugin: parsing time “2021-02-25 14:29:0,73200” as “2006-01-02 15:04:05.999999”: cannot parse “0,73200” as “05”

I am guessing the parser does not like the date time format in the csv-file and I have tried to change “csv_timestamp_format” without luck.

Appreciate all the help.
/Daniel

Csv-file:
“Record Type”,“Record Code”,“Broker Name”,“Broker UUID”,“EG Name”,“EG UUID”,“Message Flow Name”,“Message Flow UUID”,“Application Name”,“Application UUID”,“Library Name”,“Library UUID”,“Record Start Date”,“Record Start Time”,“Record GMT Start Timestamp”,“Record End Date”,“Record End Time”,“Record GMT End Timestamp”,“Total Elapsed Time”,“Average Elapsed Time”,“Maximum Elapsed Time”,“Minimum Elapsed Time”,“Total CPU Time”,“Average CPU Time”,“Maximum CPU Time”,“Minimum CPU Time”,“CPU Time Waiting for Input Messages”,“Elapsed Time Waiting for Input Messages”,“Total Number of Input Messages”,“Total Size of Input Messages”,“Average Size of Input Messages”,“Maximum Size of Input Messages”,“Minimum Size of Input Messages”,“Number of Threads in Pool”,“Time Maximum Number of Threads reached”,“Total Number of MQ Errors”,“Total Number of Messages with Errors”,“Total Number of Errors Processing Messages”,“Total Number of Time Outs Waiting for Replies to Aggregate Messages”,“Total Number of Commits”,“Total Number of Backouts”,“Accounting Origin”
“SnapShot”,“SnapShot”,“BK.ONE001Q”,“6afd0d46-564e-4b2d-b6d5-3e1ec65f77a0”,“default”,“b5fdbea4-3c01-0000-0080-8f3830e43e40”,“SVC025_ReceiveLabTestInformation”,“11c36a59-4101-0000-0080-dbc886dda341”,“SVC025_ReceiveLabTestinformation”,“c4096a59-4101-0000-0080-dbc886dda341”,"","",“2021-02-25”,“15:28:40,688999”,“2021-02-25 14:28:40,6889”,“2021-02-25”,“15:29:0,732000”,“2021-02-25 14:29:0,73200”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“20042753”,“0”,“0”,“0”,“0”,“0”,“1”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“Anonymous”
“SnapShot”,“SnapShot”,“BK.ONE001Q”,“6afd0d46-564e-4b2d-b6d5-3e1ec65f77a0”,“default”,“b5fdbea4-3c01-0000-0080-8f3830e43e40”,“SVC029_ArticleStockList”,“4ac03db1-4801-0000-0080-ddb1487caab0”,“SVC029_ArticleStockList”,“80bf3db1-4801-0000-0080-ddb1487caab0”,"","",“2021-02-25”,“15:28:44,779999”,“2021-02-25 14:28:44,7799”,“2021-02-25”,“15:28:59,868000”,“2021-02-25 14:28:59,8680”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“15087900”,“0”,“0”,“0”,“0”,“0”,“1”,“0”,“0”,“0”,“0”,“0”,“0”,“0”,“Anonymous”

.conf-file:
[[inputs.file]]
files = [“C:\Temp\telegraf_in\*”]
data_format = “csv”
csv_header_row_count = 1
csv_skip_rows = 0
csv_skip_columns = 5
csv_comment = “#”
csv_measurement_column = “measurement_name”
csv_tag_columns = [“tag_key”]
csv_timestamp_column = “Record GMT End Timestamp”
##csv_timestamp_format = “2006-01-02 15:04:00.161182”
csv_timestamp_format = “2006-01-02 15:04:05.999999”

2

The timestamps in the CSV file are formatted a bit unusual. The seconds/microseconds have a comma instead of a period. The parser can’t swallow that because the configured format doesn’t match. Maybe it works with:

csv_timestamp_format = "2006-01-02 15:04:05,999999"

However, I suspect that this won’t work either…
Could you change the output format of the csv file or is there nothing you can do about it?

3

Thanks, I also tried csv_timestamp_format = “2006-01-02 15:04:05,999999” without luck.

I can´t change the output format. It is from IBM Integration Bus.

4

Hm, unfortunately I don’t know anything about the time parser of Go either… :innocent:

If no one else has any idea how to configure the parser properly:
Is this a one-time import? If yes, could you search and fix the timestamps with regex before the import?