Error parsing time in [inputs.mqtt_consumer] plugin

Hello
.
My question is directly related to my previous topic, available here: How to convert a date (string format) into a timestamp (for primary key)?
.
I set up in telegraf a data source from an mqtt queue. I use the “time” variable available in the json stream to time stamp my data.

Here is the piece of code that allows me to do this:

[[inputs.mqtt_consumer]]
  servers = ["ssl://xxxxxxxxxxxxxx:8883"]
  topics = ["application/1/device/+/event/up"]]
  qos = 0
  username = "user
  password = "password
  data_format = "json
  tls_cert = "/etc/ssl/ca-cert.crt
  tls_key = "/etc/ssl/ca-cert.key".
  tag_keys = [
    "deviceName",
    "devEUI",
    "rxInfo_0_location_altitude",
    "rxInfo_0_location_latitude",
    "rxInfo_0_location_longitude",
  ]
  fieldpass = [
    "object_barometer_6",
    "object_humiditySensor_7",
    "object_temperatureSensor_2",
  ]
  # parse time
  json_time_format = "2006-01-02T15:04:05.000000Z" 
  json_time_key = "rxInfo_0_time". 
  json_timezone = "UTC 

In the telegraf logs, I regularly get a time parsing error that prevents data storage. This error is linked to a bad format of microseconds from the gateway (which I don’t have control over).

Here is the error:

2020-11-02T15:56:43Z E! [inputs.mqtt_consumer] Error in plugin: parsing time "2020-11-02T15:56:42.84033Z" as "2006-01-02T15:04:05.000000Z": cannot parse "" as ".000000"

My question: is it possible, in the json_time_format variable, to manage several time formats? Like “2006-01-02T15:04:05.000000Z”, “2006-01-02T15:04:05.00000Z”, “2006-01-02T15:04:05.0000Z”, “2006-01-02T15:04:05.000Z”, …
.
I’ve just tried this but it didn’t work :

  json_time_format = [
    "2006-01-02T15:04:05.000000Z", 
    "2006-01-02T15:04:05.00000Z", 
    "2006-01-02T15:04:05.0000Z", 
    "2006-01-02T15:04:05.000Z", 
    "2006-01-02T15:04:05.00Z", 
    "2006-01-02T15:04:05.0Z", 
    "2006-01-02T15:04:05Z"
  ] 

Thanking you in advance for your help
Thierry

1 Like

the solution to this problem is to replace the 0’s by 9’s to manage the deletion of end zeros :

json_time_format = "2006-01-02T15:04:05.999999Z"