I have got four 4 streams and for each streams i have got one measurement. So for one of measurement this error i am getting - [inputs.kinesis_consumer] Scan encountered an error: shard shardId-000000000000 error: parsing time “T00:15:01” as “2006/01/02T15:04:05”: cannot parse “T00:15:01” as “2006”. So i wanted to know why this error is coming and what would be solution as no other data point got written.
Hello @Pratik_Das_Baghel,
I’m not sure.
Can you please share some of your other input data as well as your telegraf config?
Does all of your data have the same timestamp format of 2006/01/02T15:04:05?
Thank you
Below we are providing [[inputs.kinesis_consumer]] as we mentioned in our telegraf.conf file
[[inputs.kinesis_consumer]]
region = "ap-south-1"
access_key =provided
secret_key = provided
profile = provided
streamname = "test_stream"
shard_iterator_type = "TRIM_HORIZON"
https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
data_format = "csv"
csv_header_row_count = 1
csv_tag_columns = ["ver","node","operatorName","type","data_type"]
csv_timestamp_column = "datetime"
csv_timestamp_format = "2006/01/02T15:04:05"
[inputs.kinesis_consumer.tagpass]
data_type = ["test","production"]
below I am mentioning telegraf logs:
2021-04-14T09:55:47Z D! [outputs.influxdb_v2] Wrote batch of 1000 metrics in 607.975855ms
2021-04-14T09:55:47Z D! [outputs.influxdb_v2] Buffer fullness: 100000 / 100000 metrics
2021-04-14T09:55:47Z W! [outputs.influxdb_v2] Metric buffer overflow; 572 metrics have been dropped
2021-04-14T09:55:47Z D! [outputs.influxdb_v2] Wrote batch of 1000 metrics in 338.465733ms
2021-04-14T09:55:47Z D! [outputs.influxdb_v2] Buffer fullness: 99000 / 100000 metrics
2021-04-14T09:55:47Z E! [inputs.kinesis_consumer] Scan encountered an error: shard shardId-000000000000 error: context canceled
2021-04-14T09:55:47Z E! [inputs.kinesis_consumer] Scan encountered an error: shard shardId-000000000000 error: context canceled
2021-04-14T09:55:47Z D! [agent] Input channel closed
2021-04-14T09:55:47Z D! [agent] Processor channel closed
2021-04-14T09:55:47Z D! [agent] Processor channel closed
2021-04-14T09:55:47Z I! [agent] Hang on, flushing any cached metrics before shutdown
My stream working fine and I am getting the data in InfluxDB as well.
Now, suppose I have 10 columns in file and I am pushing and everything is working fine like telegraf reading the data from the kinesis stream and pushing into the InfluxDB but as soon as I add one more column or remove one column into the file telegraf start giving the error as above mentioned and this error will go away only we delete the stream and create the stream with the same name again. Kinesis still reading the data properly if we add or remove the columns in the file.
here you are seeing date and time column are different but in our lambda function, we combine these two fields into one column DateTime, and in proper format as telegraf need.
when we change the data means adding columns it also gives the error:
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
2021-04-14T10:03:05Z D! [inputs.kinesis_consumer] Metric group failed to process
This error is resolved only when we create the stream and re-create the stream with the same name.
Yes the format is same for all data points. Also we get this error - “Metric group failed to process” do you know about this error? Is it related to processor plugin?
@Pratik_Das_Baghel regarding the failure to pare the timestamp, what data format are you using? Each parser plugin has information on the supported timestamp format in their corresponding README. For example for CSV you can find it here telegraf/plugins/parsers/csv at master · influxdata/telegraf · GitHub What is the timestamp format you are using?
The “Metric group failed to process” error you are seeing happens when the metric isn’t marked as delivered by Telegraf, I am not entirely sure why it is being outputted in your case.
Source code reference for the error: telegraf/kinesis_consumer.go at master · influxdata/telegraf · GitHub
I am using this format: “2006/01/02T15:04:05”.
