Telegraf_CSV Issue

Telegraf
,
1

Hi,

I have been trying to implement to read csv file from local system and push the output to InfluxDB. But getting some error.
I appreciate if someone could help me in resolving this issue.

Thanks,
Manju

Configuration details:
[[inputs.file]]
files = [“/apps/sftp-ein/palladion-1712041200-22389.csv”]
data_format = “csv”
csv_delimiter = “;”
csv_header_row_count = 1
csv_tag_columns = [“uid”,“state_details”]
csv_measurement_column = “measurement”
csv_timestamp_format = “2006-01-02T15:04:05Z07:00”

Error:
[root@sandmonitor telegraf.d]# telegraf --config telegraf_cdr.conf
2024-05-02T11:47:19Z I! Starting Telegraf 1.23.3
2024-05-02T11:47:19Z I! Loaded inputs: cpu disk diskio file kernel mem processes swap system
2024-05-02T11:47:19Z I! Loaded aggregators:
2024-05-02T11:47:19Z I! Loaded processors:
2024-05-02T11:47:19Z I! Loaded outputs: influxdb influxdb_v2
2024-05-02T11:47:19Z I! Tags enabled: host=sandmonitor
2024-05-02T11:47:19Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:“sandmonitor”, Flush Interval:10s
2024-05-02T11:47:19Z W! [outputs.influxdb] When writing to [http://localhost:8086]: database “telegraf” creation failed: 401 Unauthorized
2024-05-02T11:47:21Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-05-02T11:47:21Z W! [outputs.influxdb] Metric buffer overflow; 762 metrics have been dropped
2024-05-02T11:47:21Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-05-02T11:47:21Z W! [outputs.influxdb] Metric buffer overflow; 4635 metrics have been dropped
2024-05-02T11:47:21Z E! [outputs.influxdb_v2] Failed to write metric to cdr (will be dropped: 422 Unprocessable Entity): unprocessable entity: failure writing points to database: partial write: field type conflict: input field “init_devs” on measurement “file” is type string, already exists as type integer dropped=835
2024-05-02T11:47:21Z W! [outputs.influxdb_v2] Metric buffer overflow; 6934 metrics have been dropped
2024-05-02T11:47:21Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-05-02T11:47:21Z W! [outputs.influxdb] Metric buffer overflow; 11644 metrics have been dropped
2024-05-02T11:47:21Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-05-02T11:47:21Z E! [outputs.influxdb_v2] Failed to write metric to cdr (will be dropped: 422 Unprocessable Entity): unprocessable entity: failure writing points to database: partial write: field type conflict: input field “ingress_devs” on measurement “file” is type integer, already exists as type string dropped=911
2024-05-02T11:47:21Z W! [outputs.influxdb_v2] Metric buffer overflow; 11107 metrics have been dropped

Actual Raw File:

2

You have two errors/issues:

When writing to [http://localhost:8086]: database “telegraf” creation failed: 401 Unauthorized

This says you do not have the permissions to create a database. If you want Telegraf to create a database when it writes data it needs a certain level of permissions. Either your token is not a write token or you are using fine grained tokens and cannot create databases.

Failed to write metric to cdr (will be dropped: 422 Unprocessable Entity): unprocessable entity: failure writing points to database: partial write: field type conflict: input field “init_devs” on measurement “file” is type string, already exists as type integer dropped=835

This says that your schema for this database expects init_devs to be a string, not an integer, so that line is dropped.

3

Thank you @jpowers for your quick response.

You can ignore the 401 anauthorized error for outputs.influxdb, since I am using outputs.influxdb_v2.

I hope no action needed from my end for this schema error.

Thanks,
Manju

4

You don’t need to do anything, but you are dropping data/metrics. If this is expected then great, otherwise, yes you may need to update your database schema or drop the database and start writing data again: