CSV file returns only column names

SpaceRodento
Mon, 17 Jul 2023 05:25:16 +0000
Telegraf

Hi all

I feel like I’m getting somewhere but hitting a brick wall on last meters.

Trying to import CSV to influxdb with Telegraf File plugin. Idea is to manually place .csv into a folder which is monitored by telegraf.

Tested the connection to database with hard drive-plugin and it showed ok in Data Explorer

Only column names are showing up in Data Explorer, no data is being transferred and I cant find anything wrong with config, date format or anything else.

CSV in question should be ok. Excel opened it up without problems and MSSQL could also import it. Using V1 shell, no data can be seen in bucket

If there’s better way to achieve this or a way to test it in command line, I’m happy to change to whatever works.

Debug mode on:

2023-07-17T04:18:51Z I! Loading config: /etc/telegraf/telegraf.conf
2023-07-17T04:18:51Z I! Starting Telegraf 1.27.2
2023-07-17T04:18:51Z I! Available plugins: 237 inputs, 9 aggregators, 28 processors, 23 parsers, 59 outputs, 4 secret-stores
2023-07-17T04:18:51Z I! Loaded inputs: file
2023-07-17T04:18:51Z I! Loaded aggregators:
2023-07-17T04:18:51Z I! Loaded processors:
2023-07-17T04:18:51Z I! Loaded secretstores:
2023-07-17T04:18:51Z I! Loaded outputs: influxdb_v2
2023-07-17T04:18:51Z I! Tags enabled: dc=finland host=tupu
2023-07-17T04:18:51Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"tupu", Flush Interval:10s
2023-07-17T04:18:51Z D! [agent] Initializing plugins
2023-07-17T04:18:51Z D! [agent] Connecting outputs
2023-07-17T04:18:51Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2023-07-17T04:18:51Z D! [agent] Successfully connected to outputs.influxdb_v2
2023-07-17T04:18:51Z D! [agent] Starting service inputs
2023-07-17T04:19:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics

“Buffer fullness” keeps repeating.

Config, inputs.file -part

[[inputs.file]]
  files = ["/home/pi/database/Log_2023.csv"]
  data_format = "csv"
  csv_header_row_count = 1
  csv_skip_rows = 0
  csv_column_names = ["Row", "Sensor", "Date time", "Cumulative chip Count", "1h count Frequency", "24h Count", "Moisture", "Temperature T", "P2_Rough", "P2_Calibrated", "F0", "Fdif", "Tdif", "DeltaF", "P", "A", "B", "C", "D", "E", "F", "N", "T1", "T2", "T3", "DevResults", "CustomerID1", "CustomerID2", "CustomerID3", "Misc1", "Misc2", "Misc3"]
# trying ; and ,
csv_delimiter = ";"


  csv_trim_space = true  # Set this to true if you want to trim leading and trailing spaces in the fields
  csv_tag_overwrite = false
  csv_measurement_column = ""  # Leave this empty if you want to use the default measurement name ("file")
  csv_timestamp_column = "Date time"  # Specify the column containing the timestamp
  csv_timestamp_format = "2006-1-2 15:04:05"  # Format of the timestamp column
  csv_timezone = "Europe/Helsinki"
  csv_skip_errors = false

Few rows of the .csv in question.

Row;Sensor;Date time;Cumulative chip Count;1h	count Frequency;24h Count;Moisture;Temperature T;P2_Rough;P2_Calibrated;F0;Fdif;Tdif;DeltaF;P;A;B;C;D;E;F;N;T1;T2;T3;DevResults;CustomerID1;CustomerID2;CustomerID3;Misc1;Misc2;Misc3;
1;Engine 1;2023-3-13 13:13:36;3;0;0;0.0;1372.0;0.0;0.0;0.0;0.0;0.0;0.0;0.1;-0.007;-0.01;0.00042;0.0125;0.0003;8.58;1.0;50.0;55.0;60.0;FALSE;OM-Finland;Autotalli 1;Testipenkki 1;;12333;;
2;Engine 2;2023-3-13 13:13:36;0;0;0;0.0;19.9;0.0;0.0;0.0;0.0;0.0;0.0;0.1;-0.007;-0.01;0.00042;0.0125;0.0003;8.58;1.0;50.0;55.0;60.0;FALSE;OM-Finland;Pieksämäki 1;Turbiini 1;;;;
3;Engine 3;2023-3-13 13:13:36;0;0;0;0.0;;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;FALSE;OM-Finland;Jääkiekkohalli 5;Mopo 3;45;;;
4;Engine 1;2023-3-13 13:16:35;3;0;0;0.0;1372.0;0.0;0.0;0.0;0.0;0.0;0.0;0.1;-0.007;-0.01;0.00042;0.0125;0.0003;8.58;1.0;50.0;55.0;60.0;FALSE;OM-Finland;Autotalli 1;Testipenkki 1;;12333;;
5;Engine 2;2023-3-13 13:16:35;0;0;0;0.0;20.0;0.0;0.0;0.0;0.0;0.0;0.0;0.1;-0.007;-0.01;0.00042;0.0125;0.0003;8.58;1.0;50.0;55.0;60.0;FALSE;OM-Finland;Pieksämäki 1;Turbiini 1;;;;
6;Engine 3;2023-3-13 13:16:35;0;0;0;0.0;;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;0.0;FALSE;OM-Finland;Jääkiekkohalli 5;Mopo 3;45;;;

syslog file excerpt, does this open up anything?

Jul 16 22:47:39 tupu systemd[1]: Started Session 13 of user pi.
Jul 16 22:47:46 tupu influxd-systemd-start.sh[532]: ts=2023-07-16T19:47:46.316115Z lvl=error msg="Unable to write gathered points" log_id=0j3i5a>Jul 16 22:47:56 tupu influxd-systemd-start.sh[532]: ts=2023-07-16T19:47:56.303885Z lvl=error msg="Unable to write gathered points" log_id=0j3i5a>Jul 16 22:47:59 tupu systemd[1]: session-13.scope: Succeeded.
Jul 16 22:47:59 tupu systemd[1]: session-13.scope: Consumed 1.640s CPU time.

4 posts - 3 participants