Difficulty Importing CSV Data into InfluxDB

InfluxDB 2
,
1

Hi everyone,

I’m experiencing challenges while trying to import CSV data into InfluxDB, and I hope to get some guidance or insights from the community.

Here’s the Background:

I have a dataset that I’m trying to import into InfluxDB using the influx write command. My CSV data is intended to include measurements from sensors, specifically for a Power Usage Measurement application. However, I keep running into errors during the import process.

Example CSV Structure:

Initially, the CSV structure I tried was formatted like this:

plaintext

#datatype measurement,tag,tag,double,dateTime:RFC3339
kWh,domain,entity_id,value,time
sensor,l1_raw_kwh,66666,2024-08-17T03:15:00Z

Error Encountered:

Despite structuring my CSV to reflect what I believed was correct, I received errors related to parsing and handling timestamps. The specific error I faced was:

text

Error: failed to write data: 400 Bad Request: unable to parse 'sensor,domain=sensor,entity_id=l1_raw_kwh value=66666 1723864500000000000': time outside range -9223372036854775806 - 9223372036854775806

This indicated an issue with the timestamp, suggesting it wasn’t being interpreted correctly by InfluxDB.

Attempts to Resolve:

  1. Confirmed the Correct Timestamp Format: Ensured that the timestamps were in RFC3339 format.
  2. Checked Header Consistency: Verified that the #datatype line in the CSV header matched the data that followed.
  3. Incrementally Simplified Data: Attempted to write smaller datasets to see if that resolved the parsing issue.
  4. Used the Dry Run Command: Tried running the import with a dry run to see if it would process correctly without committing changes.

Despite these efforts, I continued to face the same issues, leading to significant frustration as I was unable to import the data as intended.

Request for Assistance:

I’m reaching out to seek advice or best practices on structuring CSV imports into InfluxDB. If anyone has faced a similar problem or has insights into what I might be missing, I would greatly appreciate your help.

Thank you for your time!

2

Did you consider line protocol instead of CSV? It’s simple to implement and it is native to InfluxDB

3

Where i can find this? i mean,Im using ssh and influx cli to communicate now,how can i implement the line protocol

4

I meant converting your dataset in advance. For example this script could help.

As for your CSV attempt, could you share which command (i.e. all arguments) are you using?
Do you specify --precision ns ?

5 
influx write \
  -b home_automation_data \
  -o sktech_organization \
  -p s \
  --format csv \
  -f l1_raw_kwh.csv

this is the command i use
ok i used this command now

  -b home_automation_data \
  -o sktech_organization \
  -p ns \
  --format csv \
  -f l1_raw_kwh.csv \
  --header "#datatype measurement,tag,tag,field,long" \
  --header "m,domain,entity_id,value,time" \
  --debug

with this sample of csv

measurement,domain,entity_id,value,time
kWh,sensor,l1_raw_kwh,66666,1723864500000000000```