Just another Influxdb1 to influxdb2 migration question (csv)

,
1

Hi there,
I’ve remote access to a InfluxDBv1 instance and would like to migrate my data to a local InfluxDBv2 instance (with gui).
I’m trying to export in CSV from v1 and then import to v2.
I’m exporting the data this way:

influx -host influxdb-remote -port 8086 -database 'telegraf' -ssl -execute 'select * from vCenter' -format csv > vCenter.csv

This creates me a file in this format:

name,time,clusters,customer_vm,customer_vm_percent,hostname,hosts,internal_vm,internal_vm_percent,linux,other,poweredoff,poweredon,total,windows
vCenter,1705276801315756940,1,145,100,vcenter001,12,0,0,142,3,14,131,145,0

BTW, trying to import that file via webgui into InfluxDB2 as CSV or Line Protocol it doesn’t work.

CSV:
error in csv.from() failed to read metadata missing expected annotation datatype

Line Protocol:
Failed to write data - invalid line protocol submitted

Ideas?
Thanks,

Simon

2

@xefil The UI CSV uploader is actually pretty strict and requires the CSV to include all CSV annotations to identify times, measurements, tags, fields, and data types AND expects the data to be in the v2 result format which is different than v1. Essentially the UI CSV uploader is only good for uploading CSV that is returned from querying an InfluxDB v2 instance.

I think your best option would but to use the v2 influx CLI.

The examples below assume the following about your data:

  • name (measurement)
  • time (timestamp, nanosecond)
  • clusters (field, int)
  • customer_vm (field, int)
  • customer_vm_percent (field, float)
  • hostname (tag)
  • hosts (field, int)
  • internal_vm (field, int),
  • internal_vm_percent (field, int)
  • linux (field, int)
  • other (field, int)
  • poweredoff (field, int)
  • poweredon (field, int)
  • total (field, int)
  • windows field, int)

You can use the v2 influx CLI’s write command to upload the CSV. The CLI supports Extended Annotated CSV and lets you inject annotation headers at execution time. The command would look something like this:

Note: The following example assumes you’ve defined your InfluxDB v2 host, org, and token in either your influx profile configuration or environment variables.

influx write \
  --bucket example-bucket \
  --header "#datatype measurement,dateTime:number,long,long,double,tag,long,long,double,long,long,long,long,long,long" \
  --file ./vCenter.csv
3

Hello @scott .
I’ve tried to remove the first line from the csv, having the header of the csv, and executing this I get this error:

# influx write --bucket telegraf \
--header "#datatype measurement,dateTime:number,long,long,double,tag,long,long,double,long,long,long,long,long,long" \
--file ./vCenter-noheader.csv \
-o MYORG \
-t MYTOKEN

Error: failed to write data: 500 Internal Server Error: unexpected error writing points to database: field type conflict

Parameters should be ok, because changing token I get 401 Unauthorized: unauthorized access or changing Org I get 404 Not Found: organization name "XYZ" not found so I think there is something error on data.

Possible?

Thanks, Simon

4

@xefil You shouldn’t remove the first line of the CSV. If you do, you’re going to have to update the #datatype annotation.

5

Thanks that was the error. I’ve then corrected come errors on datatype by myself (flat vs integer) and now it’s working.
Thanks a lot for the support @scott

1 Like