System info: InfluxDB version 1.6
Operating system: CentOS Linux release 7.3
CSV file (37GB) and InfluxDB server reside on a 16 core, 32GB RAM VM
Steps to reproduce:
- My schema consists of 50 measurements with 1000 different tag values (so 50K unique series) and field values contain 1 float, 3 int and 1 timestamp value. The timestamp values are in microsecond (‘u’) precision. I have adhered to all the recommendations given to migrate historical data. The retention policy for the DB is 3years, the shard duration is 52w and I have changed the cache-snapshot-write-cold-duration to 10s.
- I have used the influxdb-python client to pick up points from the CSV and use the default JSON format to call the write_points API with 6000 points in the JSON body.
Expected behavior: The migration of 37GB data from a CSV file should be fairly fast.
Actual behavior: I have been running the script since 18 hours and the migration is still going on. If I check the physical location where influxdb stores the data, it shows that it has transferred only 4GB till now. I know there should not be a correlation between the size of the CSV file and the data stored in the physical location of influxdb server but I have no other way to gauge the progress.
[EDIT] The physical location of influx looks like this after 19 hours (asup_metrics being the DB name and historical_data is the retention policy name):
Could anyone kindly point out where I might be going wrong?