Data ingestion using the api

Store
1

Hello,
I have csv files that i want to be inserted into influxdb .
I decide to use the api with curl after i test it with telegraf, but it was not the best solution for my situation.
So i did a big data injection , i have arounf 150279292 lines into influxdb .
At begin the files was inserted really quickly and had no problem , but after a while is stated to be longer , even if the files has mostly the same number of lines ,and i have a lot of time out randomly.
For example - this is the output of my scrip. When i have Sleep 5 s and retry… meen that i have a timeout code received and the other line when the files are well inserted.
I don’t know much in influx configuration , but maybe somebody can face already with this kind of problem, and maybe can tell me what to look for to understand and improove the situation.
Coz i need a sistem real time and if i will have timeout like that all the time i’m inserting , maybe i need to reconsider my choice for influx :(.
Hope someone can redirect me in the good direction to find the solution for this timeout.
Thank you !

File /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005304_5.dat inserted in 3.623691914 seconds
Sleep 5 s and retry for file /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005401_1.dat
File /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005404_2.dat inserted in .942420306 seconds
Sleep 5 s and retry for file /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005501_1.dat
File /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005504_2.dat inserted in .921208652 seconds
File /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005504_3.dat inserted in .588656956 seconds
Sleep 5 s and retry for file /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005601_1.dat
File /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005604_2.dat inserted in 2.770216782 seconds
Sleep 5 s and retry for file /opt/sc/dev/projChiriac/dataFromT/inFiles/result_all_20200124005701_1.dat

2

Hello @cata,
What version of InfluxDB are you using? Just to make sure, are you using telegraf? If so, have you tried:

3

Hello,
I’m using influxdb-1.7.7 version.
And no i’m not using telegraf , just the api write. i’m doing a curl
curl -i https://local:8082/write?db=database -u admin:password --data-binary @$resultFile .
I’ve tried telegraf also , but it’s not the right solution for me , because i need to know when a file is treated and telegraf can’t assure me that the file is already treated, so i choose using influx API.
Thanks for help.