Identify Tags and Fields for multiple CSV files

Store
1

Hi team,

I have already completed a project of real-time monitoring by using Python for data ingestion of the CSV file. Since we have completed this but we need know to move ahead with the more complex project and I wish someone has solved this before already.

We are thinking to create a tool in which anyone can dump multiple CSV files, and respective data is stored within Influx with corresponding tags and values at once place i.e. one DB only of each CSV. Different CSV files can be easily segregated using a tag which can state the sources of CSV.

However, each CSV files will have different types of columns and we will not be aware at first place what are the tags and fields of these columns.

So I am stuck here on how to best detect/identify tags and columns automatically for different CSV files.

I have come up with a solution like appending tag_ and field_ in each CSV column and detect that but that seems like inappropriate and not long time solution. Or different functions for each type of CSV that’s unreliable as well.

Hope anyone can help me out here and that would be wonderful

Best Regards,
Anshul

2

Hi @Anshul_choudhary ,

are you looking for a Python solution
or will you use the telegraf plugin to read csv files as explained here … csv to influxdb

best regards

3

Hi @MarcV, Yes I am looking for Python solutions as we are using multiple services to get these CSV in and push that to Influx and other tools.

I am not looking for complete solution just a method to achieve the goal and all possibilities in hand to choose the best one.

Regards,
Anshul