Migrating data from Postgresql to influxdb

influxdb

#1

Is there any way I can migrate my data from postgresql to influxdb ?


#2

@Luv The postgresql_extensible plugin might help you out. Otherwise I would advise writing a script to do it.


#3

I have just spent a full day writing a script (in Perl) to move data from a PostgreSQL table into InfluxDB. I suspect that trying to modify the postgresql_extensible plugin would take considerably longer :slight_smile:

FWIW, my script uses DBD:Pg fetchall_arrayref() to get the table rows in batches of a few thousand rows at a time, then converts each row to line protocol and uploads the batch via HTTP to InfluxDB. It even performs SELECT INTO queries to do data aggregation on each batch as it goes. (Necessary because CQs don’t do historical data … )


#4

Oh, yes @jackzampolin , that will work completely.

@JeremySTX I am not sure now that what data I need to migrate, does your script gets all the data OR we can select the database?


#5

My script pulls all data out of a specific table and adds the tags we need for our application. Table columns become field names, with some renamed to something more suitable to the current state of our application (compared to when the table was created, a few years ago).

The script is not really something that can easily be shared; my point was that the script was probably easier to write than to spend the time understanding the plugin.


#6

After having to move data from postgreSQL to InfluxDB myself, I wrote a short Python script that you can find here.