Migrate SQL Server data to InfluxDB

We have a database in SQL Server which I want ultimately migrate to Influxdb. The data is mainly time series.

As a starting point I want to keep both databases for few months while we get confident with Influxdb and then migrate completely.

I build the following code:

import pypyodbc 

import pandas as pd

import datetime

import time

import argparse

from influxdb import DataFrameClient

cnxn = pypyodbc.connect("Driver={ODBC Driver 17 for SQL Server};"

                        "Server=example\SQLEXPRESS;"

                        "Database=example;"

                        "uid=example;pwd=example")

cursor = cnxn.cursor()


df = pd.read_sql_query('SELECT TOP (1000) * FROM [something].[dbo].[something]', cnxn)



print(df)

client = DataFrameClient('localhost', 8086)

def convert_timestamps(timestamp):

    return int(time.mktime(datetime.datetime.strptime(timestamp, '%y%m%d%H%M%S').timetuple()))

df['dptimestamp'] = df['dptimestamp'].apply(convert_timestamps)

df['dptimestamp'] = pd.DatetimeIndex(df['dptimestamp'])

df.index = df['dptimestamp']

print(df.index)

print(type(df['dptimestamp']))

print(df)



client.write_points(df, tag_columns={'devicedescription': df['devicedescription']}, database='Occupancy_system', measurement='dpvalue',time_precision='s')

I am using the pypyodbc in combination with pandas to query the data into a dataframe and them trying to save to Influxdb using client.write_points.

Overall I cant make to work and searching online I can see the influx dataframe method is not stable to write back to Influx.

the error I get with the above code is:

 raise InfluxDBClientError(err_msg, response.status_code)
influxdb.exceptions.InfluxDBClientError: 400: {"error":"unable to parse following by the dataframe

open question:

is there a telegraph plugin to query SQL database and write the data?
is there any other way to achieve my task to your experience

Influxdb version 1.8

Thanks alot

Hello @Torounia,
Thanks for your question!

Have you considered using the SQL Flux package?

Hi @Anaisdg ,

I’ve the same need of migrating huge data from SQL server to influxdb, is there any plugin available for this? I went through flux but I was unable to understand the following, can you please clarify

  1. Using Influx CLI for flux seems not so smooth, other than chronograf are there anyother IED/tools to use flux?
  2. Is Migration of data (I see the Join query) from MSSQL to influxdb possible with flux, can you please provide some examples.

How much data is a huge amount?
I’m looking into this and I’ll get back to you as soon as I can.

Hi @Anaisdg,

SQL server- ~ 2 years of 1-minute data (2 * 365 * `1440) rows, 30+colums in each table, approx. 200+ tables.

Is there any plugin in discussion/development that will ease the migration process ? Or as previously asked can we handle data migration in flux, can you please share some examples.