InfluxDB 2.4 - Access migrated data from 1.6

InfluxDB 2
1

Hi everybody,

for a long time I have been desperately trying to access my data migrated from InfluxDB 1.6.7~rc0 from the Raspberry into a Docker 2.4. I managed to copy and migrate the data from /var/lib/influxdb/ from the Raspberry to the Docker.

Using influx v1 shell I can access my data from 1.6 and list it select * from mqtt.0.esp01.sensor.temperature.state. However, from the shell I cannot access the data saved with 2.4. With the Data Explorer I can access the data saved with 2.4, but not the data from 1.6.

Can anyone help me how to access my old and new data in Docker? Some little thing is missing, but for weeks I can’t figure it out.

2

Hi @MacPoInfluxData

Does your 1.6.7 data use InfluxQL, and your 2.4 data use Flux? If yes, have you considered just migrating all of the data into the new (Flux) format? I did that using Node-RED. See here.

3

Hi @grant1
this could be the reason. Do you have another idea to migrate the data without Node-RED?
Otherwise I will try to export the date with another Docker 1.8 as the current installation does not provide influx_inspect.

4

@MacPoInfluxData

I think it’s fair to assume that the 1.6.7 data uses InfluxQL. For your 2.4 data, do you view it at https://your_ip_addr:8086 and see this?

image
image1400×833 54.8 KB

FYI, the main reason I used Node-RED to the data transfer was because my old (v1.8) data was horribly misnamed, mistagged, etc. and I educated myself enough about Flux to come up with a more logical schema. So when I moved the data from v1.8 to v2.3, I added/removed/fixed tags, fieldnames, etc. It was actually quite easy.

5

@grant1
Yes, that is the new view with the data explorer inside.

I use ioBroker and I also thinking about to renaming and restructuring of the data. Perhaps it is the best way to migrate the data reading old fields and values and store them in the new way.

Thanks for the hint. I will add my solution when it is implemented and working.

6

Here is my function which I used to migrate the data with ioBroker.

/**
* Migriert Serien von der Instanz influx.0 (1.x) in die Instanz influx.1 (2.x)
* @param {string} migMessung           Objekt Id für Messung (ehemals Serie)
* @param {string} migJavascriptObjekt  Objekt Id in ioBroker
* @param {string} migWhere             Bedingungen für Datensätze
*/
function migriere(migMessung, migJavascriptObjekt, migWhere) {
    var messung = migMessung.trim();
    var javascriptObjekt = migJavascriptObjekt.trim();
    var where = migWhere.trim();

    sendTo('influxdb.0', 'query', 'SELECT * FROM "' + javascriptObjekt + '" where ' + where, function (ergebnisDb) {
        if (ergebnisDb.error) { 
            console.error(ergebnisDb.error);
        } else {
            var dbAck = true;
            var dbFrom = "system.adapter.abc.0";
            var dbQ = 0;
            var dbValue = 16.5;
            var dbTs = 1615981010728;
            var dbAckNeu = true;
            var dbFromNeu = "system.adapter.abc.0";
            var dbQNeu = 0;
            var dbValueNeu = 16.5;
            var dbTsNeu = 1615981010728;

            var zaehlerGelesen = Object.keys(ergebnisDb.result[0]).length;
            var zaehlerEingefuegt = 0;
            var zaehlerIgnoriert = 0;

            for (var index = 0; index < zaehlerGelesen; ++index) {
                dbAckNeu =   ergebnisDb.result[0][index]["ack"];
                dbFromNeu =  ergebnisDb.result[0][index]["from"];
                dbQNeu =     ergebnisDb.result[0][index]["q"];
                dbValueNeu = ergebnisDb.result[0][index]["value"];
                dbTsNeu =    ergebnisDb.result[0][index]["ts"];

                if (dbAckNeu == dbAck && dbFromNeu == dbFrom && dbQNeu == dbQ && dbValueNeu == dbValue) {
                    zaehlerIgnoriert += 1;
                } else {
                    dbAck =   dbAckNeu;
                    dbFrom =  dbFromNeu;
                    dbQ =     dbQNeu;
                    dbValue = dbValueNeu;
                    dbTs =    dbTsNeu;
                    zaehlerEingefuegt += 1;
            
                    sendTo('influxdb.1', 'storeState', { 
                        id: 'Test 1', state: [{ts: dbTs, ack: dbAck, from: dbFrom, q: dbQ, val: dbValue}]
                    });
                }
            }
        }
    });
}