Help with `ERR: partial write: field type conflict` error

InfluxDB 1
,
1

I’m having problems with the restore procedure described in restore data to an existing database.

influx --version
InfluxDB shell version: 1.8.10

I need to restore data from a temporary database efd-1015 to an existing database, but I get the following error:

> SELECT * INTO "efd".autogen.:MEASUREMENT FROM "efd-s1015".autogen./.*/ GROUP BY *
ERR: partial write: field type conflict: input field "position" on measurement "lsst.sal.ATPneumatics.logevent_m1VentsPosition" is type float, already exists as type integer dropped=2

Note that the field position is an integer in both databases, and that’s why the error message doesn’t make sense to me.

> SHOW FIELD KEYS on "efd-s1015" FROM "lsst.sal.ATPneumatics.logevent_m1VentsPosition"
name: lsst.sal.ATPneumatics.logevent_m1VentsPosition
fieldKey           fieldType
--------           ---------
position           integer
priority           integer
private_efdStamp   float
private_identity   string
private_kafkaStamp float
private_origin     integer
private_rcvStamp   float
private_revCode    string
private_seqNum     integer
private_sndStamp   float

and

> SHOW FIELD KEYS on "efd" FROM "lsst.sal.ATPneumatics.logevent_m1VentsPosition"
name: lsst.sal.ATPneumatics.logevent_m1VentsPosition
fieldKey           fieldType
--------           ---------
position           integer
priority           integer
private_efdStamp   float
private_host       integer
private_identity   string
private_kafkaStamp float
private_origin     integer
private_rcvStamp   float
private_revCode    string
private_seqNum     integer
private_sndStamp   float

Trying to investigate the error I decided to restore the data to a new (empty) database:

> CREATE DATABASE test
> SELECT * INTO "test"."autogen".:MEASUREMENT FROM "efd-s1015".autogen./.*/ GROUP BY * limit 1
name: result
time written
---- -------
0    881

For my surprise the query above is not preserving the original data type for the field position in the lsst.sal.ATPneumatics.logevent_m1VentsPosition measurement, while it seems to be preserving the data types for the other fields.

> SHOW FIELD KEYS on "test" from "lsst.sal.ATPneumatics.logevent_m1VentsPosition"
name: lsst.sal.ATPneumatics.logevent_m1VentsPosition
fieldKey           fieldType
--------           ---------
position           float
priority           integer
private_efdStamp   float
private_identity   string
private_kafkaStamp float
private_origin     integer
private_rcvStamp   float
private_revCode    string
private_seqNum     integer
private_sndStamp   float

The fact that the field position is created with type float is in agreement with the initial error. However, If I specify the measurement name in the query the original data type is preserved:

> DROP DATABASE test
> CREATE DATABASE test
> SELECT * INTO "test"."autogen"."lsst.sal.ATPneumatics.logevent_m1VentsPosition" FROM "efd-s1015"."autogen"."lsst.sal.ATPneumatics.logevent_m1VentsPosition" GROUP BY * limit 1
name: result
time written
---- -------
0    1

> SHOW FIELD KEYS on "test" from "lsst.sal.ATPneumatics.logevent_m1VentsPosition"
name: lsst.sal.ATPneumatics.logevent_m1VentsPosition
fieldKey           fieldType
--------           ---------
position           integer
priority           integer
private_efdStamp   float
private_identity   string
private_kafkaStamp float
private_origin     integer
private_rcvStamp   float
private_revCode    string
private_seqNum     integer
private_sndStamp   float

I’m wondering if there’s a bug with the query that iterates over all the measurements, because the field positon exists on different measurements as integer or float - and the underlying code might be assuming the data type of the last occurrence of the field position or something like that.

> SELECT position FROM "efd-s1015"."autogen"./.*/ GROUP BY * limit 1
name: lsst.sal.ATPneumatics.logevent_m1VentsPosition
time                position
----                --------
1655221053999000000 2

name: lsst.sal.MTMount.logevent_cameraCableWrapTarget
time                position
----                --------
1655480976542000000 -0.0008915923758653858

name: lsst.sal.MTRotator.command_move
time                position
----                --------
1655476430492000000 0

name: lsst.sal.MTRotator.logevent_target
time                position
----                --------
1655480976484000000 0.09910850419126974

> SHOW FIELD KEYS on "efd-s1015" FROM "lsst.sal.ATPneumatics.logevent_m1VentsPosition"
name: lsst.sal.ATPneumatics.logevent_m1VentsPosition
fieldKey           fieldType
--------           ---------
position           integer

> SHOW FIELD KEYS on "efd-s1015" FROM "lsst.sal.MTRotator.logevent_target"
name: lsst.sal.MTRotator.logevent_target
fieldKey           fieldType
--------           ---------
position           float
2

Hello @afausti,
What version of InfluxDB are you using?
It definitely feels like a bug. I encourage you to submit an issue here:

I’ll try to loop in an engineer as well. Thank you!

3

Thanks @Anaisdg!

I’m using InfluxDB 1.8.10, I’ll go ahead and open an issue.

4

See issue #23503.

I’ve also outlined a procedure to get around this bug which is to run an async query for each measurement to copy data from one database to another. It turns out to be really fast as InfluxDB can write over million points per minute.

1 Like