Type conflict: how to find the offending record

InfluxDB 1
1

I am getting the following error:
Caught exception: HTTP Code 400 {“error”:“partial write: field type conflict: input field "foo" on measurement "logs" is type integer, already exists as type float dropped=2”}

But i am very certain this field can only be of type int, not float. and the code has been running for months without issue.

“show field keys from data.logs” does show the field with 2 types: integer and float.

I run the following PHP code to search for float values, but it didnt find any:
$query = “select devID, foo FROM data.logs”;
$result = $influxdb->query($query);
$points = $result->getPoints();
foreach($points as $point) {
if (!is_int($point[‘foo’])) {
var_dump($point); echo “\n”;
echo “float!!!\n”;
}
}

There are other retention policies, but this measurement doesnt exist in any of the others.

Is there another way to find the record with the float?

Thanks!

2

You can use the InfluxInspect utility
The conflict exists in the current shard and may exist in the next one too depending on the first point it receives…

That means it received something like
_measurement_ value=2i _time_ instead of _measurement_ value=2 _time_ (float is the default, integer must be explicitly specified by appending an “i” to the value

1 Like
3

Nifty utility. and a good reason to upgrade to 1.11.

It indeed reports the conflict, but doesnt help me in resolving it.

There isnt much data with this value in the current shard, and that data looks super clean. its supposed to be an integer that always decreases (its negative), and that looks all good.

The lineprotocol data going into the database right now (the ones that fail) have the i at the end as well.

I tried using the commandline tool with -json, but there even floats like 10.0 are printed as 10, so cant use that method to see if a value is a float.

i could export all records that have this field, delete them and re-import them with a forced data type, but i would really like to see some proof first that the data is actually a float

Remco

1 Like
4

I don’t remember the output of the utility, but that whole shard expects that field as float (which prevents inserting integers), the only solution is to drop all the relevant points in the shard.
Do you have the timestamps of that shard begin and end?

Always via that utility, you can export raw data in line protocol, which should produce the evidence you are looking for

1 Like
5

Ah yes, in the output of ‘influx_inspect export’ does indicate if its an integer or float.

I have exported and deleted all offending data, and “SELECT foo FROM data.logs” returns nothing. but “show field keys” still shows the key, with 2 types. i guess now waiting for a compacting to happen and so the keys are removed

Thanks for the help.

Remco

6

We are now in the next chunk, and new data is beeing inserted.

But “show field keys on x from data.logs” still shows the record as beeing both float and integer.
And “select foo from data.logs” (without time range) only shows data in the new chunk.
But still cant insert the old data.

Would i need to rebuild the TSI ?

Remco

7

Are you sure there aren’t other data with the wrong datatype in that shard? Is that field key used in any other measurement?

Once all the float values in that field have been dropped the situation should be clean…

8

The issue is now not the current shard, but the previous one. The server config is default, so compacting should run when no new data is written to the shard for 4 hours.

I have checked other measurements and retention policies, this field is not used anywhere else.

and “show field keys on x from data.logs” should operates only on 1 measurement, in 1 database, for 1 retention policy if i understand it correctly.

Its a pitty that ‘influx_inspect check-schema’ doesnt show in which sharts each type occures.

9

@Giovanni_Luisotto Thanks so much for your help here and thank you @alflexRH for your questions.