I use InfluxDB 1.8.9 on a Raspberry PI 4 with Grafana and ioBroker for my smart home.
I am writing a script to import lots of historical data into InfluxDB, to use with “influx -import …”. This script reads from various files and produces >10’000 lines of line protocol output.
Somewhere in this output there’s a syntax error. When trying to import it, Influx just tells me there’s a syntax error, but not on which line this is.
How can I validate line protocol or Influx INSERT commands?
Is there a command that just validates syntax and tells you exactly WHERE an error occurs? This would be very valuable especially for people writing code which produces line protocol.
If not on Influx 1.8.9, does this exist in Influx 2? (I cannot upgrade currently but could install it on another machine to do the validation).
It’s been a while since I setup my v2 instance but remember adding some logic that tested the http return code on each batch it tried to insert to be sure it worked.
You could add something that extends on that idea, and keeps halving the batch and tries to insert the first half until it succeeds. Eventually you’ll have the row that it doesn’t like. Bit crude I know, but has the benefit of automating this fault finding task if it ever comes up again…
(I’m not aware of any tool/command that can validate line protocol records )
Yes, I was hoping for some kind of line-by-line “dry run” option baked into Influx that would take care of that for me.
Maybe this should become a feature request … there is a “lint” tool for every programming language out there. Why not line protocol?
Bad request. The line protocol data in the request is malformed. The response body contains the first malformed line in the data. InfluxDB rejected the batch and did not write any data.
And specifically the line value
line integer
First line within sent body containing malformed data
$ influx check -file some_line_protocol_data.txt
File contains syntax errors:
-- line 205, uneven quotation: INSERT home.gas.counter value="24.38 1286953200
-- line 2958, field key missing: INSERT home.gas.counter 1287126000
-- line 7159, cannot parse to float: INSERT home.gas.counter value=143x.11f 1290582000
3 errors found.
The processing should happen completely without database access so it could also be performed on machines where only the client is installed and without an account on any InfluxDB server.
