Inputs.http to parse xml creates panic

Hello everyone,

I try to process an XML response via http from openhab-REST-API to InfluxDB version 2.1.1, but the telegraf service restarts with following error message:

panic: runtime error: invalid memory address or nil pointer dereference

Here is the full output:

Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! Starting Telegraf 1.21.3
Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! Loaded inputs: http
Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! Loaded aggregators:
Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! Loaded processors:
Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! Loaded outputs: influxdb_v2
Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! Tags enabled:
Feb 09 22:36:00 hostname.lan telegraf[2172922]: 2022-02-09T21:36:00Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:10s
Feb 09 22:36:10 hostname.lan telegraf[2172922]: panic: runtime error: invalid memory address or nil pointer dereference
Feb 09 22:36:10 hostname.lan telegraf[2172922]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x94622a]
Feb 09 22:36:10 hostname.lan telegraf[2172922]: goroutine 42 [running]:
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/antchfx/xpath.eqFunc({0x5835860, 0xc00062f520}, {0x42f49a0, 0x831bd20}, {0x42fa9e0, 0xc00100a330})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/pkg/mod/github.com/antchfx/xpath@v1.2.0/operator.go:220 +0x8a
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/antchfx/xpath.(*logicalQuery).Evaluate(0xc001000b70, {0x5835860, 0xc00062f520})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/pkg/mod/github.com/antchfx/xpath@v1.2.0/query.go:724 +0x91
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/antchfx/xpath.(*logicalQuery).Select(0xc000ea1928, {0x5835860, 0xc00062f520})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/pkg/mod/github.com/antchfx/xpath@v1.2.0/query.go:711 +0x57
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/antchfx/xpath.(*NodeIterator).MoveNext(0xc00062f520)
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/pkg/mod/github.com/antchfx/xpath@v1.2.0/xpath.go:86 +0x38
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/antchfx/xmlquery.QuerySelectorAll(0xc000b28d80, 0xc00062f460)
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/pkg/mod/github.com/antchfx/xmlquery@v1.3.9/query.go:116 +0x13a
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/antchfx/xmlquery.QueryAll(0xc000ea1a18, {0xc000d3f0c1, 0x1457f})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/pkg/mod/github.com/antchfx/xmlquery@v1.3.9/query.go:98 +0x52
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/influxdata/telegraf/plugins/parsers/xpath.(*xmlDocument).QueryAll(0xc0006f9c80, {0x48daca0, 0xc000b28d80}, {0xc000d3f0c1, 0xc0006f9cd0})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/src/github.com/influxdata/telegraf/plugins/parsers/xpath/xml_document.go:18 +0x45
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/influxdata/telegraf/plugins/parsers/xpath.(*Parser).Parse(0xc000b28a00, {0xc000f12000, 0x1457f, 0x18000})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/src/github.com/influxdata/telegraf/plugins/parsers/xpath/parser.go:96 +0x2b1
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/influxdata/telegraf/plugins/inputs/http.(*HTTP).gatherURL(0xc000632000, {0x5960588, 0xc0000ad620}, {0xc000d3f001, 0x34})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/src/github.com/influxdata/telegraf/plugins/inputs/http/http.go:235 +0x80d
Feb 09 22:36:10 hostname.lan telegraf[2172922]: github.com/influxdata/telegraf/plugins/inputs/http.(*HTTP).Gather.func1({0xc000d3f001, 0x34})
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/src/github.com/influxdata/telegraf/plugins/inputs/http/http.go:144 +0x9e
Feb 09 22:36:10 hostname.lan telegraf[2172922]: created by github.com/influxdata/telegraf/plugins/inputs/http.(*HTTP).Gather
Feb 09 22:36:10 hostname.lan telegraf[2172922]:         /go/src/github.com/influxdata/telegraf/plugins/inputs/http/http.go:142 +0x188
Feb 09 22:36:10 hostname.lan systemd[1]: telegraf.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Feb 09 22:36:10 hostname.lan systemd[1]: telegraf.service: Failed with result 'exit-code'.
Feb 09 22:36:10 hostname.lan systemd[1]: telegraf.service: Service RestartSec=100ms expired, scheduling restart.
Feb 09 22:36:10 hostname.lan systemd[1]: telegraf.service: Scheduled restart job, restart counter is at 24.
Feb 09 22:36:10 hostname.lan systemd[1]: Stopped The plugin-driven server agent for reporting metrics into InfluxDB.

Here you can see my config file as well as the XML-hierarchy I want to process:

[global_tags]

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  debug = false
  quiet = false
  omit_hostname = true

[[outputs.influxdb_v2]]
  urls = ["http://hostname.lan:8086"]
  token = "..."
  organization = "Home"
  bucket = "SmartHome"

[[inputs.http]]
  urls = [
    "http://openhab.lan/addons/xmlapi/statelist.cgi"
  ]

  data_format = "xml"
  tagexclude = ["url", "host"]

  [[inputs.http.xml]]
  
# <stateList>
#   ...
#   <device name="Hygrometer_LR" ise_id="1555" unreach="false" sticky_unreach="false" config_pending="false">
#     <channel name="Hygrometer_LR:0" ise_id="1556" index="0" visible="" operate="">
#     ...
#     </channel>
#     <channel name="HM-WDS40-TH-I-2 OEQ0161025:1" ise_id="1576" index="1" visible="true" operate="true">
#       <datapoint name="BidCos-RF.OEQ0161025:1.TEMPERATURE" type="TEMPERATURE" ise_id="1578" value="21.700000" valuetype="4" valueunit="°C" timestamp="1644441533" operations="5"/>
#       <datapoint name="BidCos-RF.OEQ0161025:1.HUMIDITY" type="HUMIDITY" ise_id="1577" value="65" valuetype="16" valueunit="%" timestamp="1644441533" operations="5"/>
#     </channel>
#   </device>
#   ...
# </stateList>

  metric_name = "'hygrometer'"
  metric_selection = "/stateList/device/@name=Hygrometer_LR/channel/@ise_id=1576"
  timestamp = "datapoint/@type=TEMPERATURE/@timestamp"
  timestamp_format = "unix"

  [inputs.http.xml.tags]
    room = "LR"

  [inputs.http.xml.fields]
    humidity = "number(datapoint/@type=HUMIDITY/@value)"
    temperature = "number(datapoint/@type=TEMPERATURE/@value)"

Both InfluxDB and Telegraf are running as services on CentOS 8.

And the file information from the URL:

# file /tmp/statelist.cgi 
/tmp/statelist.cgi: XML 1.0 document, ISO-8859 text, with very long lines, with no line terminators
# curl -v http://openhab.lan/addons/xmlapi/statelist.cgi
...
< HTTP/1.1 200 OK
< Content-Type: text/xml
< Access-Control-Allow-Origin: *
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Robots-Tag: none
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: no-referrer
< Cache-Control: private, no-cache, must-revalidate, no-transform, max-age=0
< Transfer-Encoding: chunked
< Date: Wed, 09 Feb 2022 23:22:58 GMT
< 
<?xml version="1.0" encoding="ISO-8859-1" ?><stateList><device name=...

EDIT 1: I found the configuration issue with the double brackets.
EDIT 2: I think it has to do with the referencing to attributes to get the correct data:

humidity = "number(datapoint/@type=HUMIDITY/@value)"
temperature = "number(datapoint/@type=TEMPERATURE/@value)"

Try this xpath config:

[[inputs.http.xpath]]
  metric_name = "'hygrometer'"
  metric_selection = "/stateList/device[@name='Hygrometer_LR']/channel[@ise_id='1576']"
  timestamp = "datapoint[@type='TEMPERATURE']/@timestamp"
  timestamp_format = "unix"
  [inputs.http.xpath.tags]
    room = "string(../@name)"
  [inputs.http.xpath.fields]
    humidity = "number(datapoint[@type='HUMIDITY']/@value)"
    temperature = "number(datapoint[@type='TEMPERATURE']/@value)"

Thanks - that did it!

I solved it via bash and xpath before telegraf:

wget -qO- http://openhab.lan/addons/xmlapi/statelist.cgi | xmllint --xpath 'string(/stateList/device[@name="Heating_BR"]/channel[@name="HM-CC-RT-DN LEQ0863016:4"]/datapoint[@name="BidCos-RF.LEQ0863016:4.ACTUAL_TEMPERATURE"]/@value)' -

I tried this, but there were syntax error. So I started from scratch. If I only had knewn that the annotation was wrong :wink: