Problem with telegraf.conf parsing an XML file from a web page

Telegraf
1

I am just starting out with the TIG stack on a pi3B. I am trying to read temperature values from an xml file and keep getting error messages and telegraf failing to start. I have read through the “parsing XML” guides and suspect I have still made an error with the inputs section of the conf file. I used the “journalctl -fu telegraf” command which stated -
“key `data_format’ is in conflict with line b21d”
I do not understand where b21d is - perhaps in the telegraf executable?

my xml file is -

?xml version=“1.0” encoding=“utf-8”?>

<?xml-stylesheet type="text/xsl" href="values.xsl"?>

<val:Root xmlns:val=“http://www.etech.cz/XMLSchema/poseidon/values.xsd”>

3.1.7
2.36
France2










13
0
00:0A:59:00:B4:28
2287663

Poseidon model 3266 http://www.hw-group.com/ 10 18 1 Binary 1 0 2 0 0 0 0 2 Binary 2 0 2 0 0 0 0 3 Binary 3 0 2 0 0 0 0 4 Binary 4 0 2 0 0 0 0 13079 Box Room Pipe C 13.1 0.0 1.0 60.0 5.0 0 1 0 0 11642 Box Room Air C 14.3 0.0 0.0 60.0 5.0 0 1 0 0 62858 Hall Air C 14.2 0.0 0.0 60.0 5.0 0 1 0 0

I think the problem is in the inputs section of my telegraf.conf file -

[[inputs.http]]

urls = [“http://peyr.dyndns.org:2905/values.xml”]

data_format = “xml”

Drop url and hostname from list of tags

tagexclude = [“url”, “host”]

Data format to consume.

Each data format has its own unique set of configuration options, read

more about them here:

telegraf/DATA_FORMATS_INPUT.md at master · influxdata/telegraf · GitHub

data_format = “xml”

Multiple parsing sections are allowed

[[inputs.http.xml]]
## Optional: XPath-query to select a subset of nodes from the XML document.
metric_selection = “//SenSet”

## Optional: XPath-query to set the metric (measurement) name.
metric_name = "string('Creysse')"

## Optional: Query to extract metric timestamp.
## If not specified the time of execution is used.
#timestamp = "/Gateway/Timestamp"
## Optional: Format of the timestamp determined by the query above.
## This can be any of "unix", "unix_ms", "unix_us", "unix_ns" or a valid Golang
## time format. If not specified, a "unix" timestamp (in seconds) is expected.
#timestamp_format = "2006-01-02T15:04:05Z"

## Tag definitions using the given XPath queries.
[inputs.http.xml.tags]
  name   = "string(Entry/Name)"

## Integer field definitions using XPath queries.
[inputs.http.xml.fields_int]

## Non-integer field definitions using XPath queries.
## The field type is defined using XPath expressions such as number(), boolean() or string(). If no conversion is performed the field will be of type string.
[inputs.http.xml.fields]
  temperature = "number(Entry/Value)"

I would be grateful for any assistance in identifying my mistakes here.

Thank you

DoctorD

2

Welcome to the community.
Some quick remarks:

  1. The example data does not look like a valid XML file.
  2. Your post is a bit messed up. Please embed all code snippets in markdown format, so that it will be rendered correctly here in the forum and thus be displayed legibly, see example below.
```xml
put your xml file snippet here
```

```toml
put you Telegraf config code snippet here
```
3

Franky1

Thanks I realised something was not quite right.
Have reposted the code snippets below with your markdown advice and they now look as I expected.
D

my xml file is -


?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="values.xsl"?>
<val:Root xmlns:val="http://www.etech.cz/XMLSchema/poseidon/values.xsd">
<Agent>
<Version>3.1.7</Version>
<XmlVer>2.36</XmlVer>
<DeviceName>France2</DeviceName>
<Features>
<Wire1/>
<BinaryIn/>
<DHCP/>
<SNTP/>
<SNMP/>
<SMTP/>
<Modbus/>
<Telnet/>
</Features>
<Model>13</Model>
<VendorID>0</VendorID>
<MAC>00:0A:59:00:B4:28</MAC>
<Uptime>2287663</Uptime>
<Title>Poseidon model 3266</Title>
<Contact>http://www.hw-group.com/</Contact>
</Agent>
<TemperRange>
<Min>10</Min>
<Max>18</Max>
</TemperRange>
<BinaryInSet>
<Entry>
<ID>1</ID>
<Name>Binary 1</Name>
<Value>0</Value>
<Alarm>2</Alarm>
<Delay>0</Delay>
<State>0</State>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>0</EmailSMS>
</Entry>
<Entry>
<ID>2</ID>
<Name>Binary 2</Name>
<Value>0</Value>
<Alarm>2</Alarm>
<Delay>0</Delay>
<State>0</State>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>0</EmailSMS>
</Entry>
<Entry>
<ID>3</ID>
<Name>Binary 3</Name>
<Value>0</Value>
<Alarm>2</Alarm>
<Delay>0</Delay>
<State>0</State>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>0</EmailSMS>
</Entry>
<Entry>
<ID>4</ID>
<Name>Binary 4</Name>
<Value>0</Value>
<Alarm>2</Alarm>
<Delay>0</Delay>
<State>0</State>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>0</EmailSMS>
</Entry>
</BinaryInSet>
<SenSet>
<Entry>
<ID>13079</ID>
<Name>Box Room Pipe</Name>
<Units>C</Units>
<Value>13.1</Value>
<Calib>0.0</Calib>
<Min>1.0</Min>
<Max>60.0</Max>
<Hyst>5.0</Hyst>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>1</EmailSMS>
<Delay>0</Delay>
<State>0</State>
</Entry>
<Entry>
<ID>11642</ID>
<Name>Box Room Air</Name>
<Units>C</Units>
<Value>14.3</Value>
<Calib>0.0</Calib>
<Min>0.0</Min>
<Max>60.0</Max>
<Hyst>5.0</Hyst>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>1</EmailSMS>
<Delay>0</Delay>
<State>0</State>
</Entry>
<Entry>
<ID>62858</ID>
<Name>Hall Air</Name>
<Units>C</Units>
<Value>14.2</Value>
<Calib>0.0</Calib>
<Min>0.0</Min>
<Max>60.0</Max>
<Hyst>5.0</Hyst>
<SNMPTrap>0</SNMPTrap>
<EmailSMS>1</EmailSMS>
<Delay>0</Delay>
<State>0</State>
</Entry>
</SenSet>
</val:Root>

I think the problem is in the inputs section of my telegraf.conf file -


[[inputs.http]]
  
 urls = ["http://peyr.dyndns.org:2905/values.xml"]

  data_format = "xml"
  ## Drop url and hostname from list of tags
  tagexclude = ["url", "host"]

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "xml"

  ## Multiple parsing sections are allowed
  [[inputs.http.xml]]
    ## Optional: XPath-query to select a subset of nodes from the XML document.
    metric_selection = "//SenSet"

    ## Optional: XPath-query to set the metric (measurement) name.
    metric_name = "string('Creysse')"

    ## Optional: Query to extract metric timestamp.
    ## If not specified the time of execution is used.
    #timestamp = "/Gateway/Timestamp"
    ## Optional: Format of the timestamp determined by the query above.
    ## This can be any of "unix", "unix_ms", "unix_us", "unix_ns" or a valid Golang
    ## time format. If not specified, a "unix" timestamp (in seconds) is expected.
    #timestamp_format = "2006-01-02T15:04:05Z"

    ## Tag definitions using the given XPath queries.
    [inputs.http.xml.tags]
      name   = "string(Entry/Name)"

    ## Integer field definitions using XPath queries.
    [inputs.http.xml.fields_int]

    ## Non-integer field definitions using XPath queries.
    ## The field type is defined using XPath expressions such as number(), boolean() or string(). If no conversion is performed the field will be of type string.
    [inputs.http.xml.fields]
      temperature = "number(Entry/Value)"

I would be grateful for any assistance in identifying my mistakes here.

Thanks you

DoctorD

4

Thanks for formatting your config file. Can you provide the full output from telegraf as well? You mentioned part of it, but it would be good to see the full thing?

Thanks!

5

Thanks

Do you mean the full telegraf.conf file ?
There is not much “output” as telegraf fails to start properly – or do you mean the response to the status command in the terminal?

Thanks

David

6

Yes, having your full telegraf.conf file will help debug what is preventing telegraf from starting in the first place.

7

Thanks - the system says the file is too large - I exceed the body limit - is there another way to post it ?

I used the toml bracketing

D

8

First, what is the error message when Telegraf does not start at all?
I suspect a syntax error in the conf file.

9

If the telegraf.conf file is too large to paste it here in the forum you could use something like Pastebin or Github Gist or similar.

10

here is the output from the status command after I start telegraf

Will look into pastebin

D

sudo systemctl status telegraf
telegraf.service - The plugin-driven server agent for reporting metrics into InfluxDB
Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2021-11-04 15:23:36 GMT; 4min 56s ago
Docs: GitHub - influxdata/telegraf: The plugin-driven server agent for collecting & reporting metrics.
Process: 5020 ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d $TELEGRAF_OPTS (code=exited, status=1/FAILURE)
Main PID: 5020 (code=exited, status=1/FAILURE)

Nov 04 15:23:36 KODI systemd[1]: telegraf.service: Service RestartSec=100ms expired, scheduling restart.
Nov 04 15:23:36 KODI systemd[1]: telegraf.service: Scheduled restart job, restart counter is at 5.
Nov 04 15:23:36 KODI systemd[1]: Stopped The plugin-driven server agent for reporting metrics into InfluxDB.
Nov 04 15:23:36 KODI systemd[1]: telegraf.service: Start request repeated too quickly.
Nov 04 15:23:36 KODI systemd[1]: telegraf.service: Failed with result ‘exit-code’.
Nov 04 15:23:36 KODI systemd[1]: Failed to start The plugin-driven server agent for reporting metrics into InfluxDB.

11

I would start telegraf manually (instead of systemd) to maybe get more valuable information.

12

Sorry to sound dim here but I used (as in the telegraf docs)
sudo systemctl start telegraf

followed by
sudo systemctl status telegraf

what is the best method for a manual start?

D

13

We only remove one layer of complexity and try to simplify the debugging :wink:
Therefore, just run telegraf on the console (without systemd) and watch the output.

/usr/bin/telegraf --debug --test --config /etc/telegraf/telegraf.conf

I suspect a bug/typo in your config file, because telegraf seems to crash right at startup?

14

What just happened to catch my eye, you have two times the line

data_format = "xml"

in your config file, that could be the problem?
Remove one and try again.

15

Franky1

Thanks - you’re correct - telegraf runs with the additional line removed. I can see data in influxdb now.
An annoying simple mistake - as often the case!

It has raised another question for me which is how to make telegraf read all three temperature outputs rather than just the first set. Do I have to specify each element exactly in separate lines of the config?

Many thanks for your time and effort in finding my mistake.

D

16

Make the following adjustments in the config and it should work:

metric_selection = "//SenSet/Entry"
  # ....
  name = "string(/Name)"
  # ....
  temperature = "number(/Value)"
17

Thank you - it works - I am learning…

D

18

Franky1

One more question if I may.
If I want read in another xml file from a different temp monitor device - with different tags etc - but to the same influx database do I add another section of [[inputs.http]] to the existing telegraf.conf file - or should I create another conf file ? - or (seems unlikely) do i try and put it all in the same [[inputs.http]] section as I have started already ??

Thanks

D

19

Yes you can add more than one input section [[inputs.http]] to your telegraf config file.