Telegraf xml parser

Hello, can someone help with telegaf plugin to parse xml data into influx, here is sample xml data which I’m getting from my device.

What I would need to collect is:
localDn = SubNetwork=Router1,ManagedElement=node-1
measType p=“1”>VS.EthernetKBytesInRate + value 0.00
measObjLdn=“PoolId=0, PoolMember=1, Machine=Router1-interface1”

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="MeasDataCollection.xsl"?>
	<fileHeader fileFormatVersion="32.435 V15.0"
			elementType="Router,Switch" />
		<measCollec beginTime="2022-01-20T12:00:00+02:00" />
			swVersion="Router20.5.0.15" />
			<job jobId="defaultJob" />
			<granPeriod duration="PT900S" endTime="2022-01-20T12:15:00+02:00" />
			<measType p="1">VS.EthernetKBytesInRate</measType>
			<measType p="2">VS.EthernetKBytesOutRate</measType>
			<measType p="3">VS.EthernetPacketsInRate</measType>
			<measType p="4">VS.EthernetPacketsOutRate</measType>
			<measValue measObjLdn="PoolId=0, PoolMember=0, Machine=Router1-interface0, UUID=f97cb975-8b09-30cf3d51b037">
				<r p="1">0.00</r>
				<r p="2">0.00</r>
				<r p="3">0.00</r>
				<r p="4">0.00</r>
			<measValue measObjLdn="PoolId=0, PoolMember=1, Machine=Router1-interface1, UUID=10da4adc-a4bd-360cdb8342de">
				<r p="1">146.00</r>
				<r p="2">130.00</r>
				<r p="3">23.00</r>
				<r p="4">250.00</r>
		<measCollec endTime="2022-01-20T12:15:00+02:00" />

Thanks for help

How about you invest some time and try it yourself first? Read the documentation and configure the XML parser. If you get stuck, post your configuration here.

Hello, thanks for response. Trying some basics

telegraf config under /etc/telegraf/telegraf.d/
files = ["/home/user/example.xml"]
data_format = “xml”

instance = “//fileHeader/@localDn

Any idea why i’m getting error?
telegraf --once --config /etc/telegraf/telegraf.d/xml.conf
2022-01-26T12:17:19Z I! Starting Telegraf 1.21.1
2022-01-26T12:17:19Z E! [telegraf] Error running agent: Error: no outputs found, did you provide a valid config file?

You need at least one input and one output plugin configured in your telegraf config file.
For debugging purposes you can just add for example the file output plugin:

# file output only for debugging
  files = ["xml.out"]
  influx_sort_fields = true

Thanks for hint, trying to start building config, stopped here

name_override = “test”
files = ["/home/user/example.xml"]
data_format = “xml”

metric_selection = “/measCollecFile”
instance = “string(fileHeader/fileSender/@localDn)”

files = [“test.out”]
influx_sort_fields = true

but nothing is written into file.

You need at least a field in your data points.
Without a field, the data point does not exist (and so far you only mapped a tag)

I suggest to take a look at the InflxDB basic concepts here

Ok, got it

name_override = “test”
files = ["/home/user/example.xml"]
data_format = “xml”

metric_selection = "/measCollecFile"
instance = "string(/measData/managedElement/@localDn)"
measurement = "string(/measData/measInfo/measType)"
value = "number(/measData/measInfo/measValue/r)"

instance="SubNetwork=SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetKBytesInRate",value=0 1643709037000000000

how can I get all 4x “measType” objects (now I get only one) and map them with values and “measObjLdn”, it should look something like this:

instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetKBytesInRate",object="PoolId=0, PoolMember=0, Machine=Router1-interface0, UUID=f97cb975-8b09-30cf3d51b037",value=0
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetKBytesOutRate",object="PoolId=0, PoolMember=0, Machine=Router1-interface0, UUID=f97cb975-8b09-30cf3d51b037",value=0
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetPacketsInRate",object="PoolId=0, PoolMember=0, Machine=Router1-interface0, UUID=f97cb975-8b09-30cf3d51b037",value=0
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetPacketsOutRate",object="PoolId=0, PoolMember=0, Machine=Router1-interface0, UUID=f97cb975-8b09-30cf3d51b037",value=0
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetKBytesInRate",object="PoolId=0, PoolMember=1, Machine=Router1-interface1, UUID=f97cb975-8b09-30cf3d51b037",value=146
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetKBytesOutRate",object="PoolId=0, PoolMember=1, Machine=Router1-interface1, UUID=f97cb975-8b09-30cf3d51b037",value=130
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetPacketsInRate",object="PoolId=0, PoolMember=1, Machine=Router1-interface1, UUID=f97cb975-8b09-30cf3d51b037",value=23
instance="SubNetwork=Router1,ManagedElement=node-1",measurement="VS.EthernetPacketsOutRate",object="PoolId=0, PoolMember=1, Machine=Router1-interface1, UUID=f97cb975-8b09-30cf3d51b037",value=250

any suggestion how can I achieve such export ? Thanks

This works partially, however the measType is not captured right.
Quite tricky to get the right index from the parent - i think we need an xpath expert :wink:
I also added the timestamp.

  metric_selection = "/measCollecFile/measData/measInfo/measValue/r"
  timestamp = "/measCollecFile/fileFooter/measCollec/@endTime"
  timestamp_format = "2006-01-02T15:04:05-07:00"
    localDn = "string(../../../managedElement/@localDn)"
    measType = "string(../../measType)"
    measObjLdn = "string(../@measObjLdn)"
    value = "number(.)"

Thanks @Franky1 for help :slight_smile:

If I test measType with
then values is returned


My example configuration above does not work completely correctly.
I get VS.EthernetKBytesInRate for measType for all four values.

We want to read the <r> tag, but the measType is unfortunately one level higher.
I am stumped on how to write the query so to get the correct index of measType.

Anyone else has some idea what is wrong with query and how get it work?

I don’t have an exact solution to your question. I’m also not sure if it works at all with xpath. The problem is that the measType is in the parent object.

But I have an alternative solution that might be at least as good - provided that the xml payload remains as in the example. :wink:

Btw, I would limit the tags and fields to the necessary metrics. Also, I would avoid strings in fields if possible, as InfluxDB is not meant for “logging”.

  metric_selection = "/measCollecFile/measData/measInfo/measValue"
  timestamp = "/measCollecFile/fileFooter/measCollec/@endTime"
  timestamp_format = "2006-01-02T15:04:05-07:00"
    localDn = "string(../../managedElement/@localDn)"
    Machine = "string(substring-before(substring-after(@measObjLdn, 'Machine='), ', UUID='))"
    SubNetwork = "string(substring-before(substring-after(../../managedElement/@localDn, 'SubNetwork='), ',ManagedElement='))"
    ManagedElement = "string(substring-after(../../managedElement/@localDn, 'ManagedElement='))"
    measObjLdn = "string(@measObjLdn)"
    PoolId = "number(substring-before(substring-after(@measObjLdn, 'PoolId='), ', PoolMember='))"
    PoolMember = "number(substring-before(substring-after(@measObjLdn, 'PoolMember='), ', Machine='))"
    EthernetKBytesInRate = "number(r[@p='1'])"
    EthernetKBytesOutRate = "number(r[@p='2'])"
    EthernetPacketsInRate = "number(r[@p='3'])"
    EthernetPacketsOutRate = "number(r[@p='4'])"

Thanks @Franky1 seems this will not work and also this is only small part of xml file. File is quit big with many values.
I’m thinking maybe to do some processing before, for example parse xml > json and after parse json with telegraf.

If you show the whole file with real data, i might be able to give you some ideas.

I’m not sure the result will be that much easier. Then you have the same problems, only with the json parser. You go from the frying pan into the fire.

If the data is really that complex, you might be better off writing a custom inputs.execd plugin or processors.execd or processors.starlark plugin.