# Telegraf xml protobuf doubt

**URL:** <https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985>\
**Category:** Telegraf\
**Tags:** xml, protobuf\
**Created:** [February 23, 2022, 11:14am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985 "2022-02-23T11:14:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [February 23, 2022, 11:14am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/1 "2022-02-23T11:14:40Z")

</div>

We are getting protobuf message, which we are trying to read using telegraf. Few errors we are getting:

a. what value should be there for xpath\_protobuf\_type. I am trying some options but getting error “Proto” not found.

b. How should i define the tags and fields, since the example given here refers to xml format and cannot find anything for protobuf. [telegraf/plugins/parsers/xpath at master · influxdata/telegraf · GitHub](https://github.com/influxdata/telegraf/tree/master/plugins/parsers/xpath#protocol-buffers-additional-settings)

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [February 28, 2022, 3:04pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/2 "2022-02-28T15:04:24Z")

</div>

Hi,

> [@Pratik\_Das\_Baghel](#):
>
> a. what value should be there for xpath\_protobuf\_type. I am trying some options but getting error “Proto” not found.

Do you know what protobuf type are you using? Do you have a protobuf definition file?

Thanks!

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 2, 2022, 5:38am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/3 "2022-03-02T05:38:01Z")

</div>

```auto
syntax = "proto3";
option java_package = "com.ruckus.cloud.lte.sciproxy.gpb";
option java_outer_classname = "SciMessageProtocol";

```

We are not sure of protobuf type. Is this information enough for now?

Thanks

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 2, 2022, 2:03pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/4 "2022-03-02T14:03:06Z")

</div>

I am not familiar with that protobuf. However, you will need to provide the definition file and the type. Once those are resolved you can move on to creating tags and fields with the XPath parser.

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 2, 2022, 2:10pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/5 "2022-03-02T14:10:41Z")

</div>

Is there a defined set of protobuf types? Because i defined the proto file and provided protobuf type but still was getting same error proto not found.

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 2, 2022, 3:01pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/6 "2022-03-02T15:01:09Z")

</div>

no, can provide the exact error message, please?

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 3, 2022, 12:17pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/7 "2022-03-03T12:17:52Z")

</div>

Hi. Below are the screenshots:

 ![Screenshot (92)](https://us1.discourse-cdn.com/flex023/uploads/influxdata/original/2X/9/959f5f51b8ddcc817b362581927c4bba120394b2.png)  
 ![Screenshot (93)](https://us1.discourse-cdn.com/flex023/uploads/influxdata/original/2X/6/665097919cd7db885fbe6ca72e44615827925ba3.png)

Thanks

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 3, 2022, 2:21pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/8 "2022-03-03T14:21:26Z")

</div>

ok and what does your configuration file look like?

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 4, 2022, 6:45am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/9 "2022-03-04T06:45:03Z")

</div>

```auto
[[inputs.mqtt_consumer]]
   servers = ["aws_mq_broker_provided"]
   qos = 0
   topics = [
    "Topic_name_provided"
  ]
  client_id = "telegraf"
  username = "######"
  password = "####"

  data_format = "xpath_protobuf"
  xpath_protobuf_type = "com.ruckuswireless.smallcell.sciproxy.gpb"
  xpath_protobuf_file = "proto_file_location_provided"
  xpath_print_document = true

```

And proto-file is:

```auto
syntax = "proto3";
option java_package = "com.ruckuswireless.smallcell.sciproxy.gpb";

message Ap {

    double dl = 1;
    double ul = 2;    
}

message ApNon{

	uint32 StartTime = 1;
    uint64 SampleDurationSec = 2;

    repeated Ap ap = 10;
}

```

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 4, 2022, 2:05pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/10 "2022-03-04T14:05:22Z")

</div>

1. In your .proto, you need to specify the Go-specific `package` directive to correctly name your custom proto buffer. Something like this:

```nohighlight
package myproto;

```

1. In your Telegraf config the `xpath_protobuf_type` value should be set to the class + message type. Using the package name above of `myproto` + your `Ap` message type would require:

```nohighlight
xpath_protobuf_type = "myproto.Ap"

```

I used this protobuf file:

```nohighlight
syntax = "proto3";
package myproto;

message Ap {
    double dl = 1;
    double ul = 2;
}

message ApNon{
	uint32 StartTime = 1;
    uint64 SampleDurationSec = 2;

    repeated Ap ap = 10;
}

```

with this config:

```nohighlight
[agent]
[[outputs.file]]
[[inputs.file]]
  files = ["test.data"]
  data_format = "xpath_protobuf"
  xpath_protobuf_type = "myproto.Ap"
  xpath_protobuf_file = "my.proto"
  xpath_print_document = true

```

And while my file doesn’t have protobuf messages, I got the following error message, which indicates it has loaded the proto type and is attempting to read in data.

```auto
2022-03-04T13:58:40Z E! [inputs.file] Error in plugin: proto: cannot parse invalid wire-format data

```

Give that a shot!

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 7, 2022, 11:24am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/11 "2022-03-07T11:24:52Z")

</div>

Hi, Thanks for that it’s working, but its working partially. So first problem is we are getting import error for this file:

```auto
syntax = "proto3";
package myproto;
import "APnon.proto";
message Ap {
    double dl = 1;
    double ul = 2;
    ApNon apnon = 100;
}

///contents of APnon.proto:
syntax = "proto3";
package myproto;

message Apnoncumulative {
    double dl = 1;
    double ul = 2;
}
message ApNon{
	repeated Apnoncumulative apnoncumulativeobject = 10;
}

```

So after getting import error. I tried to define telegraf config for each file. But for ApNon.proto file i am getting error “cannot parse invalid wire-format data” for second message but for first message i am able to read contents. Telegraf config for ApNon.proto

```auto
xpath_protobuf_type = "package.ApNon" //invalid wire format
xpath_protobuf_type = "package.Apnoncumulative" //working correctly.

```

Any idea to resolve these 2 issues.

Thanks

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 7, 2022, 2:30pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/12 "2022-03-07T14:30:18Z")

</div>

> [@Pratik\_Das\_Baghel](#):
>
> ```auto
> xpath_protobuf_type = "package.ApNon" //invalid wire format
> xpath_protobuf_type = "package.Apnoncumulative" //working correctly.
> 
> ```

Telegraf is not able to read more than one type. It is only going to read a single value from that type option.

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 7, 2022, 3:24pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/13 "2022-03-07T15:24:07Z")

</div>

Sorry for not being clear. I meant with one telegraf config wire format was giving error and for second telegraf config it was parsing correctly. But i need to use first vonfiguration only.

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 7, 2022, 3:58pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/14 "2022-03-07T15:58:46Z")

</div>

Can you provide the exact error message and config you are using, please?

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 8, 2022, 5:10am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/15 "2022-03-08T05:10:10Z")

</div>

Proto file contents:

```auto
//ApNon.proto
syntax = "proto3";
package myproto;

message Apnoncumulative {
    double dl = 1;
    double ul = 2;
}
message ApNon{
	repeated Apnoncumulative apnoncumulativeobject = 10;
}

```

So here is first configuration:

```auto
data_format = "xpath_protobuf"
xpath_protobuf_type = "package.ApNon"
xpath_protobuf_file = "APnon.proto"

```

Telegraf log:

 ![image](https://us1.discourse-cdn.com/flex023/uploads/influxdata/original/2X/d/d73dd73d42844c9373203dbe7c83c10a0a3be738.png)

Second telegraf configuration:

```auto
data_format = "xpath_protobuf"
xpath_protobuf_type = "package.Apnoncumulative"
xpath_protobuf_file = "APnon.proto"

```

Telegraf log:

```auto
XML document equivalent: "<?xml version=\"1.0\"?><dl>5.5</dl><ul>3.0</ul>"

```

What would be solution to resolve issue in first telegraf configuration. Because requirement is to read ApNon message.

Thanks

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 8, 2022, 3:01pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/16 "2022-03-08T15:01:54Z")

</div>

@Pratik_Das_Baghel & @Ravikant_Gautam are you two working on the same project/team?

---

<div class="post-metadata">

**Author:** ![Ravikant\_Gautam](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/ravikant_gautam/32/5700_2.png) [@Ravikant\_Gautam](https://community.influxdata.com/u/Ravikant_Gautam)\
**Post date:** [March 8, 2022, 3:06pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/17 "2022-03-08T15:06:12Z")

</div>

yes we are working on same project.

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [March 8, 2022, 3:47pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/18 "2022-03-08T15:47:15Z")

</div>

This error message comes directly from the [protobuf library](https://github.com/protocolbuffers/protobuf-go/blob/3992ea83a23c00882339f33511074d251e19822c/internal/impl/decode.go#L20):

```auto
cannot parse invalid wire-format data

```

It means it was unable to parse the data you received given the type and message you specified. It could be that the data coming in is not the format you expected, your buffer message could be wrong, or some other things. At this point, Telegraf is doing what you told it to do.

I would suggest checking out some of the raw data with [Protobuf Code Generator and Parser | protobufnet | Marc Gravell](https://protogen.marcgravell.com/decode) and ensuring it matches the type you are specifying.

> yes we are working on same project.

It would be really nice to keep these questions in a single place. We are now up to three places, between this topic, the new topic you created, and the bug you filed. I’m going to let Sven help you out with importing multiple files.

---

<div class="post-metadata">

**Author:** ![Ravikant\_Gautam](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/ravikant_gautam/32/5700_2.png) [@Ravikant\_Gautam](https://community.influxdata.com/u/Ravikant_Gautam)\
**Post date:** [March 8, 2022, 4:46pm UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/19 "2022-03-08T16:46:43Z")

</div>

Hi sorry for creating a completely different thread for importing multiple proto file errors.  
The issue I raised on GitHub because Sven told me to do the same so he can re-produce the same issue and help me better.  
Thanks

---

<div class="post-metadata">

**Author:** ![Pratik\_Das\_Baghel](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/pratik_das_baghel/32/5698_2.png) [@Pratik\_Das\_Baghel](https://community.influxdata.com/u/Pratik_Das_Baghel)\
**Post date:** [March 9, 2022, 11:44am UTC](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985/20 "2022-03-09T11:44:56Z")

</div>

> [@Pratik\_Das\_Baghel](#):
>
> ```auto
> syntax = "proto3";
> package myproto;
> 
> message Apnoncumulative {
> double dl = 1;
> double ul = 2;
> }
> message ApNon{
> repeated Apnoncumulative apnoncumulativeobject = 10;
> }
> 
> ```

Hi. So does the telegraf protobuf parser supports the above protobuf format. With one message calling another message using repeated.

[Next page](https://community.influxdata.com/t/telegraf-xml-protobuf-doubt/23985.md?page=2)