How to split measurement fields into separate measurements?

Telegraf
1

Hi,

I’m using the gNMI plugin to connect to the interfaces sensors on my network devices.

[[inputs.gnmi.subscription]]
    name = "interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface"
    subscription_mode = "sample"
    suppress_redundant = false
    sample_interval = "60s"

This subscription also pulls back the optics metrics. When I store this data inside InfluxDB these are the fields that I’m seeing for the interfaces measurement:

 show field keys from interfaces
name: interfaces
fieldKey                                                                    fieldType
--------                                                                    ---------
ethernet/state/auto_negotiate                                               boolean
ethernet/state/counters/in_8021q_frames                                     integer
ethernet/state/counters/in_block_errors                                     integer
ethernet/state/counters/in_crc_errors                                       integer
optics/lanediags/lane/lane_laser_bias_current                               float
optics/lanediags/lane/lane_laser_bias_current_high_alarm                    boolean
optics/lanediags/lane/lane_laser_bias_current_high_warning                  boolean
optics/lanediags/lane/lane_laser_bias_current_low_alarm                     boolean

I want to separate the optics metrics and push them to a different measurement. For example, I want to create a measurement name called optics and save all the optics data there.

Unfortunately, the network devices do not allow me to subscribe to the optics sensor like this:

[[inputs.gnmi.subscription]]
    name = "optics"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/optics"
    subscription_mode = "sample"
    suppress_redundant = false
    sample_interval = "60s"

That gives an error message stating an invalid URL path was defined.

Is this possible to do?

I’m using Telegraf v.1.23.4

2

I think I figured it out by using the starlark processor plugin.

Here is how I was able to accomplish this:

[[processors.starlark]]
namepass = ["interfaces"]
  source = '''
def apply(metric):
  metrics = []
  for k, v in metric.fields.items():
    if "optics" in k:
      opticsMetric = Metric("optics")            # create new metric(opticsMetric) and measurement(optics)
      opticsMetric.fields[k] = v                 # assign all optics fields to the new opticsMetric
      for tagKey, tagVal in metric.tags.items(): # assign all optics tags to the new opticsMetric
        opticsMetric.tags[tagKey] = tagVal
      metrics.append(opticsMetric)               # push in optics metrics to the metrics list
    else:
      metrics.append(metric)                     # push in non-optics sensor data to the metrics list
  return metrics                                 # return both interfaces and optics metrics
'''

Now I am able to see a new measurement named “optics” in my InfluxDB which contains only my optics metrics and I also have the “interfaces” measurement that only contains my interface metrics.

3

glad you solved it.

You may be able to achieve the same result using processor override and filtering

4

@mohsin106 I second what @Giovanni_Luisotto write, probably you are better off with the override or rename processor(s) especially if you are dealing with large amounts of metrics… Please keep in mind that starlark is an interpreter language…

5

I’m not sure how to accomplish this with rename and filtering. Seems like I can either get the interfaces measurement to populate correctly or the optics measurement. I can’t get both measurements to populate correctly.

This is what I tried to do:

[[inputs.gnmi.subscription]]
    name = "interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface"
    subscription_mode = "sample"
    suppress_redundant = false
    sample_interval = "60s"

# this creates the interfaces measurement without any optics metrics, but doesn't create optics measurement
[processors.override]]
  order = 1
  namepass = ["interfaces"]
  fielddrop = ["optics*"]
  name_override = "interfaces"

[[inputs.gnmi.subscription]]
    name = "interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface"
    subscription_mode = "sample"
    suppress_redundant = false
    sample_interval = "60s"

# this creates the optics measurement without any interfaces metrics, but doesn't create interfaces measurement
[[processors.override]]
  order = 1
  namepass = ["interfaces"]
  fieldpass = ["optics*"]
  name_override = "optics"

Putting both together doesn’t emit any measurement, because I think all my optics fields are being dropped by the first override so the second override doesn’t see anything.

[[inputs.gnmi.subscription]]
    name = "interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface"
    subscription_mode = "sample"
    suppress_redundant = false
    sample_interval = "60s"

[[processors.override]]
  order = 1
  namepass = ["interfaces"]
  fielddrop = ["optics*"]
  name_override = "interfaces"

[[processors.override]]
  order = 2
  namepass = ["interfaces"]
  fieldpass = ["optics*"]
  name_override = "optics"
6

Putting both together doesn’t emit any measurement, because I think all my optics fields are being dropped by the first override so the second override doesn’t see anything.

That’s correct, you can still make it work by cloning you data and having two separate sets to work with.

[[inputs.gnmi.subscription]]
    name = "interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface"
    subscription_mode = "sample"
    suppress_redundant = false
    sample_interval = "60s"

[[processors.clone]]
  order = 1
  namepass = ["interfaces"]
  name_override = "optics"
  fieldpass = ["optics*"]

[[processors.override]]
  order = 2
  namepass = ["interfaces"]
  fielddrop = ["optics*"]
  name_override = "interfaces"

This should work, in the worst case you’ll need to clone first and then use an override (one for each set).

7

@Giovanni_Luisotto , this did not work and the end result was that only the optics measurement was created with only the optics metrics. The interfaces measurement was not created.

Would I need to create another [[inputs.gnmi.subscription]] stanza and then just pass the interfaces metrics? If so, this will establish two separate gRPC sessions on the same network device.

Is the starlark plugin more expensive to use?

8

if you create another input session you will fetch the data twice and I don’t see the point of doing that.

The processor.clone is working fine if you see the optics metrics.
Therefore the issue is in the “interfaces” processor.override that removes the fields with a given name (fielddrop = ["optics*"]).

Doing such things without sample data is not that easy, can you provide a sample of your data (line protocol)? (with both input and desired output)

9

Sample Data:
You can see the interfaces sensor is reporting metrics about the interfaces as well as optic metrics. I want to store any field that has the word “optics” in it and save it to a new measurement called “optics”

interfaces,device=device1.mgt.net,host=telegraf-agent,interface-name=et-0/0/31 in-octets=3684801013572901i,out-pkts=433625371i,in-pkts=3384240329403i,in-unicast-pkts=116402503i,out-multicast-pkts=433625371i,high-speed=100000i,parent-ae-name="ae15",carrier-transitions=1i,out-octets=53655804358i,in-multicast-pkts=3384123926900i 1670338427875000000
interfaces,device=device1.mgt.net,host=telegraf-agent,interface-name=ae2 last-change=1664204707593168000i,admin-status="UP",oper-status="UP",mtu=9100i,type="ieee8023adLag",description="device description",enabled=true 1670338427955943620
interfaces,device=device1.mgt.net,host=telegraf-agent,interface-name=ae2 mac-address="aa:18:22:f2:cc:33",port-speed="SPEED_400GB" 1670338427957359779

interfaces,device=device2.mgt.net,host=telegraf-agent,interface-name=xe-0/0/0 optics/module_temp=39.89799880981445,optics/module_temp_high_alarm_threshold=75i,optics/module_temp_low_alarm_threshold=-5i,optics/module_temp_high_warning_threshold=70i,optics/module_temp_low_warning_threshold=0i,optics/laser_output_power_high_alarm_threshold_dbm=6,optics/laser_output_power_low_alarm_threshold_dbm=-5,optics/laser_output_power_high_warning_threshold_dbm=3,optics/laser_output_power_low_warning_threshold_dbm=-1,optics/laser_rx_power_high_alarm_threshold_dbm=-3,optics/laser_rx_power_low_alarm_threshold_dbm=-30.969999313354492,optics/laser_rx_power_high_warning_threshold_dbm=-7,optics/laser_rx_power_low_warning_threshold_dbm=-26.989999771118164,optics/laser_bias_current_high_alarm_threshold=75,optics/laser_bias_current_low_alarm_threshold=25,optics/laser_bias_current_high_warning_threshold=70,optics/laser_bias_current_low_warning_threshold=30 1670338428026000000
interfaces,device=device2.mgt.net,host=telegraf-agent,interface-name=xe-0/0/0,lane_number=0 optics/lanediags/lane/lane_laser_output_power_dbm=0.29383736848831177,optics/lanediags/lane/lane_laser_receiver_power_dbm=-9.100947380065918,optics/lanediags/lane/lane_laser_bias_current=50,optics/lanediags/lane/lane_laser_output_power_high_alarm=false,optics/lanediags/lane/lane_laser_output_power_low_alarm=false,optics/lanediags/lane/lane_laser_output_power_high_warning=false,optics/lanediags/lane/lane_laser_output_power_low_warning=false,optics/lanediags/lane/lane_laser_receiver_power_high_alarm=false,optics/lanediags/lane/lane_laser_receiver_power_low_alarm=false,optics/lanediags/lane/lane_laser_receiver_power_high_warning=false,optics/lanediags/lane/lane_laser_receiver_power_low_warning=false,optics/lanediags/lane/lane_laser_bias_current_high_alarm=false,optics/lanediags/lane/lane_laser_bias_current_low_alarm=false,optics/lanediags/lane/lane_laser_bias_current_high_warning=false,optics/lanediags/lane/lane_laser_bias_current_low_warning=false 1670338428026000000
interfaces,device=device2.mgt.net,host=telegraf-agent,interface-name=xe-0/0/1 optics/module_temp=42.30799865722656,optics/module_temp_high_alarm_threshold=90i,optics/module_temp_low_alarm_threshold=-45i,optics/module_temp_high_warning_threshold=85i,optics/module_temp_low_warning_threshold=-40i,optics/laser_output_power_high_alarm_threshold_dbm=6,optics/laser_output_power_low_alarm_threshold_dbm=-5,optics/laser_output_power_high_warning_threshold_dbm=3,optics/laser_output_power_low_warning_threshold_dbm=-1,optics/laser_rx_power_high_alarm_threshold_dbm=-3,optics/laser_rx_power_low_alarm_threshold_dbm=-30.969999313354492,optics/laser_rx_power_high_warning_threshold_dbm=-7,optics/laser_rx_power_low_warning_threshold_dbm=-26.989999771118164,optics/laser_bias_current_high_alarm_threshold=75,optics/laser_bias_current_low_alarm_threshold=25,optics/laser_bias_current_high_warning_threshold=70,optics/laser_bias_current_low_warning_threshold=30 1670338428026000000

Would like the above data to look like this:

interfaces,device=device1.mgt.net,host=telegraf-agent,interface-name=et-0/0/31 in-octets=3684801013572901i,out-pkts=433625371i,in-pkts=3384240329403i,in-unicast-pkts=116402503i,out-multicast-pkts=433625371i,high-speed=100000i,parent-ae-name="ae15",carrier-transitions=1i,out-octets=53655804358i,in-multicast-pkts=3384123926900i 1670338427875000000
interfaces,device=device1.mgt.net,host=telegraf-agent,interface-name=ae2 last-change=1664204707593168000i,admin-status="UP",oper-status="UP",mtu=9100i,type="ieee8023adLag",description="device description",enabled=true 1670338427955943620
interfaces,device=device1.mgt.net,host=telegraf-agent,interface-name=ae2 mac-address="aa:18:22:f2:cc:33",port-speed="SPEED_400GB" 1670338427957359779

optics,device=device2.mgt.net,host=telegraf-agent,interface-name=xe-0/0/0 optics/module_temp=39.89799880981445,optics/module_temp_high_alarm_threshold=75i,optics/module_temp_low_alarm_threshold=-5i,optics/module_temp_high_warning_threshold=70i,optics/module_temp_low_warning_threshold=0i,optics/laser_output_power_high_alarm_threshold_dbm=6,optics/laser_output_power_low_alarm_threshold_dbm=-5,optics/laser_output_power_high_warning_threshold_dbm=3,optics/laser_output_power_low_warning_threshold_dbm=-1,optics/laser_rx_power_high_alarm_threshold_dbm=-3,optics/laser_rx_power_low_alarm_threshold_dbm=-30.969999313354492,optics/laser_rx_power_high_warning_threshold_dbm=-7,optics/laser_rx_power_low_warning_threshold_dbm=-26.989999771118164,optics/laser_bias_current_high_alarm_threshold=75,optics/laser_bias_current_low_alarm_threshold=25,optics/laser_bias_current_high_warning_threshold=70,optics/laser_bias_current_low_warning_threshold=30 1670338428026000000
optics,device=device2.mgt.net,host=telegraf-agent,interface-name=xe-0/0/0,lane_number=0 optics/lanediags/lane/lane_laser_output_power_dbm=0.29383736848831177,optics/lanediags/lane/lane_laser_receiver_power_dbm=-9.100947380065918,optics/lanediags/lane/lane_laser_bias_current=50,optics/lanediags/lane/lane_laser_output_power_high_alarm=false,optics/lanediags/lane/lane_laser_output_power_low_alarm=false,optics/lanediags/lane/lane_laser_output_power_high_warning=false,optics/lanediags/lane/lane_laser_output_power_low_warning=false,optics/lanediags/lane/lane_laser_receiver_power_high_alarm=false,optics/lanediags/lane/lane_laser_receiver_power_low_alarm=false,optics/lanediags/lane/lane_laser_receiver_power_high_warning=false,optics/lanediags/lane/lane_laser_receiver_power_low_warning=false,optics/lanediags/lane/lane_laser_bias_current_high_alarm=false,optics/lanediags/lane/lane_laser_bias_current_low_alarm=false,optics/lanediags/lane/lane_laser_bias_current_high_warning=false,optics/lanediags/lane/lane_laser_bias_current_low_warning=false 1670338428026000000
optics,device=device2.mgt.net,host=telegraf-agent,interface-name=xe-0/0/1 optics/module_temp=42.30799865722656,optics/module_temp_high_alarm_threshold=90i,optics/module_temp_low_alarm_threshold=-45i,optics/module_temp_high_warning_threshold=85i,optics/module_temp_low_warning_threshold=-40i,optics/laser_output_power_high_alarm_threshold_dbm=6,optics/laser_output_power_low_alarm_threshold_dbm=-5,optics/laser_output_power_high_warning_threshold_dbm=3,optics/laser_output_power_low_warning_threshold_dbm=-1,optics/laser_rx_power_high_alarm_threshold_dbm=-3,optics/laser_rx_power_low_alarm_threshold_dbm=-30.969999313354492,optics/laser_rx_power_high_warning_threshold_dbm=-7,optics/laser_rx_power_low_warning_threshold_dbm=-26.989999771118164,optics/laser_bias_current_high_alarm_threshold=75,optics/laser_bias_current_low_alarm_threshold=25,optics/laser_bias_current_high_warning_threshold=70,optics/laser_bias_current_low_warning_threshold=30 1670338428026000000