Collecting a non numerical value (JMX bean with jolokia2) - converting string to integer?

1

Hello,

I am collecting Kafka Connect metrics successfully with Jolokia2 input plugin, however there is a very important data returned by the API which is the status of the component.

The status is a non numerical value: ‘unassigned’, ‘running’, ‘paused’, ‘failed’, or ‘destroyed’

Jolokia seems to ignore this data since it is not returning a numerical value:

curl http://localhost:28779/jolokia/read/kafka.connect:type=connector-task-metrics,connector=*,task=* | jq
{
  "request": {
"mbean": "kafka.connect:connector=*,task=*,type=connector-task-metrics",
"type": "read"
  },
  "value": {
"kafka.connect:connector=kafka-skink-task-telegraf,task=1,type=connector-task-metrics": {
  "offset-commit-avg-time-ms": 0,
  "running-ratio": 1,
  "pause-ratio": 0,
  "offset-commit-failure-percentage": 0,
  "offset-commit-success-percentage": 0,
  "offset-commit-max-time-ms": null,
  "batch-size-avg": 0,
  "status": "running",
  "batch-size-max": 0
}
  },
  "timestamp": 1540115333,
  "status": 200
}

The metrics returned by jolokia2 input:

kafka_connect.connector-task.offset-commit-failure-percentage;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 0 1540113960
kafka_connect.connector-task.offset-commit-success-percentage;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 1 1540113960
kafka_connect.connector-task.pause-ratio;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 0 1540113960
kafka_connect.connector-task.batch-size-max;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 209 1540113960
kafka_connect.connector-task.offset-commit-avg-time-ms;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 0 1540113960
kafka_connect.connector-task.batch-size-avg;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 68.66666666666667 1540113960
kafka_connect.connector-task.running-ratio;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 1 1540113960
kafka_connect.connector-task.offset-commit-max-time-ms;connector=kafka-skink-task-telegraf;host=telegraf-kafka-connect;jolokia_agent_url=http:--kafka-connect-3:38779-jolokia;task=2 0 1540113960

My telegraf.conf jolokia2 input:

[[inputs.jolokia2_agent.metric]]
name         = "connector-task"
mbean        = "kafka.connect:type=connector-task-metrics,connector=*,task=*"
tag_keys = ["connector", "task"]

Is there any way of transforming this data into a numerical value (using a processor plugin ?) that will allow the metric to be ingested ?

FYI output format is graphite with tags support.

Many thanks for your help !

Regards,

Guilhem

2

What does the output look like in line protocol? Try running with the test flag:

telegraf --input-filter=jolokia2_agent --test

If the field is being collected as a string field it should be visible in this output, and you could use the converter or enum processors to convert it to a tag or numeric value.

3

Yeah I found my way :wink:

For the records, I have been able to get the result required using the enum processor:

   # Kafka connect return a status value which is non numerical
   # Using the enum processor with the following configuration replaces the string value by our mapping
   [[processors.enum]]
     [[processors.enum.mapping]]
       ## Name of the field to map
       field = "status"

       ## Table of mappings
       [processors.enum.mapping.value_mappings]
         paused = 0
         running = 1
         unassigned = 2
         failed = 3
         destroyed = 4
2 Likes
4

Hi Daniel,

Thank you very much for your reply, I couldn’t unfortunately reply before as my account was temporarily blocked for some reasons.
I had found my way with enum processor, as you mentioned one of the 2 ways, magic :wink:

Thanks!