Getting container name with Jolokia Plugin running in Kubernetes

1

I’ve got a Kubernetes cluster running some Springboot applications instrumented with the Jolokia JVM agent plugin. I’m running Telegraf as deployment alongside my Java containers to get the JVM metrics from Jolokia.

The problem I’m having is that the Jolokia plugin always reports the hostname as the Telegraf container running Telegraf collecting the Jolokia metrics and I want the payloads to contain the actual container hostname which is running the Jolokia agent. I’m aware of the inputs.server portion of the config where you can specify a name and host but the problem is when I have Jolokia running against multiple pods I can’t hardcode in the specific values.

For what it’s worth here here is my config map. Is it possible to get the container hostname within the payloads?

apiVersion: v1
kind: ConfigMap
metadata:
  name: telegraf-config
data:
  telegraf.conf: |+
    [global_tags]
      datacenter = "aws"
    [agent]
      interval = "10s"
      round_interval = true
      metric_batch_size = 1000
      metric_buffer_limit = 10000
      collection_jitter = "0s"
      flush_interval = "10s"
      flush_jitter = "0s"
      precision = ""
      omit_hostname = false
    [[outputs.http]]
      <ommitted>
        
      [[inputs.jolokia2_agent]]
       urls = ["http://sample-tomcat.default.svc.cluster.local:8778/jolokia"]
       name_prefix = "tomcat."

    [[inputs.jolokia2_agent.metric]]
      name  = "OperatingSystem"
      mbean = "java.lang:type=OperatingSystem"
      paths = ["ProcessCpuLoad","SystemLoadAverage","SystemCpuLoad"]
    

    [[inputs.jolokia2_agent.metric]]
      name  = "jvm_runtime"
      mbean = "java.lang:type=Runtime"
      paths = ["Uptime"]
   

    [[inputs.jolokia2_agent.metric]]
      name  = "jvm_memory"
      mbean = "java.lang:type=Memory"
      paths = ["HeapMemoryUsage", "NonHeapMemoryUsage", "ObjectPendingFinalizationCount"]


    [[inputs.jolokia2_agent.metric]]
      name     = "jvm_garbage_collector"
      mbean    = "java.lang:name=*,type=GarbageCollector"
      paths    = ["CollectionTime", "CollectionCount"]
      tag_keys = ["name"]

    [[inputs.jolokia2_agent.metric]]
      name       = "jvm_memory_pool"
      mbean      = "java.lang:name=*,type=MemoryPool"
      paths      = ["Usage", "PeakUsage", "CollectionUsage"]
      tag_keys   = ["name"]
      tag_prefix = "pool_"

    [[inputs.jolokia2_agent.metric]]
      name     = "GlobalRequestProcessor"
      mbean    = "Catalina:name=*,type=GlobalRequestProcessor"
      paths    = ["requestCount","bytesReceived","bytesSent","processingTime","errorCount"]
      tag_keys = ["name"]

    [[inputs.jolokia2_agent.metric]]
      name     = "JspMonitor"
      mbean    = "Catalina:J2EEApplication=*,J2EEServer=*,WebModule=*,name=jsp,type=JspMonitor"
      paths    = ["jspReloadCount","jspCount","jspUnloadCount"]
      tag_keys = ["J2EEApplication","J2EEServer","WebModule"]

    [[inputs.jolokia2_agent.metric]]
      name     = "ThreadPool"
      mbean    = "Catalina:name=*,type=ThreadPool"
      paths    = ["maxThreads","currentThreadCount","currentThreadsBusy"]
      tag_keys = ["name"]

    [[inputs.jolokia2_agent.metric]]
      name     = "Servlet"
      mbean    = "Catalina:J2EEApplication=*,J2EEServer=*,WebModule=*,j2eeType=Servlet,name=*"
      paths    = ["processingTime","errorCount","requestCount"]
      tag_keys = ["name","J2EEApplication","J2EEServer","WebModule"]

    [[inputs.jolokia2_agent.metric]]
      name     = "Cache"
      mbean    = "Catalina:context=*,host=*,name=Cache,type=WebResourceRoot"
      paths    = ["hitCount","lookupCount"]
      tag_keys = ["context","host"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: telegraf
spec:
  selector:
    matchLabels:
      app: telegraf
  minReadySeconds: 5
  template:
    metadata:
      labels:
        app: telegraf
    spec:
      containers:
        - image: telegraf:1.10.0
          name: telegraf
          volumeMounts:
            - name: telegraf-config-volume
              mountPath: /etc/telegraf/telegraf.conf
              subPath: telegraf.conf
              readOnly: true
      volumes:
        - name: telegraf-config-volume
          configMap:
            name: telegraf-config
2

Hi,

Telegraf sets the hostname tag when running by looking at the system you are running on, hence why telegraf is showing up currently. It sounds like you want to set the hostname dynamically for metrics, based on the jolokia system it was collected from.

I’m not entirely familiar with what data comes back from jolokia, but is there data in there that you could use to then update the hostname tag using a processor?