Telegraf: Processing json into prometheus_client

,
1

Hi, I’m attempting to use Telegraf to convert incoming json using the socket_listener plugin (from a utility called njmon on AIX) to the prometheus_client output plugin. It’s working well but I’m stuck trying to parse some elements of the incoming json that’s in the following format.

"filesystems": {
  "/": {
    "mount": "/",
    "device": "/dev/hd4",
    "size_mb": 1408,
    "free_mb": 551.121,
    "used_percent": 60.858,
    "inode_percent": 0.704
  },
  "/usr": {
    "mount": "/usr",
    "device": "/dev/hd2",
    "size_mb": 5248,
    "free_mb": 358.918,
    "used_percent": 93.161,
    "inode_percent": 5.311
  }
}

To use this in Prometheus, I’d like to get it into something like:

socket_listener_filesystems_size_mb{mount=“/usr”}
socket_listener_filesystems_free_mb{mount=“/usr”}

Is there a way to make this conversion within the json data formatter?

Many thanks

2

So the provided input isn’t valid JSON, so I assume you mean that there’s an enclosing bracket pair { .. }.

Okay, so … this isn’t going to look simple but it’s largely because there’s no easy way to select two keys from an nested object, iterating over them to create records (at least as far as I know!).

I’ll explain the config as I go.

# the first few lines are a placeholder for whatever your input is.
[[inputs.tail]]
  files = ["some_file_with_json.txt"]

# stick it in a metric with a special name we can grab later.
  name_override = "filesystems_raw"

# store it all as one string of json text
  data_format = "value" 
  data_type = "string"

# pull the first device out.. 
# You could repeat this for any number of devices. 
# Unfortunately it's not really dynamic, but you could generate this..
[[processors.parser]]
# override the metric name
  metric_name = "filesystems"
# pay attention to order
  order = 1
# select this metric
  namepass = "filesystems_raw"
# read the value field
  parse_fields = ["value"]
# .. as json
  data_format = "json"
  json_strict = true
# navigate to this path
  json_query = "filesystems./"
# read these as keys (rest as fields)
  tag_keys = [
    "mount",
    "device"
  ]

# repeat for /usr
[[processors.parser]]
  metric_name = "filesystems"
  order = 2
  namepass = "filesystems_raw"
  drop_original = false
  parse_fields = ["value"]
  data_format = "json"
  json_strict = true
  json_query = "filesystems./usr"
  tag_keys = [
    "mount",
    "device"
  ]

# now drop the filesystems_raw metric. 
# We're just going to do this by deleting all its fields.
[[processors.strings]]
  order = 3
  namepass = "filesystems_raw"
  fielddrop = ["value"] # deletes the metric

# if you want to view the results, you can write them to stdout.
[[outputs.file]]
  files = ["stdout"]

Let me know if you have any questions.

Steven

1 Like