Telegraf http JSON parser errors out with JSON time key could not be foun

Telegraf
1
  1. I am trying to use telegraf to call a http end point that has the following json output
{
"capacity_usage": [
    {
        "usage_type": "NUMBER_OF_GROUPS",
        "display_name": "Groups",
        "current_usage_count": 0,
        "max_supported_count": 10000,
        "min_threshold_percentage": 30,
        "max_threshold_percentage": 50,
        "current_usage_percentage": 0,
        "severity": "INFO"
    },
    {
        "usage_type": "NUMBER_OF_DHCP_IP_POOLS",
        "display_name": "System-wide DHCP Ranges/Pools",
        "current_usage_count": 0,
        "max_supported_count": 10000,
        "min_threshold_percentage": 70,
        "max_threshold_percentage": 100,
        "current_usage_percentage": 0,
        "severity": "INFO"
    }
],
"meta_info": {
    "last_updated_timestamp": 1589228467895,
    "min_global_threshold_percentage": 70,
    "max_global_threshold_percentage": 100
},
"unreported_usage_types": [],
"_protection": "NOT_PROTECTED"
}
  1. I have the following telegraf config
[[inputs.http]]
#URL for NSXT in JSON format
 urls = [
    "https://abc/policy/api/v1/infra/capacity/dashboard/usage"
  ]

  ## HTTP method
  method = "GET"

  ## Optional HTTP headers
  # headers = {"X-Special-Header" = "Special-Value"}

  ## HTTP entity-body to send with POST/PUT requests.
  # body = ""

  ## HTTP Content-Encoding for write request body, can be set to "gzip" to
  ## compress body or "identity" to apply no encoding.
  # content_encoding = "identity"

  ## Optional file with Bearer token
  ## file content is added as an Authorization header
  # bearer_token = "/path/to/file"

  ## Optional HTTP Basic Auth Credentials
  username = "admin"
  password = "abc"


  #Overwrite measurement name from default `http` to `citibikenyc`
  name_override = "nsxt"

  #Exclude url and host items from tags
  tagexclude = ["url", "host"]

  #Data from HTTP in JSON format
  data_format = "json"

  #Parse `stationBeanList` array only
  #json_query = "capacity_usage"

  #Set station metadata as tags
  tag_keys = ["usage_type", "display_name"]

  #Do not include station landmark data as fields
  fielddrop = ["landMark"]

  #JSON values to set as string fields
  json_string_fields = ["current_usage_count", "max_supported_count", "min_threshold_percentage", "max_threshold_percentage", "current_usage_percentage","last_updated_timestamp"]

  #Latest station information reported at `lastCommunicationTime`
  json_time_key = "meta_info.last_updated_timestamp"

  #Time is reported in Golang "reference time" format
  json_time_format = "unix_ms"
  1. The time stamp is in unix_ms format under meta_info.last_updated_timestam
  2. I tried to run the config using telegraf -config ~/tel.conf -test -debug=true
  3. I get the following error JSON time key could not be found
  4. What am i doing wrong in the config
2

Try the following:

  json_time_key = "meta_info_last_updated_timestamp"

If you omit the json_time_key and json_time_format config options and run Telegraf, the output will have the additional field in the output. That is how I figured out the correct name.

My config:

[[inputs.file]]
  files = ["data.json"]
  data_format = "json"
  name_override = "nsxt"
  tag_keys = ["usage_type", "display_name"]
  tagexclude = ["url", "host"]
  fielddrop = ["landMark"]
  json_time_key = "meta_info_last_updated_timestamp"
  json_time_format = "unix_ms"
  json_string_fields = [
    "current_usage_count",
    "max_supported_count",
    "min_threshold_percentage",
    "max_threshold_percentage",
    "current_usage_percentage",
    "last_updated_timestamp"
  ]

[[outputs.file]]

Using your above JSON doc (thank you for including that), produces this output:

nsxt capacity_usage_0_max_supported_count=10000,capacity_usage_1_min_threshold_percentage=70,meta_info_min_global_threshold_percentage=70,capacity_usage_0_current_usage_percentage=0,capacity_usage_1_current_usage_count=0,capacity_usage_1_max_supported_count=10000,capacity_usage_1_current_usage_percentage=0,meta_info_max_global_threshold_percentage=100,capacity_usage_0_min_threshold_percentage=30,capacity_usage_0_max_threshold_percentage=50,capacity_usage_1_max_threshold_percentage=100,capacity_usage_0_current_usage_count=0 1589228468000000000
nsxt capacity_usage_0_max_supported_count=10000,capacity_usage_1_max_supported_count=10000,meta_info_min_global_threshold_percentage=70,capacity_usage_0_current_usage_percentage=0,capacity_usage_0_max_threshold_percentage=50,capacity_usage_1_max_threshold_percentage=100,capacity_usage_1_current_usage_percentage=0,capacity_usage_0_min_threshold_percentage=30,capacity_usage_0_current_usage_count=0,capacity_usage_1_current_usage_count=0,meta_info_max_global_threshold_percentage=100,capacity_usage_1_min_threshold_percentage=70 1589228468000000000
3

@jpowers thanks so much. I broke my head around this for some time.

1 Like