Telegraf json parsing with string inputs

Telegraf
1

I’m using [[inputs.http]] to collect VPN status information using an https API call that returns data in JSON format.

The following is a sample JSON input:

JSON input 
{
"user": {
    "login": "myusername",
    "register_date": "2018-07-14 22:50:53",
    "register_unix": "1531608653",
    "premium": true,
    "expiration_unix": "1586906223",
    "expiration_date": "2020-04-14 23:17:03",
    "expiration_days": 375,
    "last_attempt_unix": "1554475586",
    "posts": "0",
    "last_post": null,
    "last_visit_unix": "1554387254",
    "last_activity_unix": "1554446767",
    "credits": 0,
    "credit": [],
    "connected": true
},
"sessions": [
    {
        "device_name": "Server",
        "device_description": "OVPN Outgoing",
        "vpn_ip": "10.23.44.175",
        "vpn_ipv4": "10.23.44.175",
        "vpn_ipv6": "fde6:7a:7d20:132c::10ad",
        "exit_ip": "64.45.139.43",
        "exit_ipv4": "64.45.139.43",
        "exit_ipv6": "2645:1f50:2000:25:848d:e3b5:8bc8:34f",
        "entry_ip": "64.45.139.42",
        "entry_ipv4": "64.45.139.42",
        "entry_ipv6": "2645:1f50:2000:25:e46c:8e0d:ee6b:38d7",
        "server_name": "Muscat",
        "server_country": "United States",
        "server_country_code": "us",
        "server_continent": "America",
        "server_location": "Dallas, Texas",
        "server_bw": "1000",
        "bytes_read": "1744743821329",
        "bytes_write": "936713144587",
        "connected_since_date": "2019-03-08 05:04:36",
        "connected_since_unix": "1552021476",
        "speed_read": "537.1167",
        "speed_write": "14872.4167"
    },
    {
        "device_name": "Desktop",
        "device_description": "Desktop",
        "vpn_ip": "10.29.204.219",
        "vpn_ipv4": "10.29.204.219",
        "vpn_ipv6": "fde6:7a:7d20:19cc::10d9",
        "exit_ip": "193.37.254.19",
        "exit_ipv4": "193.37.254.19",
        "exit_ipv6": "2a0d:5600:2:6:f362:ba43:29f3:b8a",
        "entry_ip": "193.37.254.18",
        "entry_ipv4": "193.37.254.18",
        "entry_ipv6": "2a0d:5600:2:6:994d:de38:fedf:47a7",
        "server_name": "Chalawan",
        "server_country": "United States",
        "server_country_code": "us",
        "server_continent": "America",
        "server_location": "Phoenix, Arizona",
        "server_bw": "1000",
        "bytes_read": "0",
        "bytes_write": "0",
        "connected_since_date": "2019-04-05 14:46:25",
        "connected_since_unix": "1554475585",
        "speed_read": null,
        "speed_write": null
    }
],
"connection": {
    "device_name": "Server",
    "device_description": "OVPN Outgoing",
    "vpn_ip": "10.23.44.175",
    "vpn_ipv4": "10.23.44.175",
    "vpn_ipv6": "fde6:7a:7d20:132c::10ad",
    "exit_ip": "64.45.139.43",
    "exit_ipv4": "64.45.139.43",
    "exit_ipv6": "2645:1f50:2000:25:848d:e3b5:8bc8:34f",
    "entry_ip": "64.45.139.42",
    "entry_ipv4": "64.45.139.42",
    "entry_ipv6": "2645:1f50:2000:25:e46c:8e0d:ee6b:38d7",
    "server_name": "Muscat",
    "server_country": "United States",
    "server_country_code": "us",
    "server_continent": "America",
    "server_location": "Dallas, Texas",
    "server_bw": "1000",
    "bytes_read": "1744743821329",
    "bytes_write": "936713144587",
    "connected_since_date": "2019-03-08 05:04:36",
    "connected_since_unix": "1552021476",
    "speed_read": "537.1167",
    "speed_write": "14872.4167"
},
"result": "ok"

}

The fields I want are:

expiration_days
device_description
exit_ipv4
exit_ipv6
bytes_read
bytes_write
speed_read
speed_write

The following are the relevant lines from my .conf file:

tag_keys = ["device_description"]
name_override = "VPN"
json_query = "sessions"
json_string_fields = [
"exit_ipv4",
"exit_ipv6"
]
data_format = "json"

The telegraf --test command returns exit_ipv4 and exit_ipv6 for tags ‘OVPN Outgoing’ and ‘Desktop’. It took me a while to figure out why nothing else was being returned – my numbers all have quotes around them making them strings.

To make the data more readable in Grafana, I don’t want a 13-digit number for bytes_read for example, and would like to set units within Grafana to ‘bytes’ so that it will display ‘1.74 TB’ instead of ‘1744743821329’.

Is there a way to convert the string format numbers into an actual number?

Interestingly while I was learning and experimenting with json input, I had my desired fields defined as tag_keys and bytes_read for example was treated as a number by Grafana. The problem is this only works when there is one device in sessions (Server is connected 24/7) and once I add a session (Desktop) the parsing gets confused.

1 Like