Python client not working (ValueError)

1

Hi all,
I’m writing a python code that pings a list of hosts and saves them to a bucket if they respond. It’s simple enough but I keep encountering this error

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (57,) + inhomogeneous part.

However, when I change the range from 5 minutes to 2-3 seconds the code works fine. Also when I test this code on another instance of InfluxDB where we have fewer hosts, it works fine.
I’ll add my entire code below, please take a look.

from influxdb_client import InfluxDBClient, Point, WriteOptions
import numpy as np
import pandas as pd
import warnings
from influxdb_client.client.write_api import SYNCHRONOUS
import requests
import socket
import time
from datetime import datetime, timedelta

urls = "url"
token = ""
organization = ""

# I got warnings about not pivoting the query so I put this in
warnings.simplefilter("ignore", MissingPivotFunction)

# query
last_value = f'''
from(bucket: "bucket")
  |> range(start: -5m)
  |> filter(fn: (r) => r["_measurement"] == "system")
  |> filter(fn: (r) => r["_field"] == "uptime")
  |> filter(fn: (r) => r["environment"] == "production")
  |> filter(fn: (r) => r["vmtype"] == "vmware")
  '''

# connecting to InfluxDB via API, running the query and making a dataframe
client = InfluxDBClient(url="url", token=token, org=organization, debug=False)
system_stats = client.query_api().query_data_frame(org=organization, query=last_value)
df = pd.DataFrame(system_stats)

hosts = df["host"].values.tolist()

#saving the hosts to an external file
with open("hosts.txt", "w") as file:
    for host in hosts:
        file.write(host + "\n")

write_api = client.write_api(write_options=SYNCHRONOUS)
#reading from that file
with open("hosts.txt", "r") as file:
    lines = file.readlines()

#i need to remove everything after the dot because it easier
modified_names = [hostname.split('.', 1)[0] for hostname in lines]

#here i iterate thru all the hosts and lookup their IP's and save them to ip_add
ip_add = []
errors = []
index = 0

while index < len(modified_names):
    host = modified_names[index]
    try:
        ip = socket.gethostbyname(host)
        ip_add.append(ip)
    except Exception as e:
        errors.append(e)
    index += 1


#function to get host name, i need this to save the name to influxdb bucket
def getHost(ip):
    host = socket.gethostbyaddr(ip)
    return host[0]


for host in ip_add:
    try:
        r = requests.get(f'http://{host}')
        data = Point("system").tag("live hosts_test1", getHost(host)).field("host reponse code", r.status_code)
        write_api.write(bucket="bucket", org="org", record=data)
    except Exception as e:
        pass
2

Hello @Zcarevic,
A couple things I noticed about your code that might help.
First this is small but
df = pd.DataFrame(system_stats)
Is redundant. You already have a dataframe.
Next I recommend doing that work within a dataframe instead of converting to array, splitting, writing to a file and pulling it back out.

That way you can write the entire dataframe back to influxdb. I think that’s more efficient than looping through values to write individual points.

This type of error occurs because we do not put an equal number of elements in each row . To solve this problem, we must put an equal number of values in all the rows of an array. So if you want to continue working with your approach I’d consider checking to make sure that you’re getting values for each loop, or that your arrays are the same size and that you handle the case where you don’t have values.

I hope that helps some.

3

Thanks Anais,
I made it work, the problem was in the unequal number of rows. I now face a different problem; my code takes too long to execute, about 45 s. Is there a way to make it run a bit quicker? I tried a couple of different things but nothing really works.