Why does Grafana take time to read data from InfluxDB

Sarindra_Therese_Ran
Thu, 28 Mar 2024 12:44:56 +0000
InfluxDB 2

Hey, I have a sysstat bucket with measurements such as pidstat, iostat, mpstat, vmstat, ram-memory, and diskspace. When I run my JAR file, it populates each measurement every N seconds. However, locally, I don’t have any issues with data loading on Grafana; everything seems perfect and works in real-time.
However, when I switch to a robust test server with multiple buckets, I find myself waiting for 20 to 30 minutes for the data to load, which loses the real-time aspect. I’ve already removed unnecessary tags and fields, but nothing has changed.
I use Java and Kafka for data ingestion.

//for pidstat

            logger.info("pidstat - Polling ...");

            Point point = Point.measurement(measurement)
                              .time(System.currentTimeMillis(), WritePrecision.MS);

            // Ajouter tous les champs de GrafanaLog à InfluxDB
            point.addTag("user", pidstatValue.getUsername());
            point.addTag("host", pidstatValue.getHost());
            point.addField("%usr", pidstatValue.getUsrPrct());
            point.addField("%system", pidstatValue.getSystemPrct());
            point.addField("%guest", pidstatValue.getGuestPrct());
            point.addField("%wait", pidstatValue.getWaitPrct());
            point.addField("%CPU", pidstatValue.getCpuPrct());
            point.addTag("CPU", pidstatValue.getCpu());
            point.addField("minflt/s", pidstatValue.getMinfltS());
            point.addField("majflt/s", pidstatValue.getMajfltS());
            point.addField("VSZ", pidstatValue.getVsz());
            point.addField("RSS", pidstatValue.getRss());
            point.addField("%MEM", pidstatValue.getMemPrct());
            point.addField("threads", pidstatValue.getThreads());
            point.addTag("Command", pidstatValue.getCommand());
            point.addTag("insideCommand", pidstatValue.getInsideCommand());
            writeApi.writePoint(point);
          }
          pidstatConsumer.commitSync();

// for vmstat

            logger.info("vmstat - Polling ...");
            Point point = Point.measurement(measurement)
                .time(System.currentTimeMillis(), WritePrecision.MS);

            point.addTag("user", vmstatValue.getUsername());
            point.addTag("host", vmstatValue.getHost());
            point.addTag("r", vmstatValue.getR());
            point.addField("b", vmstatValue.getB());
            point.addField("swpd", vmstatValue.getSwpd());
            point.addField("free", vmstatValue.getFree());
            point.addField("buff", vmstatValue.getBuff());
            point.addField("cache", vmstatValue.getCache());
            point.addField("si", vmstatValue.getSi());
            point.addField("so", vmstatValue.getSo());
            point.addField("bi", vmstatValue.getBi());
            point.addField("bo", vmstatValue.getBo());
            point.addField("in", vmstatValue.getIn());
            point.addField("cs", vmstatValue.getCs());
            point.addField("us", vmstatValue.getUs());
            point.addField("sy", vmstatValue.getSy());
            point.addField("id", vmstatValue.getId());
            point.addField("wa", vmstatValue.getWa());
            point.addField("st", vmstatValue.getSt());
            writeApi.writePoint(point);
          }
          vmstatConsumer.commitSync();

// …

7 posts - 2 participants