EMC Recovery Point with influxdb and Grafana

1

Hi,

I researched about EMC RP monitoring with influxdb,

But I can’t find any documentation.

2

Hi @yalcinkyildiz,
hmm could you explain a little more about what you are looking to monitor and your expectations? Could you also link to the articles you read about influxdb and EMC RP monitoring? Not one I have come across before

3

Hi @Jay_Clifford,

First of all, I apologize for the late reply and thank you for your interest.

I want to monitor the EMC Recovery Point application using influxdb, but I could not do it. I am sharing the 2 pages that I base on below.

I followed this article and followed the steps. But I could not transfer the data to influxdb. I am sharing the code block I used to transfer the log data I received to influxdb.

import scala.util.Try
import java.util.concurrent.TimeUnit

import org.influxdb.{InfluxDB, InfluxDBFactory}
import org.influxdb.InfluxDB.ConsistencyLevel
import org.influxdb.dto.Point.Builder
import org.influxdb.dto.{BatchPoints, Point}

// Output to InfluxDB
class Influx(name: String, config: Map[String, String],
               sysConfig: Map[String, Option[String]])
  extends Output(name, config, sysConfig) with Logger {

  val loggerFile: String = propOrElse("USC_HOME", "") + "log/collector-error.log"

  // Common parameters
  val address: Option[String] =
    if (config.contains("address"))
      Some(config("address"))
    else
      None

  val port: Option[Int] =
    if (config.contains("port"))
      Some(config("port").toInt)
    else
      None

  val dbname: Option[String] =
    if (config.contains("dbname"))
      Some(config("dbname"))
    else
      None

  var influxDB: InfluxDB = _

  // Validation by common parameters
  def isValid: Boolean = config.contains("address") & config.contains("port") &
    config.contains("dbname")

  // Connection initialization before data output
  def start(): Unit = {
    influxDB = InfluxDBFactory.connect("http://" + address.get + ":" + port.get,"username","password")
  }

  // Convert numbers in data from String type to Double type
  def parseDouble(s: String): Option[Double] = Try { s.toDouble }.toOption

  // Output data comes in 'msg' and 'data' structures
  def out(msg: Map[Int, (String, String)], timestamp: Long, data: Map[String, String]): Unit = {

    val header: Map[String, String] = (msg.values map (v => v._1 -> v._2)).toMap

    val batchPoints: BatchPoints = BatchPoints
      .database(dbname.get)
      .retentionPolicy("autogen")
      .consistency(ConsistencyLevel.ALL)
      .build()

    val point: Builder = Point.measurement(header("measurement"))
      .time(timestamp.toLong, TimeUnit.SECONDS)
      .tag("storage", sysConfig("name").get)
      .tag("class", sysConfig("class").get)

    if (sysConfig("type").isDefined) point.tag("type", sysConfig("type").get)

    (header.keySet - "measurement") foreach {k => point.tag(k, header(k))}

    data.foreach {p =>
      if (parseDouble(p._2).isDefined) point.addField(p._1, parseDouble(p._2).get)
      else point.addField(p._1, p._2)
    }

    batchPoints.point(point.build())
    influxDB.write(batchPoints)
  }

  // Closing connection after data output
  def stop(): Unit = {
    influxDB.close()
  }
}

I’m pulling the variables used in this code block from my collector file.

And this is my Output.scala file :

package universalstoragecollector

// Template for outputs
abstract class Output(name: String, config: Map[String, String],
                      sysConfig: Map[String, Option[String]] = Map()) {
  def isValid: Boolean
  def start(): Unit
  def out(msg: Map[Int, (String, String)], timestamp: Long, data: Map[String, String]): Unit
  def stop(): Unit
}

// Known outputs are listed here
object Output {
  def apply(name: String, config: Map[String, String],
            sysConfig: Map[String, Option[String]] = Map()): Output = {
    config.getOrElse("type", "") match {
      case "carbon" => new Carbon("carbon", config, sysConfig)
      case "influxdb" => new Influx("influxdb", config, sysConfig)
      case _ => new Output(name, config, sysConfig) {
                  def isValid = false
                  def start(): Unit = {}
                  def out(msg: Map[Int, (String, String)],
                          timestamp: Long,
                          data: Map[String, String]): Unit = {}
                  def stop(): Unit = {}
                }
    }
  }
}

Log File:

Wed Nov 30 23:23:52 UTC 2022 rp01, /clusters/statistics: java.util.NoSuchElementException: key not found: measurement
Wed Nov 30 23:23:53 UTC 2022 rp01, /rpas/statistics: java.util.NoSuchElementException: key not found: measurement
Wed Nov 30 23:23:53 UTC 2022 rp01, /groups: net.liftweb.json.MappingException: No usable value for innerSet
No usable value for groupCopiesInformation
No usable value for vmsInformation
Do not know how to convert JObject(List(JField(vmUID,JObject(List(JField(uuid,JString(500823dd-6738-b445-c705-7612367c)), JField(virtualCenterUID,JObject(List(JField(uuid,JString(1a5e7115-0def-4796-b155-1353622dg)))))))), JField(vmName,JString(YalcinMustafa_Management-win)), JField(vmBiosUuid,JString(4288312e-3aae-11bb-03a8-49b3ad33e1fc)))) into class java.lang.String

I can share the whole project if you want.

1 Like
4

Hi @yalcinkyildiz,
This is brilliant, if you would like to contribute it to our influx community repo that would be awsome. We could always look to create a blog for it aswell.

Let me know what you think: Influx Community · GitHub

If you are interested I will add you as a contributing member.

5

Hi @Jay_Clifford,

Thanks, but this project does not belong to me, I am just trying to adapt it to my own project. I also developed monitoring applications such as influxdb and Prometheus and VMware, Zerto, Fortigate Firewall by finding projects from the internet.

In this project, I was stuck on why the data could not be transferred to influxdb.
Many thanks for your offer.