Unable to parse data within InfluxDB

Telegraf
1

I am using InfluxDB v2.3 and within Telegraf section I have created a configuration and selected ‘System’. After this config was running and exposed all ‘System’ metrics I can see that within the metrics I am unable to see the data I have stored in the DB.

In the the Telegraf config, I used - telegraf/README.md at master · influxdata/telegraf · GitHub in order to expose the metrics, however, as mentioned above I cannot view the data in the database as part of my metrics.

I need my data on Influx measurement to be displayed on Prometheus in order to send certain alerts when specific parameters are triggered

Am I doing something wrong or need to do something differently?

2

Hi,

In order for InfluxDB to get the metrics Telegraf is collecting, users generally use the InfluxDB_v2 output plugin. This is what sends metrics directly to InfluxDB, which you can then view in whatever tools you want.

It might help to see your config to understand the flow of things better as well.

Thanks!

1 Like
3

Hi Jay,

Thanks for getting back. however, what I would like to achieve is the opposite. I do not want to send the data to Influx, but the contrary. I already have data in the InfluxDB v2.3. Example data is in line protocol;

Measurement_name,clientIP=“x.x.x.x”,Path=“xxx/xxx/xxx” count=1234

What I would like is to expose this data as a metric to show on Prometheus to configure further alerts from there. So far I have managed to expose metrics with the link I provided above, however the metrics do not include the actual data within the database. Is what I am looking a possibility since usually the flow is the other way round, from Prometheus to Influx?

4

Ah, you want to pull the data out of InfluxDB? This is why sharing your config would help a lot :slight_smile:

metrics do not include the actual data within the database

You are probably using the influxdb input, which only collects metrics from InfuxDB, not the actual data, as you are seeing.

I do not believe we have a no-code solution to this. You could use the exec plugin to do queries and transform the data into the Prometheus format.

5

Hi Jay,

I have reviewed the plugin you mentioned and visually it seems like it does the job. However I am running into some issues when trying to implement it. Below is my telegraf.conf;

[[outputs.influxdb_v2]]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ##   ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
  urls = ["http://x.x.x.x:8086"]

  ## API token for authentication.
  token = "2MT9KHzN0-H.....bv1TwaY7w=="

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "company_name"

  ## Destination bucket to write into.
  bucket = "cf_influx_import"

#### EXEC PLUGIN:
[[inputs.exec]]
  command = ["cat /var/tmp/cf_data/cf_influx_import.txt"]
  #command = ["/var/tmp/cf_data/script.sh"]
  data_format = "prometheus"

I have a script “script.sh” that is getting JSON data from Cloudflare and converting it to line protocol. The converted line protocol is in a text file “cf_influx_import.txt”. This is the data that is being fed into InfluxDB v2.3.

I have tried with both [“cat /var/tmp/cf_data/cf_influx_import.txt”] and [“/var/tmp/cf_data/script.sh”] but when I run the telegraf --config command from setup instructions I am facing the following error:

2022-08-04T14:26:13Z E! [telegraf] Error running agent: Error loading config file http://x.x.x.x:8086/api/v2/telegrafs/xxxx: error parsing exec, line 53: (exec.Exec.Command) cannot unmarshal TOML array into string (need slice)

Line 53 - command = [“cat /var/tmp/cf_data/cf_influx_import.txt”]

To integrate the Exec plugin I followed https://github.com/influxdata/telegraf/blob/release-1.23/plugins/outputs/exec/README.md and https://www.influxdata.com/blog/plugin-spotlight-exec-execd/ but I can’t seem to get it right. Am I mixing something up or possibly not understanding the concept of how to implement this?

6

So close :slight_smile: You want to look at the input exec readme, not the output.

The command should look like:

[[inputs.exec]]
  commands = ["cat /var/tmp/cf_data/cf_influx_import.txt"]

Also if the data is in line protocol, you will want to set data_format = "influx"

Finally, if you are only going to be reading a file, then you can also use the [[inputs.file]] plugin to read the file as well:

[[inputs.file]]
   files = ["/var/tmp/cf_data/cf_influx_import.txt"]
   data_format = "influx"

In both cases, whether with file or exec, Telegraf will read the entire file at each interval. Keep that in mind.

Hope that helps and good luck!