Problems with telegraf + influxdb

Telegraf
,
1

Hi to everyone guys!

First of all, sorry for my English, as I’m not from an English speaker country. After a lot of time spend on searching for a solution with no results, I come here as my last hope. I’m trying to migrate the data of a csv file to my influx database, using Telegraf (both in local). The data I’m trying to import is basically the timestamp and the amount of ping at that certain time. After several configurations I didn’t found a way to it correctly.

Here is the telegraf.conf related part:

# Parse a complete file each interval
[[inputs.file]]
  ## Files to parse each interval.  Accept standard unix glob matching rules,
  ## as well as ** to match recursive files and directories.
  files = ["./prueba.csv"]
  data_format = "csv"

  ## Name a tag containing the name of the file the data was parsed from.  Leave empty
  ## to disable. Cautious when file name variation is high, this can increase the cardinality
  ## significantly. Read more about cardinality here:
  ## https://docs.influxdata.com/influxdb/cloud/reference/glossary/#series-cardinality
  # file_tag = ""
  #

  ## Character encoding to use when interpreting the file contents.  Invalid
  ## characters are replaced using the unicode replacement character.  When set
  ## to the empty string the data is not decoded to text.
  ##   ex: character_encoding = "utf-8"
  ##       character_encoding = "utf-16le"
  ##       character_encoding = "utf-16be"
  ##       character_encoding = ""
  # character_encoding = ""

  ## The dataformat to be read from files
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md

  csv_header_row_count = 0
  ##csv_column_types = ["string","float"]
  csv_skip_rows = 0
  csv_skip_columns = 0
  csv_column_names = ["timestamp","ping"]
  csv_tag_columns = ["Time"]
  csv_measurement_column = "timestamp"
  csv_trim_space = true

  ##csv_timestamp_column = "timestamp"
  ##csv_timestamp_format = "2006-01-02 15:04:05"
  ##csv_timezone = "Europe/Madrid"

Yup, I’m aware that the las 3 lines are commented. It’s because of the frustration level, makes you try every option.

The csv structure:

timestamp,ping
2022-03-14 08:21:06,25
2022-03-14 08:21:09,49
2022-03-14 08:21:12,4
2022-03-14 08:21:15,3.50
2022-03-14 08:21:18,22
… and so on

As far as now, the only thing I succeeded in, is in importing some data but in the wrong way:

image
image1299×429 4.18 KB

As you can see, every date is imported as a “table” and the ping is not even written. Influx version is 1.8 and Telegraf version is 1.20. Feel free to ask for more info if needed!

Thanks in advance!

2

I think this might be more of what you are after, but let me know otherwise:

[agent]
  omit_hostname = true

[[inputs.file]]
  ## Files to parse each interval.  Accept standard unix glob matching rules,
  ## as well as ** to match recursive files and directories.
  files = ["test.csv"]
  data_format = "csv"

  csv_header_row_count = 1
  csv_column_names = ["timestamp","ping"]
  csv_column_types = ["string", "float"]
  csv_timestamp_column = "timestamp"
  csv_timestamp_format = "2006-01-02 15:04:05"

[[outputs.file]]

and produced the following:

file ping=25 1647246066000000000
file ping=49 1647246069000000000
file ping=4 1647246072000000000
file ping=3.5 1647246075000000000
file ping=22 1647246078000000000
1 Like
3

Thank for the reply!

It shows me an error I had before but didn’t know how to resolve it:
image

All I want is having all the data in the csv in one table, so It would show me the timestamp and the consequent ping, e.g.:

00/00/00 00:00:00 42.5

Thanks!

Edit:

If I run telegraf with -test option will show me the next (don’t know if It’s useful info):

image

4

That error is coming from Telegraf trying to parse your CSV file. It is saying that the float column is empty. I would check your data and ensure that it is actually complete.

1 Like
5

Yeah! That It’s, at glance I thought the file was correct, but It looks like it isn’t. Anyway… how can I skip null values?

Thanks!

6

I have not used this option before, but try adding the following option to skip empty fields:

  ## Indicates values to skip, such as an empty string value "".
  ## The field will be skipped entirely where it matches any values inserted here.
  csv_skip_values = [""]
1 Like
7

Cheers mate! Is working as expected, owe you 100 thanks!