Telegraf CSV input formatting problem

Telegraf
, ,
1

I want to import a csv file every 5 seconds to my influxdb. My input part of my config file looks as follows:

[[inputs.http]]
  urls = ["http://192.168.X.X/getvar.csv"]
  data_format = "csv"
  csv_header_row_count = 1
  csv_measurement_column = ["name"]
  csv_tag_columns = ["id"]
  csv_column_types = ["string","float","string","string","string","float"]

and the CSV has the following structure:

name id desc type access val
CTA_B91_Temp 1760 B91 Temp. - Quelle (WP) Eintritt [°C] REAL RW 6.03

​

However, the Docker log gives my this error:

E! [inputs.http] Error in plugin: [url=http://192.168.X.X/getvar.csv]: column type: parse float error strconv.ParseFloat: parsing "val": invalid syntax

and the influxdb data explorer this one:

 unsupported input type for mean aggregate: string

Did I specify the csv_column_types wrong?

2

Hi @Globgogabgalab,
hmm, I am just trying this example on my setup using the file parser which seems to be working fine. Can you check via postman what the full body of the message is you getting back using http? I am wondering if there might be an extra row of metadata before the row headers.

Here was my example which worked:

3

Hi Jay thanks for the quick answer! I have just checked the CSV file again and noticed that the numbers is the val column are all in quotes. I suppose that they are strings. Is there a way to read such data or convert the val column into floats?

Sample CSV:
“CTA_B92_Temp”,1766,“B92 Temp. - Quelle (WP) Austritt [°C]”,REAL,RW,“2.53”

4

Hi @Globgogabgalab,
You could make use of this processor to convert your string value to a float: telegraf/README.md at master · influxdata/telegraf · GitHub

So essentially in your input plugin read the value in as type string → then convert to float in the processor:

[[processors.converter]]
  ## Fields to convert
  ##
  ## The table key determines the target type, and the array of key-values
  ## select the keys to convert.  The array may contain globs.
  ##   <target-type> = [<field-key>...]
  [processors.converter.fields]
    float = ["val"]
1 Like
5

Thank you for taking the time to help me out! I copied your code at the bottom of my config file and get this error:

2022-01-19T14:08:57Z E! [processors.converter] error converting to float [string]: val

2022-01-19T13:51:32Z E! [outputs.influxdb_v2] Failed to write metric (will be dropped: 422 Unprocessable Entity): unprocessable entity: failure writing points to database: partial write: field type conflict: input field “val” on measurement “http” is type float, already exists as type string dropped=1000

6

@Globgogabgalab
Can you provide the first few lines of your csv file in raw format here:

```text
put your csv file here
```
7 
name,id,desc,type,access,val
"CTA_B91_ChannelType",1757,"",UDINT,R,"14"
"CTA_B91_PrbType",1758,"Fühlertyp -  B91 (0-3:NTC 1,10,10,50k; 4:PT1000, 999:UI cfg)",INT,RW,"1"
"CTA_B91_Res",1759,"Siemens NTC 10K - Quelle Ein [Ohm]/xx",REAL,RW,"487.64"
"CTA_B91_Temp",1760,"B91 Temp. - Quelle (WP) Eintritt  [°C]",REAL,RW,"5.78"
"CTA_B91_TempMin",1761,"B91 - Langzeit Minimalwert",REAL,RW,"3.61"
"CTA_B91_TempMin_Daily",1762,"B91 - Tages Minimalwert",REAL,RW,"5.41"
"CTA_B92_ChannelType",1763,"",UDINT,R,"14"
"CTA_B92_PrbType",1764,"Fühlertyp -  B92 (0-3:NTC 1,10,10,50k; 4:PT1000, 999:UI cfg)",INT,RW,"1"
"CTA_B92_Res",1765,"Siemens NTC 10K - Quelle Aus [Ohm/xx",REAL,RW,"572.94"
"CTA_B92_Temp",1766,"B92 Temp. - Quelle (WP) Austritt  [°C]",REAL,RW,"2.53"
"CTA_B92_TempMin",1767,"Min Quellentemperatur aktuell",REAL,RW,"-0.90"
"CTA_B92_TempMin_Daily",1768,"B92 - Tages Minimalwert",REAL,RW,"0.88"
"CTA_B92_correction",1769,"Korrektur Fühlerwert B92 (Verdampfer aus) - Abgleich Manuell",REAL,RW,"0.00"

here you go. Thanks for the support.

Here is the entire csv file:
https://www.dropbox.com/s/0jyt6bqahg9vrge/getvar.csv?dl=0

1 Like
8

InfluxDB does not like it when the data type of an already existing metric changes, hence this error message.
Can you delete the InfluxDB bucket and re-read it again?
If not, you would have to rename the field with the processors.rename plugin as a workaround - as far as I know.

Plugin directory | Telegraf Documentation

[[processors.rename]]
  [[processors.rename.replace]]
    field = "val"
    dest = "value"

PS: Be aware of the order if you run more than one processors plugin:

https://docs.influxdata.com/telegraf/v1/configuration/

2 Likes
9

It worked. Thanks for the great support. Very much appreciated

1 Like
10

I have created conf file and CSV file but I see only the last row in the influxdb . Can someone help:
[[inputs.file]]

Commands array

files = [“/etc/telegraf/scripts/file.txt”]
csv_header_row_count = 1
csv_delimiter = “,”
data_format = “csv”

and here is the file:

name,id,desc
movie,1,test
part,2,test1
car,3,test2
bus,4,test3

image
image1512×684 55.1 KB

**I am using distinct funtion in InfluxDB , because mean function gives me " ```
unsupported input type for mean aggregate: string

11

Hi @Dimitar_Kinchev,
You will need to parse your id as tag since the timestamp will be equivalent for all lines meaning the values will overwrite one another.