I come back with my progression and share my experience.
the real problem was the “,” (comma) for the decimal number in the log file.
it is possible to extract them with this custom_pattern:
BASE10NUMCOMMA (?<![0-9,+-])(?>[+-]?(?:(?:[0-9]+(?:\,[0-9]+)?)|(?:\,[0-9]+)))
i didn’t create this custom_pattern, i used the BASE10NUM existing pattern and replaced de “.” (dot) by the “,” (comma) 
And i could extract the datas from this log line:
2018-01-19 07:00:17;VV_TU_VENUS_P11_HLU00;VPD83T3:6000144000000010607d2411e815bb92;0,1;0,4;0;148;0
With this pattern:
%{DATEFORM:date};%{DATA:VirtualVolume:tag};%{VPID:vpid:string};%{BASE10NUMCOMMA :FeLuOpsCountByS};%{BASE10NUMCOMMA:FeLuReadKbs};%{BASE10NUMCOMMA:FeluReadWriteKbs};%{BASE10NUMCOMMA:FeLuReadLatRecentAverage};%{BASE10NUMCOMMA:FeLuWritedLatRecentAverage}
And the custom patterns associated:
DATEFORM %{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"}
VPID %{DATA:vpd}:%{DATA:id}
BASE10NUMCOMMA (?<![0-9,+-])(?>[+-]?(?:(?:[0-9]+(?:\,[0-9]+)?)|(?:\,[0-9]+)))
I could get the right result:
{
"date": [
[
"2018-01-19 07:00:17"
]
],
"timestamp": [
[
"2018-01-19 07:00:17"
]
],
"YEAR": [
[
"2018"
]
],
"MONTHNUM": [
[
"01"
]
],
"MONTHDAY": [
[
"19"
]
],
"HOUR": [
[
"07",
null
]
],
"MINUTE": [
[
"00",
null
]
],
"SECOND": [
[
"17"
]
],
"ISO8601_TIMEZONE": [
[
null
]
],
"VirtualVolume": [
[
"VV_TU_VENUS_P11_HLU00"
]
],
"vpid": [
[
"VPD83T3:6000144000000010607d2411e815bb92"
]
],
"vpd": [
[
"VPD83T3"
]
],
"id": [
[
"6000144000000010607d2411e815bb92"
]
],
"BASE10NUMCOMMA": [
[
"0,1"
]
],
"FeLuReadKbs": [
[
"0,4"
]
],
"FeluReadWriteKbs": [
[
"0"
]
],
"FeLuReadLatRecentAverage": [
[
"148"
]
],
"FeLuWritedLatRecentAverage": [
[
"0"
]
]
}
with the patterns as they are, if i don’t use modifiers, all the data are considered as “string” in the influxDB.
Check it in the influxdb shell:
#> show field keys on vplex
name: virtual_volume_log
fieldKey fieldType
-------- ---------
FeLuOpsCountByS string
FeLuReadKbs string
FeLuReadLatRecentAverage string
FeLuWritedLatRecentAverage string
FeluReadWriteKbs string
VirtualVolume string
date string
id string
vpd string
vpid string
So all datas are unusable in grafana, unable to graph, convert, etc. i’m able just to print as table the contents of the influxdb database.
When trying to use modifiers and convert the datas to float when injecting in influxdb, i have parsing and convert errors, it seems that influxdb doesn’t understand “,” (comma) decimal.
So i have replace “,” (comma) by “.” (dot) in my log file to get lines as this:
2018-01-19 07:00:17;VV_TU_VENUS_P11_HLU00;VPD83T3:6000144000000010607d2411e815bb92;0.1;0.4;0;148;0
And i have personnalize my pattern with the BASE10NUM pattern core and add modifier “float” to convert data as float when injecting in influxDB:
patterns = ['%{DATEFORM:date};%{DATA:VirtualVolume:tag};%{VPID:vpid:string};%{BASE10NUM:FeLuOpsCountByS:float};%{BASE10NUM:FeLuReadKbs:float};%{BASE10NUM:FeluReadWriteKbs:float};%{BASE10NUM:FeLuReadLatRecentAverage:float};%{BASE10NUM:FeLuWritedLatRecentAverage:float}']
custom_patterns = '''
DATEFORM %{TIMESTAMP_ISO8601:date:ts-"2006-01-02 15:04:05"}
VPID %{HOSTNAME:vpd}:%{WORD:id}
'''
And i could get float fileds in influxDB:
#> show field keys on vplexdoted
name: virtual_volume_log
fieldKey fieldType
-------- ---------
FeLuOpsCountByS float
FeLuReadKbs float
FeLuReadLatRecentAverage float
FeLuWritedLatRecentAverage float
FeluReadWriteKbs float
date string
id string
vpd string
vpid string
But with this configuration, Graph in grafana seems doesn’t work…
i can extract datas in grafana as table, but as soon as i try to graph datas with graph panel, Nothing appear.
Forever i can fin the, tag, the measurements, etc.
Thanks in advance
