Using Telegraf(logparser plugin) to send GeoPoint info to Elasticsearch

1

Hi,

I have a measurement setup to record various measurements that are processed by telegraf and output to elasticsearch. I am using logparser plugin to parse the input and create measurements in elasticsearch. I have no issues with datatypes such as number or string. When i try to use data type geo_point(as supported by logstash) telegraf shows up parsing error as below

2017-06-13T20:26:51Z E! Error parsing 50.354,16.662 to time layout [geo_point]: parsing time “50.354,16.662” as “geo_point”: cannot parse “50.354,16.662” as "geo_point"
2017-06-13T20:26:51Z E! Error parsing log line: Metric cannot be made without any fields

Here below is my configuration for logparser input

[[inputs.logparser]]
files = ["/home/biadmin/files/input.log"]
from_beginning = true
[inputs.logparser.grok]
patterns = [’%{TIMESTAMP_ISO8601:timestamp:ts-“2006-01-02 15:04:05”} LatLon=%{NOTSPACE:location:geo_point}’]
measurement = “GeoPoint”

And a sample input from log file
2017-06-13 18:20:04 LatLon=50.354,16.662
2017-06-13 18:20:14 LatLon=50.354,16.662
2017-06-13 18:20:24 LatLon=50.354,16.662

I also tried to create the measurement as geo point in elasticsearch and send input as string. However a new measurement with type string gets created.

It would be very helpful to know if telegraf supports sending geographic coordinates to elastic search and if so, how can this be achieved.

Thanks!

2

@user_1985 Being that you are writing to ElasticSearch I would think that logstash might be better for you in this case. We use an open source Grok implementation that may or may not support everything logstash does. Why are you using telegraf instead of logstash?

3

@jackzampolin I use the statsd plugin in telegraf to aggregate data and send measurements. One of the reasons i was using telegraf. I managed to fix the issue. It was a mistake in telegraf.conf file. I did not specify the right measurement name. With the change i had to create a geo_point type manually using
PUT my_index
{
“mappings”: {
“geoip”: {
“properties”: {
“location”: {
“type”: “geo_point”
}
}
}
}
}

and modify the config file as below :

from_beginning = true
[inputs.logparser.grok]
patterns = [’%{TIMESTAMP_ISO8601:timestamp:ts-“2006-01-02 15:04:05”} LatLon=%{NOTSPACE:location:geo_point}’]
measurement = “geoip”

Thanks a lot for the help!

1 Like