Telegraf processor.starlark illegal token

Telegraf
,
1

Hi everyone,

I am currently setting up Telegraf to use a Starlark processor script to add GeoLite2 geolocation data to my metrics. However, I keep encountering a syntax error at line 17, column 1, which says:

[telegraf] Error running agent: could not initialize processor processors.starlark: :17:1: got illegal token, want primary expression

Here is the relevant part of my telegraf.conf configuration file:

[[processors.starlark]]
  source = '''
def apply(metric):
    for point in metric:
        if 'geoip' in point.tags:
            city, country, latitude, longitude = geoip_lookup(point.tags['geoip'])
            point.tags['city'] = city
            point.tags['country'] = country
            point.tags['latitude'] = latitude
            point.tags['longitude'] = longitude
    return metric

def geoip_lookup(ip):
    db_path = '/usr/share/GeoIP/GeoLite2-City.mmdb'
    geolite2 = Geolite2(db_path)
    location = geolite2.lookup(ip)
    return location['city'], location['country'], location['latitude'], location['longitude']

class Geolite2:
    def __init__(self, db_path):
        import geoip2.database
        self.reader = geoip2.database.Reader(db_path)

    def lookup(self, ip):
        response = self.reader.city(ip)
        return {
            'city': response.city.name,
            'country': response.country.name,
            'latitude': response.location.latitude,
            'longitude': response.location.longitude,
        }
'''

I’ve made sure that:

  • The Starlark script is enclosed within triple single quotes (''').
  • The indentation is consistent within the script.
  • There are no invisible characters or extra spaces at the beginning of the lines.

Despite these checks, I still get the same error. I also ran telegraf --config telegraf.conf --test, but the error persists.

If anyone has experienced a similar issue or can spot what might be going wrong, I would greatly appreciate your help.

Thank you!

2

If you count down to the 17th line, you will come across your class. Class is not supported by starlark. See the language overview with features are not supported:

The following Python features are not supported:

  • class (see struct function).
3

@stxdig to add onto this, you cannot access the filesystem with starlark! That’s one of the design goals of the language to avoid security issues when using scripts in configs…

What you probably want is to use the execd processor and run a python script with it… Alternatively, you could open a feature request for Telegraf to add a geoip processor…