Telegraf Plugin For Third Party Platform

Hi all,

I have the Telegraf agent running on a box collecting system data through InfluxDB/Kapacitor.

I’ve been asked to start logging more custom analytics to a 3rd party as-a-service platform. This can be acheived with http requests.

Being concerned with performance, and assuming Telegraf is designed to offload this type of work from the mission critical process, I thought I could use the existing agent + an output plugin to send the http requests for me.

However… I notice that there isnt an off-the-shelf http output plugin, and that there arent as many output plugins as input plugins. Usually not finding other people trying to solve my problem means I’m doing something a little… wonky.

(Im aware of this Telegraf output http and this https://github.com/influxdata/telegraf/pull/2491)

Does my solution make sense or am I abusing Telegraf?

Thanks for any insights,

Elmore

1 Like

Yeah, that was my post you referenced. I’m kinda wondering the same thing - why is there not a default HTTP output plugin?

Im guessing in general you need more control of the data formatting. In that PR you would have to configure a serializer to use, so if the consumer you are targetting didnt like any of the existing serializers you might end up writing one of those anyway…?

json output sounds perfect to me https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md

Also, the Datadog output uses HTTP as far as I can see (I have no go experience):

Well thanks for hijacking my thread there Jono…

I would think the point is that a plugin needs to format and transfer the data, and http only answers half the question. JSON may be perfect for you but even then the structure of the object may not suit everyone, so it would need to be configurable. My guess would be that there are relatively few use cases where thats actually more useful than a plugin tailored for the consumer.

BUT… I dont know much about it and my question still stands… if anyone who knows can comment on my use case I would be very appreciative.

Elmore

For formatting the data we have the various Serializers, we could probably use a more flexible version of the JSON serializer.

However, I suspect we can’t create one that is flexible enough for all 3rd party services. In this case you probably need to write a custom output plugin for formatting the data. It could still use the code from the http output plugin, either a copy of it or perhaps we can add some of the functionality to a shared location in the Telegraf code base.