Ingesting stream of messages from RabbitMQ into Telegraf's zipkin plugin

influxdb
telegraf

#1

Hi,

I’m interested in using influxdb as zipkin’s backend. Going through various blogs and articles, I found that Telgraf already has a plugin but it collects the spans from a rest endpoint.

We have a push mechanism, where spans are pushed to RabbitMQ. Does anyone have any idea how to ingest rabbitmq messages into influxdb via any telegraf’s zipkins plugin?

Thanks in advance,
Gagan


#2

How do your spans look in the RabbitMQ messages? I think the basic idea would be to translate them to match the span encoding used by the zipkin input.


#3

Thanks for the response @daniel

The spans are generated in Zipkins format by default. The conversion to the format expected by Telegraf’s Zipkins input plugin can be done easily. The concern is to how to feed the input plugin from rabbitmq messages.


#4

I think right now there are two methods you could use:

The first is to parse the rabbitmq data with the JSON data format, and then massage it to be the same as the zipkin input creates. Honestly, I’m not sure if our JSON support is capable enough, and you would definitely need the new JSON features from the upcoming Telegraf 1.8.

The second method would be to have Telegraf receive the data in the amqp_consumer and POST it back to the zipkin input:

rabbitmq -> [amqp_consumer -> http output] -> [zipkin input -> influxdb]

This could be done with two Telegraf’s, one for each bracketed section, or with creative use of measurement filtering and tag based routing you could do this with a single Telegraf instance.