Remove newline from Telegraf-Kafka JSON

1

I’m trying to publish to Kafka using Telegraf and push to Druid. I’m running into errors parsing the JSON that is being published to Kafka. Upon investigation, I’ve noticed at the end of the JSON object, it adds a newline ("\n"). I believe this is the cause of the parsing issues using Tranquility Kafka.

I did a quick Nifi flow to check the queue of the Kafka topic and sometimes, some message are just completely empty/null. I believe the newline added at the end of each messages is causing this error. I would like to just quick test and see if it’s causing the error by removing the newline

I’ve been trying to find a way to remove the newline ("\n") and just cannot find it. Could someone direct me to the docs that allows me to remove the newlines at the end of the JSON object or tell me how I may avoid having these newlines? Thank you

2

I don’t think it is possible to remove the newline without modifying Telegraf, you would need to do it when reading from Kafka.

3

Why does even create the newline in the first place? And how would I go about modifying it? Can the next update allow to have properties file remove the newline?

4

On many outputs, it is possible to have multiple metrics per message which are delimited with a newline. I don’t think this is possible currently with the Kafka output though. If you want to test if removing it is enough to solve you problem here is where the newline is added.