Non fixed tag_keys array

InfluxDB 2
1

Hi all,

Is there a way to tell to a telegraf input plugin such as kafka consumer, not to drop tags that are not specified in the tag_keys property for json data format?

For example, with reference to this json

    {
        "measurement": "my-measurement",
        "tags": {
            "tag1": "my-tag1",
            "tag2": "my-tag2"
        },
        "fields": {
            "my-field": "my-value"
        }
    }

if I don’t set tag1 and tag2 in the telegraf.conf, the kafka-consumer plugin will ignore both.

Is there a way to skip this behaviour?

Thank you

3

Hi @vladbragoi,

I’m sorry, but I do not understand what you are asking. Can you elaborate a little and please share your Telegraf configuration.

Thanks

4

Hi @rawkode,

For kafka consumer plugin, if I have a topic with the following json data:

    {
        "measurement": "my-measurement",
        "tags": {
            "tag1": "my-tag1",
            "tag2": "my-tag2",
            "tag3": "my-tag3"
        },
        "fields": {
            "my-field": "my-value"
        }
    }

and a Telegraf configuration such as:

[[inputs.kafka_consumer]]
    brokers = ["kafka:9093"]
    topics = ["my-topic"]
    tag_keys = ["my-tag1", "my-tag2"]

I found out that the tag “my-tag3” will be ignored for all entries.

Considering a context in which the tag keys are dynamically created and cannot be determined at the configuration step, is there a way to take them all without the need to be specified in the “tag_keys” property?