Duplicate Data Point in InfluxDB

Nilesh786
Tue, 06 Feb 2024 08:49:32 +0000
Telegraf

Hello All,

My usecase to read from a json output file wriiten by K6 (an opensource load testing tool) and write every new json to influxdb via telegraf. I have deployed two pods in k8s cluster for this purpose, one is a k6-telegraf pod and another is an influxdb pod, below is my telegraf conf file

  [agent]
    interval      = "$POLL_INTERVAL"
    omit_hostname = true
    metric_batch_size = 1000
    metric_buffer_limit = 10000
    flush_interval = "5s"

  [[inputs.tail]]
    files = ["/outputs/result.json"]
    data_format = "json"
    from_beginning = false
    path_tag = ""
    json_name_key = "metric"
    json_string_fields = ["data_value", "data_tags_name", "data_tags_scenario", "data_tags_testrun", "data_tags_workflow", "type", "data_time"]

  [[processors.starlark]]
  source='''
  def apply(metric):
    if metric.fields["type"] == "Point":
        return metric
    return None
  '''

  [[outputs.influxdb]]
    urls = ["$INFLUX_HOST"]
    database = "$INFLUX_DATABASE"
    username = "$INFLUX_USERNAME"
    password = "$INFLUX_PASSWORD"
    timeout = "5s"

I observed that telegraf is writing twice the datapoints in influxdb, I was following this issue raised earlier here
I am not able to find any viable solution to my problem
Here is the influxdb output for one of the measurement this is the case for every measurement

I have tested for similar cases also but it worked perfectly(like writing to a example.out file and push it to influxdb using tail) but only for this case
Happy to get some help regarding this

6 posts - 3 participants