# Do batch import timeouts result in dropped data?

**URL:** <https://community.influxdata.com/t/do-batch-import-timeouts-result-in-dropped-data/9578>\
**Category:** Telegraf\
**Tags:** influxdb, telegraf\
**Created:** [April 24, 2019, 7:08pm UTC](https://community.influxdata.com/t/do-batch-import-timeouts-result-in-dropped-data/9578 "2019-04-24T19:08:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![John\_Sobanski](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/john_sobanski/32/2777_2.png) [@John\_Sobanski](https://community.influxdata.com/u/John_Sobanski)\
**Post date:** [April 24, 2019, 7:08pm UTC](https://community.influxdata.com/t/do-batch-import-timeouts-result-in-dropped-data/9578/1 "2019-04-24T19:08:18Z")

</div>

I have a Telegraf node that ingests a newline delimited JSON file and emits to influxdb.

I have the following configuration for **inputs.tail** :

```auto

[[inputs.tail]]
   files = ["./*.txt"]
   from_beginning = true
   json_time_key = "time"
   json_time_format = "2006-01-02T15:04:05Z07:00"
   json_name_key = "foo"
   tag_keys = ["foo","bar","stooge"]
   data_format = "json"

```

This input configuration works as expected. Telegraf ingests a 2GB file of ~3M JSON data points without issue.

I then decided to get fancy and emit to a separate DB per **stooge**.

```auto
[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8086"]
  database = "MOE"
  [outputs.influxdb.tagpass]
  stooge = ["MOE"]

[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8086"]
  database = "LARRY"
  [outputs.influxdb.tagpass]
  stooge = ["LARRY"]

[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8086"]
  database = "CURLEY"
  [outputs.influxdb.tagpass]
  stooge = ["CURLEY"]

```

This works as expected, except that I see some errors.

```auto
2019-04-24T18:52:34Z E! [agent] Error writing to output [influxdb]: could not write any address
2019-04-24T18:52:36Z E! [outputs.influxdb] when writing to [http://127.0.0.1:8086]: Post http://127.0.0.1:8086/write?db=MOE: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2019-04-24T18:52:39Z E! [outputs.influxdb] when writing to [http://127.0.0.1:8086]: Post http://127.0.0.1:8086/write?db=LARRY: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2019-04-24T18:52:39Z D! [outputs.influxdb] buffer fullness: 10000 / 10000 metrics. 
2019-04-24T18:52:39Z E! [agent] Error writing to output [influxdb]: could not write any address
2019-04-24T18:55:31Z W! [agent] output "influxdb" did not complete within its flush interval

```

I have two questions

- Are these timeouts of any concern? Or will telegraf re-try to emit the data until it receives a successful WRITE ACK from InfluxDB?
- Should I tune the parameters, e.g. decrease the batch size, increase the buffer size, and increase the timeouts or is this a bad idea?

---

<div class="post-metadata">

**Author:** ![MarcV](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/marcv/32/7676_2.png) [@MarcV](https://community.influxdata.com/u/MarcV)\
**Post date:** [April 24, 2019, 7:48pm UTC](https://community.influxdata.com/t/do-batch-import-timeouts-result-in-dropped-data/9578/2 "2019-04-24T19:48:01Z")

</div>

Hi , your first question …

> [@John\_Sobanski](#):
>
> D! [outputs.influxdb] buffer fullness: 10000 / 10000

this means metrics are dropped … and the buffer should be increased

```
  ## For failed writes, telegraf will cache metric_buffer_limit metrics for each
  ## output, and will flush this buffer on a successful write. Oldest metrics
  ## are dropped first when this buffer fills.
  ## This buffer only fills when writes fail to output plugin(s).
  metric_buffer_limit = 10000

```