Telegraf only writes one entry to influxdb

Telegraf
,
1

I have the latest version of influxdb and telegraf installed on my linux machine.
I already had questdb running and did fill it with telegraf without problems.

Now I wanted to also will a influxdb database but I have problems to fill the database.
Telegraf is connected to various mqtt broker to collect data.

This is my telegraf config:

# Configuration for Telegraf agent
[agent]
  ## Default data collection interval for all inputs
  interval = "500ms"
  hostname = "qdb"
  debug = true
  quiet = false
  logfile = "./telegraf.log"
  metric_batch_size = 2000


# -- OUTPUT PLUGINS -- #
[[outputs.influxdb_v2]]
  urls = ["http://localhost:8086"]
  token = "TOKEN"
  bucket = "data"
  organization = "myorg"
 
[[outputs.socket_writer]]
  # Write metrics to a local QuestDB instance over TCP
  address = "tcp://localhost:9009"

# -- INPUT PLUGINS -- #
[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

[[inputs.mqtt_consumer]]
servers = ["tcp://xxx.xxx.xxx.xxx:xxxx"]
topics = [
    "telegraf"
	]
data_format = "influx"

My problem is, that telegraf only writes on entry to influxdb but all to questdb. When one broker sends 80 entries, only one gets added to influxdb but questdb will have all 80. I also tryed to remove the writing to questdb but that didn’t help.

Thats the latest telegraf log (Here the brokers where sending many messages, but the same error also happens when only one message per second is sended):

2021-09-22T13:11:15Z I! Loaded inputs: mqtt_consumer (8x)
2021-09-22T13:11:15Z I! Loaded aggregators: 
2021-09-22T13:11:15Z I! Loaded processors: 
2021-09-22T13:11:15Z I! Loaded outputs: influxdb_v2 socket_writer
2021-09-22T13:11:15Z I! Tags enabled: host=qdb
2021-09-22T13:11:15Z I! [agent] Config: Interval:500ms, Quiet:false, Hostname:"qdb", Flush Interval:10s
2021-09-22T13:11:15Z D! [agent] Initializing plugins
2021-09-22T13:11:15Z D! [agent] Connecting outputs
2021-09-22T13:11:15Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2021-09-22T13:11:15Z D! [agent] Successfully connected to outputs.influxdb_v2
2021-09-22T13:11:15Z D! [agent] Attempting connection to [outputs.socket_writer]
2021-09-22T13:11:15Z D! [agent] Successfully connected to outputs.socket_writer
2021-09-22T13:11:15Z D! [agent] Starting service inputs
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:15Z I! [inputs.mqtt_consumer] Connected [tcp://xxx.xxx.xxx.xxx:xxxx]
2021-09-22T13:11:20Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 258.122367ms
2021-09-22T13:11:20Z D! [outputs.socket_writer] Buffer fullness: 2824 / 10000 metrics
2021-09-22T13:11:21Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 283.759389ms
2021-09-22T13:11:21Z D! [outputs.socket_writer] Buffer fullness: 1355 / 10000 metrics
2021-09-22T13:11:21Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 628.340928ms
2021-09-22T13:11:21Z D! [outputs.influxdb_v2] Buffer fullness: 5866 / 10000 metrics
2021-09-22T13:11:22Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 434.808884ms
2021-09-22T13:11:22Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:22Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 818.051315ms
2021-09-22T13:11:22Z D! [outputs.socket_writer] Buffer fullness: 4159 / 10000 metrics
2021-09-22T13:11:22Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 661.796128ms
2021-09-22T13:11:22Z D! [outputs.socket_writer] Buffer fullness: 4000 / 10000 metrics
2021-09-22T13:11:22Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 243.637718ms
2021-09-22T13:11:22Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:23Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 1.035787335s
2021-09-22T13:11:23Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:23Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 165.858905ms
2021-09-22T13:11:23Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:23Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 680.346228ms
2021-09-22T13:11:23Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:23Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 208.873579ms
2021-09-22T13:11:23Z D! [outputs.influxdb_v2] Buffer fullness: 5334 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 14.868442ms
2021-09-22T13:11:24Z D! [outputs.socket_writer] Buffer fullness: 2122 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 28.365693ms
2021-09-22T13:11:24Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 198.263252ms
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 178.448985ms
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 200.774778ms
2021-09-22T13:11:24Z D! [outputs.socket_writer] Buffer fullness: 2402 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 26.760135ms
2021-09-22T13:11:24Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 287.383646ms
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 121.701401ms
2021-09-22T13:11:24Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 208.040011ms
2021-09-22T13:11:24Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 97.550648ms
2021-09-22T13:11:25Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 238.320594ms
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 17.744744ms
2021-09-22T13:11:25Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 238.926477ms
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 61.014386ms
2021-09-22T13:11:25Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 150.991203ms
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 159.392803ms
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Buffer fullness: 5982 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 28.874232ms
2021-09-22T13:11:25Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 55.740035ms
2021-09-22T13:11:25Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 202.457343ms
2021-09-22T13:11:25Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 43.850559ms
2021-09-22T13:11:26Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 267.923321ms
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 48.026967ms
2021-09-22T13:11:26Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 185.483972ms
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 82.994119ms
2021-09-22T13:11:26Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 150.862366ms
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 82.999419ms
2021-09-22T13:11:26Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 259.553403ms
2021-09-22T13:11:26Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 102.254064ms
2021-09-22T13:11:27Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 225.439509ms
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 61.374593ms
2021-09-22T13:11:27Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 156.451577ms
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 56.570799ms
2021-09-22T13:11:27Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 143.244062ms
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 47.46042ms
2021-09-22T13:11:27Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 295.974316ms
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 117.166078ms
2021-09-22T13:11:27Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 220.049767ms
2021-09-22T13:11:27Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 174.092493ms
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 115.772652ms
2021-09-22T13:11:28Z D! [outputs.socket_writer] Buffer fullness: 2095 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 60.881804ms
2021-09-22T13:11:28Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 222.2657ms
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 77.354965ms
2021-09-22T13:11:28Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 180.149806ms
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 202.11471ms
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 89.594878ms
2021-09-22T13:11:28Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 105.057341ms
2021-09-22T13:11:28Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 296.785021ms
2021-09-22T13:11:28Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 177.929244ms
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 110.226403ms
2021-09-22T13:11:29Z D! [outputs.socket_writer] Buffer fullness: 2534 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 84.602815ms
2021-09-22T13:11:29Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 223.056397ms
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 39.204008ms
2021-09-22T13:11:29Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 211.679127ms
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 112.185976ms
2021-09-22T13:11:29Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 291.511864ms
2021-09-22T13:11:29Z D! [outputs.influxdb_v2] Buffer fullness: 6000 / 10000 metrics
2021-09-22T13:11:30Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 155.464416ms
2021-09-22T13:11:30Z D! [outputs.influxdb_v2] Buffer fullness: 5585 / 10000 metrics
2021-09-22T13:11:30Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 44.404214ms
2021-09-22T13:11:30Z D! [outputs.socket_writer] Buffer fullness: 2000 / 10000 metrics
2021-09-22T13:11:30Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 156.981196ms
2021-09-22T13:11:30Z D! [outputs.influxdb_v2] Buffer fullness: 4000 / 10000 metrics
2021-09-22T13:11:40Z D! [outputs.socket_writer] Wrote batch of 2000 metrics in 11.992402ms
2021-09-22T13:11:40Z D! [outputs.socket_writer] Buffer fullness: 0 / 10000 metrics
2021-09-22T13:11:40Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 106.700383ms
2021-09-22T13:11:40Z D! [outputs.influxdb_v2] Wrote batch of 2000 metrics in 144.557663ms
2021-09-22T13:11:40Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2021-09-22T13:11:50Z D! [outputs.socket_writer] Buffer fullness: 0 / 10000 metrics
2021-09-22T13:11:50Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2021-09-22T13:12:00Z D! [outputs.socket_writer] Buffer fullness: 0 / 10000 metrics
2021-09-22T13:12:00Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2

In your log file, I don’t see any error, therefore the data are being written as expected.

I’ve got few questions:

  1. How many points are you sending in 500ms?
  2. What’s your configured precision? (I don’t see it so by default it’s seconds)
  3. Are the gathered data all equal (except for timestamp)?
  4. Do you have the timestamp in the messages themselves? if yes can you use the timestamp coming from the data itself?

if you don’t provide a timestamp, telegraf will generate one by itself, equal for all points.
If you have points of the same series, with the same timestamp you will get an update, not an insert, I gave some suggestions about it in this post

Once you go through the questions and a bunch of replies in the linked post you should be able to understand what’s going on by yourself.
Le t me know if you need more info

3
  1. I tried to send 1000 each 500ms, 100 each 500ms, 10 each 500ms and 80 each 10s.
  2. I don’t have it configured. First I thought that because the timestamp will be the same, the data will only get updated and not inserted but when I send 80 entries to the broker every 10s it wount write any data except one entry. Only after restarting the broker or telegraf a new entry will be written to influxdb.
  3. The opposide. The data is all random at 10 fields, but the timestamp can be the same because I just generate test data and like every 500’ish entry have the same timestamp
  4. Yes, I have the timestamp in the message

I will try to set a precision and see what will happen. (I think not much because even with 10s delay between sending new data no entry is written)

4

Ok I got it…

I saw that I do create a set of entries before sending and after that I don’t update the timestamp within the entries before sending…

So the error is on my part