Telegraf [[inputs.influxdb_listener]] write data

Telegraf
1

Hi,

This post is more a question then a problem.

I Node Red i collect all the data from different PLC’s. From Node Red we go through a [[inputs.influxdb_listener]] plugin to telegraf Agent and then to InfluxDB.
Everything works fine but before i upscale the system i want to know if the basic are good or need improvements and the 2 points i thought about are the following:

  1. I have opted to use a Telegraf Agent because in the future we want to use different servers for each departments and i read that multiple Telegraf can write to a single InfluxDB server. But is there a certain protocol in use for (that telegraf is using checking of someone else is already writing)or is it the same if i write directly from different Node red clients to InfluxDB (without looking if someone is sending data already).
    2.For InfluxDB it’s scales up the performances if you sort your tags, timestamp, write in batches,…
    In my case do i still have doing all that before i write to telegraf or is telegraf doing that already with the data i give him. And do i best write in batches or intervals to telegraf or can i just input new data when i finished collecting them in node red regardless the size?

I hope i make some sense :sweat_smile:

Thanks at advance!
Kind Regards,
Ward

2

Also my telegraf.conf settings

[agent]
interval = “10s”
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = “0s”
flush_interval = “10s”
flush_jitter = “0s”
precision = “”
omit_hostname = true

#[[inputs.influxdb_listener]]
service_address = “:8186”
read_timeout = “10s”
write_timeout = “10s”
max_body_size = 0
max_line_size = 0
database_tag = “DBTag”

#[[outputs.influxdb]]
urls = [“http://localhost:8086”] //would become the ip adress off the InfluxDB eventually
database = “telegraf”
database_tag = “DBTag”
exclude_database_tag = true

3

you don’t need to worry about writing to influxDB unless you are fetching duplicate data (different gatherers capture and write the exact same data), which is useless and causes updates in InfluxDB.
You can write as many points as you want, even to the “Telegraf Gateway” (influxDB Listener") without worrying about who is already writing.

Telegraf has some advantages as it manages:

  • data batching
  • tag/fields sorting

About timestamp sorting, I’m not sure. But I don’t really worry about it and I have no clue about the performance difference.
That said, I don’t even think you can achieve it (globally speaking), especially if you have multiple telegraf writing to the database.

About writing single points vs batches from NodeRed to Telegraf, that’s all up to what you need and how you do it, but in general I think using batches might be more performing. (it creates less traffic for sure)
some sample questions:
do you provide the timestamp yourself?
do you need all the points fetched in the same “round” to have the same timestamp?
Is it a problem to buffer the points before sending them?