How can I use different Influx Database for various telegraf.conf files?

Telegraf
1

Hello,

I have my main telegraf.conf file in /etc/telegraf/ which contains the Influx database name and password and all my telegraf configuration files (about 30) are in /etc/telegraf/telegraf.d.

They just run SNMP queries again remote devices and puts the results my single influx database, it’s great and then into Grafana.

Most just start off with this:

[[inputs.snmp]]
  agents = [ "10.1.1.1:161" ]
  version = 2
  community = "password123"

  [inputs.snmp.tags]
    locationd = "NY"

  [[inputs.snmp.field]]
    name = "hostname"
    oid = "1.3.6.1.2.1.1.5.0"

  [[inputs.snmp.field]]
    name = "uptime"
    oid = "1.3.6.1.2.1.25.1.1.0"

  [[inputs.snmp.field]]
    name = "cpu1"
    oid = "1.3.6.1.2.1.25.3.3.1.2.1"

Now I have created a new influx database and a new telegraf.conf file in /etc/telegraf/telegraf.d. and want to send the data to this new database, but it won’t. Instead this new database gets all the new data from the other telegraf files. So I’m getting the same data in both databases now. It seems the telegraf files I have in /etc/telegraf/telegraf.d/ populate both databases now even though I’ve not told them to.

My new telegraf file that I wanted to do to this new database is configured like this:

[[outputs.influxdb]]
  urls = ["http://127.0.0.1:8086"]
  database = "newdb"
  skip_database_creation = true
  username = "user123"
  password = "passwoord"

[[inputs.snmp]]
  agents = [ "10.2.2.2.2:161" ]
  version = 2
  community = "password"
  [inputs.snmp.tags]
    location = "NY"

  [[inputs.snmp.field]]
    name = "temp"
    oid = "1.3.6.1.4.1.48690.5.3"

Very confused here, does that make any sense?

2

It makes perfectly sense. On startup telegraf reads config from all files and ‘merges’ that all together as if it would be 1 big config file.

So yeah, if you configured 2 outputs without anything extra it will send all metrics to both. You might want to look into metric filtering.

3

Ah I see!

Apparently I need a another telegraf daemon/instance to be installed so I use /etc/telegraf123/ for example?

Now how do I install a second instance of telegraf on the same server. :S

4

No, there is no need for that. See the docs I linked before, you can select which metrics go to which output.

5

Oh I see, I’m guessing you mean https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md#routing-metrics-to-different-outputs-based-on-the-input

I think it drops everyone then you point to your database and choose the metrics to include.

I just can’t work out from my snmp example on how I can do this (all very new to me).

It s great to know they merge as I thought I was going mad!

6

You need to think how you can differentiate the metrics. You could add a tag saying which db you want it to go, or base on the agent_host tag.

7

Do you mean like this? Using A Telegraf Gateway | InfluxData

I’ve looked at mine https://pastebin.com/hDbQwppy and the above link and can’t seem to understand how mine should look like, I’m not sure if you can give me an example to work on to help me on my way, I’d be so thankful.

8

There are indeed 2 options, use metric filtering as in my docs I linked earlier or put the name of the bucket in a tag. Both options require a tag to be set on the input to be able to differentiate. I still don’t know how you want to differentiate both inputs. (Hint based on the config file is not possible as they all get merged at startup)

9

Hey @Hipska thanks for stinking with me on this, as I know know from what you said all the config files all merge at startup so I need to somehow tag this new config to point to another database and struggling to write a config for this.

My of my configs in telegraf.d (main 1 in telegraf) so I created this new config that I wanted to point to a new database, all my configs include:

telegraf-1.conf

[[inputs.snmp]]
 agents = [ "1.1.1.4:161" ]
 version = 2
 community = "mypassword"
  [inputs.snmp.tags]
 location = "New York"
 
[[inputs.snmp.field]]
 name = "lightsensor"
 oid = "1.3.6.1.4.1.48690.5.3"

telegraf-2.conf

[[inputs.snmp]]
 agents = [ "1.1.2.4:161" ]
 version = 2
 community = "mypassword"
  [inputs.snmp.tags]
 location = "Paris"
 
[[inputs.snmp.field]]
 name = "lightsensor"
 oid = "1.3.6.1.4.1.48690.5.3"

telegraf-3.conf

[[inputs.snmp]]
 agents = [ "1.1.3.4:161" ]
 version = 2
 community = "mypassword"
  [inputs.snmp.tags]
 location = "London"
 
[[inputs.snmp.field]]
 name = "lightsensor"
 oid = "1.3.6.1.4.1.48690.5.3"

So in this example config 1 and 2 will use the default telegraf.conf file and merge with it and use the database configure int there. Config 3 is what I want to go into a new database.

On config 3 is it possible to add a tag to it called “custom123” which the other configs don’t have and be used as the differentiation and then send to the new database? Then any new configs I add I can use that tag to send to this new database and not get merged with the other configs and into the default database?

If so how should this config look?

10

Exactly as you already found here.

So the device-3.conf would look like this:

[[inputs.snmp]]
  alias = "device-3" # useful for debugging (the logs will show errors came from this config)
  agents = [ "1.1.3.4:161" ]
  version = 2
  community = "mypassword"
  [inputs.snmp.tags]
    location = "London"
    influxdb_database = "custom123" # here you specify if this needs to go to another DB

  [[inputs.snmp.field]]
    name = "lightsensor"
    oid = "1.3.6.1.4.1.48690.5.3"
11

Thanks what about the database password?

12

What is the problem with the password?

13

So I was wondering where I put the password for the new database, this is my reconfigured config, how does it look? I’ve set the DB output at the top and tagged it to use this database and not the main default one, am I missing anything?

[[outputs.influxdb]]
urls = [“http://192.168.100.123:8086”]
database = “routersdb”
skip_database_creation = true
username = “routeruser”
password = “password1234”

[[inputs.snmp]]
agents = [ “1.1.3.4:161” ]
version = 2
community = “password1”
[inputs.snmp.tags]
location = “London”
influxdb_database = “routersdb”

[[inputs.snmp.field]]
name = “lightsensor”
oid = “1.3.6.1.4.1.48690.5.3”

14

Yeah, you forgot the parts of tagpass and tagdrop as explained here.

15

Like this Hipska? As you can see Im trying not to use the current default database called “defaultdatabase” but use “newdatabase” instead. Also where to I tell the config file to use the “newdatabase” password I configured?

[[outputs.influxdb]]
urls = [“http://1.2.3.4:8086”]
#database I don’t wnat to use
database = “defaultdatabase”
[outputs.influxdb.tagdrop]
influxdb_database = [“*”]

[[outputs.influxdb]]
urls = [“http://1.2.3.4:8086”]
#database I want too use
database = “newdatabase”
#database I don’t wont to use
tagexclude = [“defaultdatabase”]
[outputs.influxdb.tagpass]
influxdb_database = [“newdatabase”]

[[inputs.snmp.field]]
name = “lightsensor”
oid = “1.3.6.1.4.1.48690.5.3”
[inputs.disk.tags]
influxdb_database = “newdatabase”

16

If it needs a password, you need to put that in the corresponding outputs.influxdb setting.

But, note if they are both on the same server (with same credentials), you could use the database_tag setting and have only one output configured…

17

They are 2 different databases, the main one and this new one.

I just want this new telegraf file to use this new databases which has a different username and password to access it.