TLS configuration with telefgraf and influxdb behind nginx proxy TLS Termination endpoint

Telegraf
,
1

I am trying to use an existing https terminating nginx reverse proxy for nginx, since that is already handling all my other services just fine including let’s encrypt cert renew.

So the infrastructure (roughly speaking) should look something like this:

telegraf <==https==> nginx reverse proxy <--http--> influxdb

Relevant influx section of my nginx.conf

server {
# adding http2 does not make a difference
		listen 8086 ssl;
		listen [::]:8086 ssl;
        server_name influx.some.where;

		ssl	 on;
		ssl_certificate		 /etc/letsencrypt/live/some.where/fullchain.pem;
		ssl_certificate_key	 /etc/letsencrypt/live/some.where/privkey.pem;
		ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384';
		ssl_protocols		TLSv1.1 TLSv1.2;
		ssl_prefer_server_ciphers on;
		ssl_session_timeout	 5m;
		
        location / {
			proxy_pass http://influx;
			proxy_redirect off;
			proxy_http_version 1.1;
			proxy_max_temp_file_size 10m;
			proxy_connect_timeout 20;
			proxy_send_timeout 20;
			proxy_read_timeout 20;
			proxy_set_header Host $host;
			proxy_set_header Upgrade $http_upgrade;
			proxy_set_header Connection keep-alive;
			proxy_set_header X-Real-IP $remote_addr;
			proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
			proxy_set_header X-Forwarded-Proto http;
			proxy_set_header X-Original-For $proxy_add_x_forwarded_for;
			proxy_set_header X-Original-Proto https;
			proxy_cache_bypass $http_upgrade;
        }

}

I extracted the influxdb.conf from the official docker container, where the http section reads like this:

[http]
  enabled = true
  bind-address = ":8086"
  auth-enabled = false
  log-enabled = true
  suppress-write-log = false
  write-tracing = false
  pprof-enabled = true
  debug-pprof-enabled = false
  https-enabled = false
  https-certificate = "/etc/ssl/influxdb.pem"
  https-private-key = ""
  max-row-limit = 0
  max-connection-limit = 0
  shared-secret = ""
  realm = "InfluxDB"
  unix-socket-enabled = false
  unix-socket-permissions = "0777"
  bind-socket = "/var/run/influxdb.sock"
  max-body-size = 25000000
  access-log-path = ""
  max-concurrent-write-limit = 0
  max-enqueued-write-limit = 0
  enqueued-write-timeout = 30000000000

which should be fine, since influxdb should only handle plain http but not https.

But there are options for TLS in the client too and the description is very minimal.

What should I put into telegraf.conf

[[output.influxdb]]
urls = ["https://influx.some.where:8086"]
# --- SNIP. --- 
  ## Optional TLS Config for use on HTTP connections.
#tls_ca = "/etc/telegraf/ca.pem"
#tls_cert = "/etc/telegraf/cert.pem"
#tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
#insecure_skip_verify = false

I tried various combinations but it seems all of them give me

server gave HTTP response to HTTPS client

errors when launching telegraf.

Any hints what to put there? Should this be the host OS Cert Chain?

2

Well this was a misconfiguration on my end, caused on a different level. I accidentally exposed the ports on the influxdb container rather than the nginx container which were in the same network, so all API accces (https) went directly to influxdb which expected http only.

3

I have made some more testing and have discovered that I can use curl to insert data into the databas IF I add the --insecure flag to the command. I have tried adding “insecure_skip_verify = true” in the telegraf.config but it still does not work.