Issues with InfluxDB, Telegraf, and Docker

InfluxDB 2
1

I am running InfluxDB and Grafana in docker, and running the Telegraf agent on bare metal.

I am mounting directories to /var/lib/influxdb and /etc/influxdb on the docker container, but every time I spin up the container, my configurations disappeared. There’s no files in the mounted directories. Grafana on the other hand maintains a persistent config through the mounted directories.

Also I was only able to actually see data from docker once. Every time I try to set things up now, it does not work. I spin up InfluxDB, create the Org and Bucket, and configure a Telegraf data source. Usually when I check for connections it immediately finds the Telegraf instance.

I try to explore the data, and I get Failed to load buckets.

My logs for Telegraf say

[agent] Error writing to outputs.influxdb: could not write any address
[outputs.influxdb] Metric buffer overflow; 23 metrics have been dropped
[outputs.influxdb] when writing to [http://localhost:9999]: 200 OK

I have no idea how to fix this. If I replace localhost with the IP that is generated in by the data source, I get permissions errors.

[outputs.influxdb_v2] when writing to [http://10.1.10.151:9999]: failed to write metric: unauthorized: unauthorized access

When I try and “Test” the influxDB datasource in Grafana, I get Network Error: Bad Gateway(502).

1 Like
2

InfluxDB 2 doesn’t use those directories and cannot be configured through a file.

In order to persist data, please setup /root/.influxdbv2 as a volume mount.

3

Great, mounting the new directory seems to work. Now the only problem is actually getting data.

When I set up the Telegraf connection initally, it says it is “receiving data loud and clear”. But none of the dashboards or explore data show any data coming in.

Telegfraf still complains

Metric buffer overflow; 105 metrics have been dropped
when writing to [http://10.1.10.151:9999]: failed to write metric: unauthorized: unauthorized access

I’m a bit confused about what to do with the generated telegraf.conf because I get errors if I use telegraf as a system service while pointed at that file on my disk, as well as when I point it to the API config.