Influxdb v2.0 error

ts=2022-01-13T06:38:08.016187Z lvl=error msg="Unable to gather" log_id=0Yjh2ppW000 error="Get \"http://192.168.4.18:8086/metrics\": dial tcp 192.168.4.18:8086: i/o timeout"

@Jay_Clifford :pray: :pray: :pray: :pray:

Hi @loneWolf666 :slight_smile:

Please answer my previous questions as discussed in the other feed:

Telegraf 1.20.4 (git: HEAD 34ad5aa1)

Hi, @Jay_Clifford

telegraf: Telegraf 1.20.4 (git: HEAD 34ad5aa1)
mqtt : mosquitto version 1.6.12

I would advise updating to the latest Telegraf version when you can. As I believe we updated the go MQTT client library which had a fix for disconnects. Could you also check the logs of your Mosquitto broker and see if there is anything funky happening on that side. This appears to be an issue when the MQTT client fails to find the Mosquitto broker. So this could be a network issue or potentially a broker disconnect. Did you say the throughput was 2000 samples each minute?

Hi,@Jay_Clifford
The frequency of data received by the production environment telegraf should be 6000 interactions per second, max_undelivered_messages What does this indicator mean? :pray: :pray: :pray:

Hi @loneWolf666, That is quite a heavy workload. Did you check the mosquito broker logs and see if there is any tell, tell logs?

Max_undelivered_messages acts as a buffer trigger. This can help with the general throughput of incoming messages which may help your use case. Here is the description from the docs:

  ## Maximum messages to read from the broker that have not been written by an
  ## output.  For best throughput set based on the number of metrics within
  ## each message and the size of the output's metric_batch_size.
  ##
  ## For example, if each message from the queue contains 10 metrics and the
  ## output metric_batch_size is 1000, setting this to 100 will ensure that a
  ## full batch is collected and the write is triggered immediately without
  ## waiting until the next flush_interval.
  # max_undelivered_messages = 1000

Our product manager for IoT @samdillard is working on a cool MQTT feature for InfluxDB. Would you be open to discussing your use case further with him?

Hi, @Jay_Clifford
This is the mqtt broker log
Looking at the log, telegraf probably broke the chain actively :thinking:

148B1 (p2, c1, k60).
1642415990: New client connected from 10.44.0.138 as auto-A83A0726-8B21-800A-2AF8-88E85CA49C6A (p2, c1, k60).
1642415990: New client connected from 10.44.0.179 as auto-BFDB46CC-D0DD-AA10-867C-E11A9289DE11 (p2, c1, k60).
1642415990: New client connected from 10.44.0.154 as auto-DF3A63FE-A0C2-796F-8316-A16E9991B6FE (p2, c1, k60).
1642415990: Client auto-1DC641A8-2525-69D5-8EE9-FC9DFCCEF43C disconnected.
1642415990: Client auto-3388CFB9-4765-DF78-BA77-F8582E79779F disconnected.
1642415990: Client auto-F756C5B3-4459-0B0B-B92D-B68872F45D54 disconnected.
1642415990: New connection from 10.44.0.186 on port 1883.
1642415990: New connection from 10.44.0.148 on port 1883.
1642415990: New client connected from 10.44.0.148 as auto-023C4659-537E-2C65-E964-CA0CCB09D22C (p2, c1, k60).
1642415990: Client auto-AA8B97B0-A6CF-698C-9D0E-D9D5B6D148B1 disconnected.
1642415990: New connection from 10.44.0.206 on port 1883.
1642415990: New client connected from 10.44.0.206 as auto-A4391E3C-29A4-F7FA-219E-4DBAB185E791 (p2, c1, k60).
1642415990: New connection from 10.44.0.190 on port 1883.
1642415990: New client connected from 10.44.0.186 as auto-4A9947EE-C417-40C5-20D7-DBA67B78B5A5 (p2, c1, k60).
1642415990: New client connected from 10.44.0.190 as auto-740A896B-C7B1-2925-6265-FC6F2247E794 (p2, c1, k60).
1642415990: Client auto-023C4659-537E-2C65-E964-CA0CCB09D22C disconnected.
1642415990: Client auto-DF3A63FE-A0C2-796F-8316-A16E9991B6FE disconnected.
1642415990: New connection from 10.44.0.143 on port 1883.
1642415990: New client connected from 10.44.0.143 as auto-5329190E-48AB-1C87-C10C-054B471882FC (p2, c1, k60).
1642415990: Client auto-24C3B8AA-D5C0-E703-667A-CEB4A271DA7C disconnected.
1642415990: Client auto-4C65F522-920C-90EF-28E3-6F16891A2ABD disconnected.
1642415990: New connection from 10.44.0.194 on port 1883.
1642415990: New client connected from 10.44.0.194 as auto-E42BC7B4-C5F6-A8A5-93CD-23B5ABD58A7F (p2, c1, k60).
1642415990: Client auto-BFDB46CC-D0DD-AA10-867C-E11A9289DE11 disconnected.
1642415990: New connection from 10.44.0.181 on port 1883.
1642415990: New client connected from 10.44.0.181 as auto-2CFE2D32-8B70-D6DE-3752-3FFA1EBA401B (p2, c1, k60).
1642415990: New connection from 10.44.0.210 on port 1883.
1642415990: New client connected from 10.44.0.210 as auto-F351C42E-A7F8-ADCA-8FBA-823BD20F0806 (p2, c1, k60).
1642415990: Client auto-A83A0726-8B21-800A-2AF8-88E85CA49C6A disconnected.
1642415990: New connection from 10.44.0.134 on port 1883.
1642415990: Client auto-A4391E3C-29A4-F7FA-219E-4DBAB185E791 disconnected.
1642415990: Client auto-4A9947EE-C417-40C5-20D7-DBA67B78B5A5 disconnected.
1642415990: New client connected from 10.44.0.134 as auto-47E3047C-7F6A-4ADF-6782-9A7E24BD15B0 (p2, c1, k60).

@Jay_Clifford I’d be happy to talk to your product manager :smile: :hear_no_evil:

Awesome, it is Martin Luther King Day in the states so they are all off at the moment. When he is ready I’m sure he will reach out to you on here :).

Ah I am sorry @loneWolf666, these are logs from your InfluxDB instance. my apologies since conversations span a few threads I can get muddled :smiley:. Are you running any scrapperers within your InfluxDB instance i.e.: 192.168.4.18/metrics

@loneWolf666 Hi! Let’s do it! What’s the best email to use for us to coordinate?