Hi everyone,
I’m encountering an issue with a Dockerized Python script on my machine, which is failing to connect to a Dockerized InfluxDB instance running on another machine in my local network. The script worked fine before Dockerization, successfully connecting to and writing data in the remote Dockerized InfluxDB. However, after Dockerizing it, the script can no longer write data to InfluxDB.
Python Script (rabbitmq_to_influxdb.py):
the python script:
import os
import pika
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS
import json
import time
import pytz
from datetime import datetime
# Environment variables for service hosts with defaults for local development
rabbitmq_host = os.getenv('RABBITMQ_HOST', 'rabbitmq')
# InfluxDB connection details
influxdb_host = os.getenv('INFLUXDB_HOST', 'OTHER_INFLUX_IP')
token = os.getenv('INFLUXDB_TOKEN', 'OTHER_INFLUX_TOKEN')
org = os.getenv('INFLUXDB_ORG', 'OTHER_INFLUX_ORG')
bucket = os.getenv('INFLUXDB_BUCKET', 'OTHER_INFLUX_BUCKET')
# Initialize InfluxDB client
client = InfluxDBClient(url=influxdb_host, token=token, org=org)
write_api = client.write_api(write_options=SYNCHRONOUS)
# Counter for successful writes
successful_writes = 0
# Connect to RabbitMQ
def connect_to_rabbitmq():
while True:
try:
connection = pika.BlockingConnection(pika.ConnectionParameters(rabbitmq_host))
channel = connection.channel()
# Declare the queue here to ensure it exists
channel.queue_declare(queue='sensor_info', durable=True)
return connection, channel
except pika.exceptions.AMQPConnectionError:
print("Connection to RabbitMQ failed. Retrying...")
time.sleep(5) # Wait 5 seconds before retrying
# Callback function to process incoming messages
def callback(ch, method, properties, body):
global successful_writes
data = json.loads(body)
print(f"Received message: {data}")
# Convert the timestamp to a format that InfluxDB understands (RFC3339)
timestamp = datetime.strptime(data['timestamp'], "%Y-%m-%d %H:%M:%S")
timestamp = timestamp.replace(tzinfo=pytz.UTC) # Set timezone to UTC
# Constructing the point to write to InfluxDB
point = Point("sensor_info").tag("sensor", f"antenna_{data['antenna_id']}")
for key in ['temperature', 'humidity', 'wind_speed']:
point = point.field(key, float(data[key]))
point = point.field("gps_latitude", float(data['gps']['latitude']))
point = point.field("gps_longitude", float(data['gps']['longitude']))
point = point.field("inclinometer_x", float(data['inclinometer']['x']))
point = point.field("inclinometer_y", float(data['inclinometer']['y']))
point = point.time(timestamp, WritePrecision.NS)
try:
write_api.write(bucket, org, point)
successful_writes += 1 # Increment the counter
print(f"Message sent to InfluxDB. -> NUMBER: {successful_writes}")
except Exception as e:
print(f"Error sending to InfluxDB: {e}")
def main():
connection, channel = connect_to_rabbitmq()
# Listen to each specific antenna queue
for i in range(5): # Assuming you have 5 antennas
queue_name = f'sensor_info_antenna_{i}'
channel.queue_declare(queue=queue_name, durable=True)
channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
print(' [*] Waiting for messages from all antenna queues. To exit press CTRL+C')
while True:
try:
channel.start_consuming()
except pika.exceptions.StreamLostError:
print("Lost connection to RabbitMQ. Attempting to reconnect...")
connection, channel = connect_to_rabbitmq()
# Re-declare the consumption setup after reconnection
for i in range(5):
queue_name = f'sensor_info_antenna_{i}'
channel.queue_declare(queue=queue_name, durable=True)
channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
except KeyboardInterrupt:
print(f"\nTotal successful writes to InfluxDB: {successful_writes}")
connection.close()
break
if __name__ == '__main__':
main()
Docker Compose File:
version: '3.8'
services:
rabbitmq:
image: rabbitmq:3-management
environment:
RABBITMQ_DEFAULT_USER: guest
RABBITMQ_DEFAULT_PASS: guest
ports:
- "5672:5672"
- "15672:15672"
app-1:
image: luisevoleo/my-python-app-1:v2
depends_on:
- rabbitmq
environment:
- RABBITMQ_HOST=rabbitmq
app-2:
image: luisevoleo/python-app-2:latest
network_mode: host
environment:
- RABBITMQ_HOST=localhost
- INFLUXDB_HOST=OTHER_INFLUX_IP
- INFLUXDB_TOKEN=OTHER_INFLUX_TOKEN
- INFLUXDB_ORG=OTHER_INFLUX_ORG
- INFLUXDB_BUCKET=OTHER_INFLUX_BUCKET
(Note: In the actual Docker Compose file, “OTHER_INFLUX_IP”, “OTHER_INFLUX_TOKEN”, “OTHER_INFLUX_ORG”, and “OTHER_INFLUX_BUCKET” are replaced with the real values.)
the log of “my-python-app-1:v2”:
024-01-11 09:29:58 [*] Waiting for messages from all antenna queues. To exit press CTRL+C
2024-01-11 09:29:58 Received message: {'antenna_id': 0, 'timestamp': '2024-01-07 15:49:31', 'gps': {'latitude': 36.864788334924214, 'longitude': -6.251224008665378}, 'temperature': 12.183960983137183, 'humidity': 62.32437121500195, 'wind_speed': 17.817718482363087, 'inclinometer': {'x': 164.14445635809955, 'y': -31.701584384447187}}
2024-01-11 09:30:01 Error sending to InfluxDB: <urllib3.connection.HTTPConnection object at 0x7fa6098faf50>: Failed to establish a new connection: [Errno 113] No route to host
2024-01-11 09:30:01 Received message: {'antenna_id': 0, 'timestamp': '2023-12-22 15:52:22', 'gps': {'latitude': 36.864788334924214, 'longitude': -6.251224008665378}, 'temperature': 0.6879560629965127, 'humidity': 44.81791862827329, 'wind_speed': 24.415517279710393, 'inclinometer': {'x': -153.10337700323, 'y': -139.99972670559453}}
2024-01-11 09:30:04 Error sending to InfluxDB: <urllib3.connection.HTTPConnection object at 0x7fa6098fb280>: Failed to establish a new connection: [Errno 113] No route to host
The container with the script is receiving data but cannot write it to the Dockerized InfluxDB on another machine.
Attempts to Resolve:
-Confirmed InfluxDB is accessible from other machines on the network (I can access it through my browser).
-Tried various Docker network configurations (bridge, host, etc.) without success.
I’m new to Docker and Influx and suspect the issue might be with how I’ve Dockerized the Python script. Can anyone help identify what I’m doing wrong or suggest a solution?
Thanks in advance!
