Telegraf on AWS ECS: how can I install additional packages?

Telegraf
1

Hi,

I sucessfully ran Telegraf on AWS Elastic Container Service (ECS). The task configuration included the following command:

"/bin/bash", "-c",
 telegraf --config https://eu-central-1-1.aws.cloud2.influxdata.com/api/v2/telegrafs/0123456abcd"

Now I need to change the Telegraf configuration because I need to switch from username / password login (provided in the Telegraf config) to a certificate based authentication. So my new Telegraf configuration looks something like this:

[[inputs.mqtt_consumer]]
  servers = ["abcd1234-ats.iot.eu-west-1.amazonaws.com"]
  tls_ca = "/tmp/telegraf/AmazonRootCA1.pem"
  tls_cert = "/tmp/telegraf/cert.pem"
  tls_key = "/tmp/telegraf/key.pem"

The certificates and CA files are on S3 which I want to copy over at the start of the docker image by using the AWS CLI. This means that I need to install some packages like aws cli and copy over some files into the docker container at startup time. I had several unsuccessful attempts to change my AWS ECS docker config command.
For example:

/bin/bash, -c,

mkdir /tmp/aws-cli;
cd /tmp/aws-cli;
curl https://s3.amazonaws.com/aws-cli/awscli-bundle.zip -o awscli-bundle.zip;
apt-get update;
apt-get install -y unzip;
unzip awscli-bundle.zip;
ln -s /usr/bin/python3 /usr/bin/python;
apt install -y python3.11-venv;
./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws;

mkdir /tmp/telegraf;
cd /tmp/telegraf;
aws s3 cp s3://my-bucket/telegraf-certificate.pem cert.pem;
chmod a+r cert.pem;
aws s3 cp s3://my-bucket/telegraf-private.pem.key key.pem;
chmod a+r key.pem;
aws s3 cp s3://my-bucket/AmazonRootCA1.pem AmazonRootCA1.pem;
chmod a+r AmazonRootCA1.pem;

telegraf --config https://eu-central-1-1.aws.cloud2.influxdata.com/api/v2/telegrafs/abcd0123456

AWS Cloud Watch shows following error because the docker container runs as a non-root user (as announced here) and so ‘apt-get’ fails:

E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied))

How can I install the AWS CLI in the ‘telegraf:latest’ image, ideally without creating my own image but to just configure my ECS task to retrieve the latest image and install the packages?

2

As the error message says you don’t have permission to apt. Most likely you need to run that command as root via sudo apt-get update.

3

sudo doesn’t seem to be an option, if I add it before the relevant commands, it prints:

/bin/bash: line 1: sudo: command not found

4

Ah that’s right, you would need to install packages as root then.

The better solution is to create a custom image, given that is essentially what you are doing. Additionally, the entry point script that we use to prevent running as root might be causing you issues with running those various commands.

5

So, no way to “inject” my packages as ‘root’ at the beginning and then switching back to user “telegraf”? I tried to do this in ECS’s command field but failed to make it run…

6

You need to do the user switching in a script or similar or, as @jpowers suggested, create a custom image for your ECS instance.