(Part 1) Migrate and deploy Docker containers using AWS ECS Fargate with/without a Load Balancer

A step-by-step guide to build from scratch.

In this comprehensive guide, I will take you through each step involved in migrating and deploying your Docker containers using the power of AWS ECS Fargate and other related services. In this series, I will include the set-ups for all the necessary AWS Cloud components to achieve the goal.

Solution diagram for the migration and deployment of Docker container using AWS ECS Fargate

Here’s a glimpse of what you can get throughout this blog:

• Setting up key AWS Infrastructure (Part 1): This includes VPC (Virtual Private Cloud), subnets, security groups, and internet gateway.
• Pushing existing docker image to Amazon ECR (Part 1): This includes creating private repository and set up proper policies and permissions.
• Creating IAM roles for ECS (Part 1): A role with full ECS access will be created for demonstration purpose.
• Creating AWS ECS Cluster and configuring Task Definitions (Part 1): Cluster will be used to manage containerized applications and Task Definition defines the properties and requirements of containers.
• Launching ECS Services without Application Load Balancer (Part 2): ECS Service will run containers with direct exposure to the internet.
• Launching ECS Services with Application Load Balancer (Part 2): Options to include ALB will be explored to ensure high availability.
• Testing the deployment and Troubleshooting (Part 2): It will also include the different testing methods for deployment with or without ALB.

Step 1: Setting up key AWS Infrastructure

When deploying your containers on Fargate, it is crucial to set up a VPC, subnets, and security groups properly to ensure security, networking, and connectivity. Also, this step is done with the assumptions that users have already gained administrative access.

1. Go to VPC service on AWS, click on ‘Create VPC’ button, and fill in the following information.

2. Once the VPC is created, find ‘Subnets’ on the left panel and click on ‘Create subnets’. Select my-vpc-example from last step and create 2 subnets residing in different zones. For this VPC, I use 10.0.0.0/16 which includes 65,536 available IPs for IPv4 CIDR block and 10.0.0.0/24 and 10.0.1.0/24 for subnet CIDR blocks.

3. Go to EC2 service, select security groups from the left panel, and click on ‘Create security groups’. My docker container runs on Port 3000 so it needs to be included in your inbound rules. Note that it is NOT recommended to allow SSH access from all IP addresses (0.0.0.0/0) to your instance for security reasons. I only use it for testing purpose and for a short time.

Step 2: Pushing existing docker image to Amazon ECR

Once the AWS infrastructure has been set up, the next step is to migrate your existing docker container to AWS services. The very first thing is to push your docker image to Amazon ECR. Amazon Elastic Container Registry (ECR) is a fully managed container registry that allows you to store and deploy your containers or applications images efficiently and securely.

1. Go to ECR service and create a private repository for your example on ECR.

2. Corresponding repository policies need to be defined to allow docker images to be uploaded. First, select waha/test-example repository, click on ‘Actions’, and choose ‘Permission’ in the drop-down menu. Click on ‘Edit policy JSON’ and paste the following JSON file.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "permission",
      "Effect": "Allow",
      "Principal": "*",
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:BatchGetImage",
        "ecr:CompleteLayerUpload",
        "ecr:DescribeImages",
        "ecr:DescribeRepositories",
        "ecr:GetAuthorizationToken",
        "ecr:GetDownloadUrlForLayer",
        "ecr:InitiateLayerUpload",
        "ecr:ListImages",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ]
    }
  ]
}

3. Next you can upload your existing docker image to ECR repository via AWS CLI commands. You need to install the latest version of AWS CLI and Docker first.

First, you authenticate your docker client to the registry.

aws ecr get-login-password --region {region} | docker login --username AWS --password-stdin {Account_ID}.dkr.ecr.{region}.amazonaws.com

Then, view your available images using ‘docker ps -a’ and tag your docker image you want to deploy using the following command line.

docker tag {your_docker_image_name}:latest {ECR_repository_URI}:latest 

Finally, push the docker image to the repository.

docker push {ECR_repository_URI}:latest

Step 3: Creating IAM roles for ECS

1. Create IAM role for ECS use case. Go to IAM service and select Roles on the left panel.

2. Attach two policies to this role. These include having full access for ECS, which might not be ideal in practice, and specific permission policies instead shall be assigned accordingly.

3. Give the role a name ecs_full_access_role, review the IAM role, and click on ‘Create role’ to complete the step.

Step 4: Creating AWS ECS Cluster and configuring Task Definitions

Up til this point, all the necessary AWS infrastructures have been set up. Now it is ready to dive into the world of ECS and create relevant ECS services.

An ECS Fargate cluster consists of several essential components.

• First, container instances serve as the compute resources, responsible for running containers and executing tasks within the cluster.
• Task definitions define how containers should be configured, including resource requirements and networking settings.
• Tasks represent running instances of containerized applications, while services enable managing applications and handling load balancing and task recovery.

1. Navigate to ECS service, click on ‘Create cluster’, and choose serverless AWS Fargate to deploy your containers.

2. Select ‘Task definitions’ on the left panel and click on ‘Create a new task definition’. Note that the task size depends on the task workload and the smallest CPU and memory are chosen in this blog.

Then fill in the information for the container you uploaded in Step 2, including Image URI and Port number. After this, review the task definitions and complete the creation.

The status of created task definition should be ‘ACTIVE’ after successful creation

This concludes the Part 1 of this blog and the rest of the steps will be continued in the Part 2.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -

Thank you for your time reading this blog and any feedback is greatly appreciated.