Blog

Making Amazon ECS Container Service as easy to use as Docker run

31 Aug, 2015

One of the reasons Docker caught fire was that it was soo easy to use. You could build and start a docker container in a matter of seconds. With Amazon ECS this is not so. You have to learn a whole new lingo (Clusters, Task definitions, Services and Tasks), spin up an ECS cluster, write a nasty looking JSON file or wrestle with a not-so-user-friendly UI before you have your container running in ECS.
In the blog we will show you that Amazon ECS can be as fast, by presenting you a small utility named ecs-docker-run which will allow you to start a Docker container almost as fast as with Docker stand-alone by interpreting the Docker run command line options. Together with a ready-to-run CloudFormation template, you can be up and running with Amazon ECS within minutes!


ECS Lingo

Amazon ECS uses different lingo than Docker people, which causes confusion. Here is a short translation:

  • Cluster – one or more Docker Hosts.
  • Task Definition – A JSON representation of a docker run command line.
  • Task – A running docker instance. When the instance stops, the task is finished.
  • Service – A running docker instance, when it stops, it is restarted.
    In the basis that is all there is too it. (Cutting a few corners and skimping on a number of details).
    Once you know this, we are ready to use ecs-docker-run.

    ECS Docker Run

    ecs-docker-run is a simple command line utility to run docker images on Amazon ECS. To use this utility you can simply type something familiar like:
    [code]
    ecs-docker-run \
    –name paas-monitor \
    –env SERVICE_NAME=paas-monitor \
    –env SERVICE_TAGS=http \
    –env "MESSAGE=Hello from ECS task" \
    –env RELEASE=v10 \
    -P \
    mvanholsteijn/paas-monitor:latest
    [/code]
    substituting the ‘docker run’ with ‘ecs-docker-run’.
    Under the hood, it will generate a task definition and start a container as a task on the ECS cluster. All of the following Docker run command line options are functionally supported.

    -Ppublishes all ports by pulling and inspecting the image.
    –namethe family name of the task. If unspecified the name will be derived from the image name.
    -padd a port publication to the task definition.
    –envset the environment variable.
    –memorysets the amount of memory to allocate, defaults to 256
    –cpu-sharesset the share cpu to allocate, defaults to 100
    –entrypointchanges the entrypoint for the container
    –linkset the container link.
    -vset the mount points for the container.
    –volumes-fromset the volumes to mount.

    All other Docker options are ignored as they refer to possibilities NOT available to ECS containers. The following options are added, specific for ECS:

    –generate-onlywill only generate the task definition on standard output, without starting anything.
    –run-as-serviceruns the task as service, ECS will ensure that ‘desired-count’ tasks will keep running.
    –desired-countspecifies the number tasks to run (default = 1).
    –clusterthe ECS cluster to run the task or service (default = cluster).

    Hands-on!

    In order to proceed with the hands-on part, you need to have:

  • jq installed
  • aws CLI installed (version 1.7.44 or higher)
  • aws connectivity configured
  • docker connectivity configured (to a random Docker daemon).

    checkout ecs-docker-run

    Get the ecs-docker-run sources by typing the following command:
    [code]
    git clone git@github.com:mvanholsteijn/ecs-docker-run.git
    cd ecs-docker-run/ecs-cloudformation
    [/code]

    import your ssh key pair

    To look around on the ECS Cluster instances, import your public key into Amazon EC2, using the following command:
    [code]
    aws ec2 import-key-pair \
    –key-name ecs-$USER-key \
    –public-key-material "$(ssh-keygen -y -f ~/.ssh/id_rsa)"
    [/code]

    create the ecs cluster autoscaling group

    In order to create your first cluster of 6 docker Docker Hosts, type the following command:
    [code]
    aws cloudformation create-stack \
    –stack-name ecs-$USER-cluster \
    –template-body "$(<ecs.json)" \
    –capabilities CAPABILITY_IAM \
    –parameters \
    ParameterKey=KeyName,ParameterValue=ecs-$USER-key \
    ParameterKey=EcsClusterName,ParameterValue=ecs-$USER-cluster
    [/code]
    This cluster is based upon the firstRun cloudformation definition, which is used when you follow the Amazon ECS wizard.

    And wait for completion…

    Wait for completion of the cluster creation, by typing the following command:
    [code]
    function waitOnCompletion() {
    STATUS=IN_PROGRESS
    while expr "$STATUS" : ‘^.*PROGRESS’ > /dev/null ; do
    sleep 10
    STATUS=$(aws cloudformation describe-stacks \
    –stack-name ecs-$USER-cluster | jq -r ‘.Stacks[0].StackStatus’)
    echo $STATUS
    done
    }
    waitOnCompletion
    [/code]

    Create the cluster

    Unfortunately, CloudFormation does (not) yet allow you to specify the ECS cluster name, so need to manually create the ECS cluster, by typing the following command:
    [code]
    aws ecs create-cluster –cluster-name ecs-$USER-cluster
    [/code]
    You can now manage your hosts and tasks from the Amazon AWS EC2 Container Services console.

    Run the paas-monitor

    Finally, you are ready to run any docker image on ECS! Type the following command to start the paas-monitor.
    [code]
    ../bin/ecs-docker-run –run-as-service \
    –number-of-instances 3 \
    –cluster ecs-$USER-cluster \
    –env RELEASE=v1 \
    –env MESSAGE="Hello from ECS" \
    -p :80:1337 \
    mvanholsteijn/paas-monitor
    [/code]

    Get the DNS name of the Elastic Load Balancer

    To see the application in action, you need to obtain the DNS name of the Elastic Load Balancer. Type the following commands:
    [code]

    Get the Name of the ELB created by CloudFormation

    ELBNAME=$(aws cloudformation describe-stacks –stack-name ecs-$USER-cluster | \
    jq -r ‘.Stacks[0].Outputs[] | select(.OutputKey =="EcsElbName") | .OutputValue’)

    Get the DNS from of that ELB

    DNSNAME=$(aws elb describe-load-balancers –load-balancer-names $ELBNAME | \
    jq -r .LoadBalancerDescriptions[].DNSName)
    [/code]

    Open the application

    Finally, we can obtain access to the application.
    [code]
    open https://$DNSNAME
    [/code]
    And it should look something like this..

    hostreleasemessage# of callsavg response timelast response time
    b6ee7869a5e3:1337v1Hello from ECS from release v1; server call count is 82684536
    4e09f76977fe:1337v1Hello from ECS from release v1; server call count is 68684138
    65d8edd41270:1337v1Hello from ECS from release v1; server call count is 82684037

    Perform a rolling upgrade

    You can now perform a rolling upgrade of your application, by typing the following command while keeping your web browser open at https://$DNSNAME:
    [code]
    ../bin/ecs-docker-run –run-as-service \
    –number-of-instances 3 \
    –cluster ecs-$USER-cluster \
    –env RELEASE=v2 \
    –env MESSAGE="Hello from Amazon EC2 Container Services" \
    -p :80:1337 \
    mvanholsteijn/paas-monitor
    [/code]
    The result should look something like this:

    hostreleasemessage# of callsavg response timelast response time
    b6ee7869a5e3:1337v1Hello from ECS from release v1; server call count is 1241104337
    4e09f76977fe:1337v1Hello from ECS from release v1; server call count is 1101104135
    65d8edd41270:1337v1Hello from ECS from release v1; server call count is 1241104037
    ffb915ddd9eb:1337v2Hello from Amazon EC2 Container Services from release v2; server call count is 43151994238
    8324bd94ce1b:1337v2Hello from Amazon EC2 Container Services from release v2; server call count is 41414138
    7b8b08fc42d7:1337v2Hello from Amazon EC2 Container Services from release v2; server call count is 41413839

    Note how the rolling upgrade is a bit crude. The old instances stop receiving requests almost immediately, while all requests seem to be loaded onto the first new instance.

    You do not like the ecs-docker-run script?

    If you do not like the ecs-docker-run script, do not dispair. Below are the equivalent Amazon ECS commands to do it without the hocus-pocus script…

    Create a task definition

    This is the most difficult task: Manually creating a task definition file called ‘manual-paas-monitor.json’ with the following content:
    [code]
    {
    "family": "manual-paas-monitor",
    "containerDefinitions": [
    {
    "volumesFrom": [],
    "portMappings": [
    {
    "hostPort": 80,
    "containerPort": 1337
    }
    ],
    "command": [],
    "environment": [
    {
    "name": "RELEASE",
    "value": "v3"
    },
    {
    "name": "MESSAGE",
    "value": "Native ECS Command Line Deployment"
    }
    ],
    "links": [],
    "mountPoints": [],
    "essential": true,
    "memory": 256,
    "name": "paas-monitor",
    "cpu": 100,
    "image": "mvanholsteijn/paas-monitor"
    }
    ],
    "volumes": []
    }
    [/code]

    Register the task definition

    Before you can start a task it has to be registered at ECS, by typing the following command:
    [code]
    aws ecs register-task-definition –cli-input-json "$(<paas-monitor.json)"
    [/code]

    Start a service

    Now start a service based on this definition, by typing the following command:
    [code]
    aws ecs create-service \
    –cluster ecs-$USER-cluster \
    –service-name manual-paas-monitor \
    –task-definition manual-paas-monitor:1 \
    –desired-count 1
    [/code]
    You should see a new row appear in your browser:

    hostreleasemessage# of callsavg response timelast response time
    ….
    5ec1ac73100f:1337v3Native ECS Command Line Deployment from release v3; server call count is 37373736

    Conclusion

    Amazon EC2 Container Services has a higher learning curve than using plain Docker. You need to get passed the lingo, the creation of an ECS cluster on Amazon EC2 and most importantly the creation of the cumbersome task definition file. After that it is almost as easy to use as Docker run.
    In return you get all the goodies from Amazon like Autoscaling groups, Elastic Load Balancers and multi-availability zone deployments ready to use in your Docker applications. So, check ECS out!

    More Info

    Checkout more information:

Explore related posts