Tutorial – Using Deployit Cloud Pack with Amazon EC2 – Part 1

Deployit’s Cloud Pack provides you with the ability to create and destroy environments on virtualized infrastructure from Deployit. It supports both EC2 and vSphere. In this first part of the tutorial, I am going to show you how to setup Amazon AWS and populate the Deployit repository in such a way that you can create and destroy virtual machines  from the Deployit console.

I assume that you already have signed up for Amazon AWS. If you have not done so, do it now!
Within Amazon AWS we are going to build a Amazon Machine Image (AMI) that has Apache Webserver and Apache Tomcat preinstalled. From Deployit, we are going to define two different host templates, one for a tomcat host and one for a webserver host and they are going to be bundled into an environment template.

Create a new user on AWS

We are going to create a specific user that is going to do all the work for us on Amazon. Using a different user for Deployit allows us to audit what Deployit does and control the actions of Deployit.

Create the user ‘cloud-pack-user’ on IAM.

Goto the IAM management console and create a new user called ‘cloud-pack-user’.

 

Download credentials to ~/.ec2/cloud-pack-user-credentials.csv

Click on create and download the credentials to ~/.ec2/cloud-pack-user-credentials.csv and change the permission to read only. In the file, you will find the access key and secret key that you need to fill later to give Deployit access.
$ chmod 0400 ~/.ec2/ cloud-pack-user-credentials.csv

Grant Power user Access permissions to ‘cloud-pack-user’.

Of course, deployit needs permissions to create stuff on AWS!

Create a security group ‘cloud-pack-group’

Deployit will assign a security group to newly instantiated machines. We are going to create a single security group for our example. Goto the EC2 management console  and create a security group ‘cloud-pack-group’. With the following networking rules:

On the AWS console it look like the screenshot below.

Download cloud-pack-user private key to ~/.ec2/cloud-pack-user-keypair.pem

In order for Deployit to log on to the machines, you need Amazon AWS to generate a SSH private key pair.  Goto the EC2 Management console and create a new key pair ‘cloud-pack-user’ .. Save the private key in ~/.ec2/cloud-pack-user-keypair.pem. Change the permissions on the file to read only

$ chmod 0400 ~/.ec2/cloud-pack-user-keypair.pem

Create a cloud AMI

For our example, we are going to create a specific machine image that contains both Apache, Tomcat and MySQL and has non-interactive sudo rights.

Launch an Amazon linux image

Using Amazon AWS console, launch an instance of the image ‘Amazon Linux AMI 2013.03.1, 32 bit name ‘cloud-server-template’.

Install apache, tomcat and mysql

Login to the machine:

$ ssh -i ~/.ec2/cloud-pack-user-keypair.pem \
–l ec2-user \
<ip-address of created host>
$ yum update
$ yum install httpd
$ yum install tomcat7
$ chkconfig –level 3 httpd on
$ chkconfig –level 3 tomcat7 on

Allow non-interactive sudo

Edit the file /etc/sudoers and comment out the ‘Defaults requiretty’.

#         You have to run "ssh -t hostname sudo <cmd>".
#
#Defaults    requiretty

Stop the machine

Now stop the machine:

$ shutdown –h now

Create an AMI based on this instance.

In order to be able to create new machine instances you need to create a new Amazon Machine Instance.  Goto the Instances screen on the (EC2 management console) and create a new AMI based on our ‘cloud-server-template’ instance.


Find the AMI ID from the (AMI screen of your EC2 management console) in our case ami-9e9c78e9.

Install deployit 3.9.3 plugins

Install the following plugins in Deployit:

• Cloud pack plugin
• EC2 plugin
• Tomcat plugin
• All available plugins from $DEPLOYIT_HOME/available_plugins

Add the AWS credentials to the Deployit repository

Login to Deployit and create the AWS credentials ‘cloud-pack-user-credentials’ under ‘Configuration’. You can read the access key and secret credentials from ~/.ec2/cloud-pack-user-credentials.csv.

Create a EC2 host template ‘apache-webserver-template’

Create in the Deployit repository a EC2 host template named ‘apache-webserver-template’ under ‘Configuration’. The following table shows how to fill in the different fields in the host template.  The instance descriptor describes how an instance of the SshHost is going to look like once the EC2 instance is instantiated.

<#escape x as x?xml>
  <list>
    <cloud.SshHost id="${hostsPath}/${hostTemplate.name}_${hostAddress}">
      <template ref="${hostTemplate.id}"/>
      <cloudId>${cloudId}</cloudId>
      <address>${hostAddress}</address>
      <#if hostTemplate.privateKeyFile??><privateKeyFile>${hostTemplate.privateKeyFile}</privateKeyFile></#if>
      <username>${hostTemplate.username}</username>
      <#if hostTemplate.password??><password>${hostTemplate.password}</password></#if>
      <os>${hostTemplate.os}</os>
      <connectionType>${hostTemplate.connectionType}</connectionType>
      <sudoUsername>root</sudoUsername>
    </cloud.SshHost>
    <www.ApacheHttpdServer id="${hostsPath}/${hostTemplate.name}_${hostAddress}/httpd">
      <host ref="${hostsPath}/${hostTemplate.name}_${hostAddress}"/>
      <startCommand>sudo apachectl stop</startCommand>
      <startWaitTime>3</startWaitTime>
      <stopCommand>sudo apachectl stop</stopCommand>
      <stopWaitTime>3</stopWaitTime>
      <restartCommand>sudo apachectl restart</restartCommand>
      <restartWaitTime>10</restartWaitTime>
      <defaultDocumentRoot>/var/www</defaultDocumentRoot>
      <configurationFragmentDirectory>/etc/httpd/conf.d</configurationFragmentDirectory>
    </www.ApacheHttpdServer>
  </list>
</#escape>

The following two screen shots show the host template definitions.

Instantiate a host based on the ‘apache-webserver-template’

Now right click on repository item ‘apache-webserver-template’ under the ‘Configuration’ and select ‘instantiate’ . Fill in the name for the instance ‘my-apache-webserver’ and execute the deployment plan!

Restart the apache webserver

Refresh the repository and goto the httpd folder under the newly created host ‘apache-webserver-template-….’ under ‘Infrastructure’. Right click and select restart.

Create a EC2 host template ‘apache-tomcat-template’

Create in the Deployit repository a EC2 host template named ‘apache-tomcat-template’ under ‘Configuration’.  All the parameters are the same as for the Apache Webserver template except for the instance descriptor. The instance descriptor describes how an instance of the SshHost is going to look like once the EC2 instance is instantiated.

ec2 credentials	cloud-pack-user-credentials
(security) group	cloud-pack-group’
AMI ID	taken from your newly create AMI in the previous step
Region	eu-west-1
Instance type	T1micro
Key pair	cloud-pack-user-keypair
Username	ec2-user
Private key file	$HOME/.ec2/cloud-pack-user-keypair.pem
Instance descriptor	<#escape x as x?xml> <list> <cloud.SshHost id="${hostsPath}/${hostTemplate.name}_${hostAddress}"> <template ref="${hostTemplate.id}"/> <cloudId>${cloudId}</cloudId> <address>${hostAddress}</address> <#if hostTemplate.privateKeyFile??><privateKeyFile>${hostTemplate.privateKeyFile}</privateKeyFile></#if> <username>${hostTemplate.username}</username> <#if hostTemplate.password??><password>${hostTemplate.password}</password></#if> <os>${hostTemplate.os}</os> <connectionType>${hostTemplate.connectionType}</connectionType> <sudoUsername>root</sudoUsername> </cloud.SshHost> <tomcat.Server id="${hostsPath}/${hostTemplate.name}_${hostAddress}/tomcat"> <host ref="${hostsPath}/${hostTemplate.name}_${hostAddress}"/> <startCommand>sudo service tomcat7 start</startCommand> <startWaitTime>90</startWaitTime> <stopCommand>sudo service tomcat7 stop</stopCommand> <stopWaitTime>10</stopWaitTime> <home>/usr/share/tomcat7</home> </tomcat.Server> <tomcat.VirtualHost id="${hostsPath}/${hostTemplate.name}_${hostAddress}/tomcat/localhost"> <server ref="${hostsPath}/${hostTemplate.name}_${hostAddress}/tomcat"/> </tomcat.VirtualHost> </list>

Instantiate a host based on the ‘apache-tomcat-template’

Just to test that your template is really working, you can instantiate a host. Now right click on repository item ‘apache-tomcat-template’ under the ‘Configuration’ and select ‘instantiate’. Fill in the name for the instance ‘my-tomcat-appserver’ and execute the deployment plan!
After this you can access tomcat on port:8080 on the IP address displayed by Deployit.

Restart the tomcat app server

Refresh the repository and goto the tomcat folder under the newly created host ‘apache-tomcat-….’ under ‘Infrastructure’. Right click and select stop. Do not forget to  select start again!

Destroy the instances

Cleanup and save money! You can now destroy the hosts and remove them from AWS and the repository by selecting ‘Destroy’ on the newly created host items in the repository. If you need it again, just select instantiate on the host templates.

Create a Cloud environment

Now the two hosts template will be grouped together into a single environment template.

Create the environment template ‘linux-apache-tomcat-envtemplate’

Create in the Deployit repository a cloud Environment template named ‘linux-apache-tomcat-envtemplate’ under ‘Configuration’. Add the apache-tomcat  and the apache-webserver host templates as the members. Fill in the following environment template. The template creates a dictionary and an environment containing two hosts.

<#escape x as x?xml>
<list>
<udm.Dictionary id="${environmentId}-dictionary">
<entries>
<#list hosts as h>
<#if h.name?starts_with("apache-tomcat")>
<entry key="tomcat-address">${h.address}:8080</entry>
</#if>
<#if h.name?starts_with("apache-webserver")>
<entry key="apache-address">${h.address}</entry>
</#if>
</#list>
</entries>
</udm.Dictionary>
<cloud.Environment id="${environmentId}">
<dictionaries>
<ci ref="${environmentId}-dictionary"/>
</dictionaries>
<members>
<#list hosts as h>
<ci ref="${h.id}" />
<#if h.name?starts_with("apache-webserver")>
<ci ref="${h.id}/httpd" />
</#if>
<#if h.name?starts_with("apache-tomcat")>
<ci ref="${h.id}/tomcat" />
<ci ref="${h.id}/tomcat/localhost" />
</#if>
<#if h.name?starts_with("mysql")>
<ci ref="${h.id}/mysql" />
</#if>
</#list>
</members>
</cloud.Environment>
</list>
</#escape>

Create a directory ‘test’ under infrastructure

Create a directory test under infrastructure.

Instantiate an entire environment based on the template

Right click on the ‘linux-apache-tomcat-envtemplate’ and select ‘instantiate environment’. Fill in the ‘Environments/test’ and ‘Infrastructure/test’ as parameters. Click next and execute the command.

The resulting Infrastructure and environment should look like this.

Now you can create as many environments as you want, based on this template! Have fun!
Do not forget to destroy you cloud enviroment to save money!

In the next blog we will use this cloud template to deploy applications to.