I was recently introduced to Amazon's new Ec2 services. The idea of cloud computing really intrigued me after I heard about it so I decided to take the dive. There is a bit of a learning curve with getting started but once you get started you realize the unlimited potential that cloud computing offers. Ec2 offers the ability to deploy pre-configured (linux based) images (called AMI's). The AMI's can be created from scratch or based on prebuit versions that Amazon or other users have exposed. You can quickly deploy to several different types of machines depending on your requirements. The base system has a 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth. Currently this will cost you $.10 per computing hour plus bandwidth costs. You are only charged for the time that the virtual machine is running and you can start and stop multiple instances at your will to scale as you need to. There are also beefier 64-bit machines available at a higher cost. On limitation (depending on how you look at it) is that persistent storage is not offered on the instances. After you start it up if at any time it crashes you lose everything on the instance. There are ways to overcome this as I will explain later but it makes things a bit more challenging. I found that the simplest way to get started is to find a public AMI that meets you needs, make the modifications to the instance then save it as your own instance into Amazon S3. S3 is another service that Amazon offers for storage, S3 and Ec2 work hand-in-hand with one another.

To get started you will need an account with Amazon Web Services at http://aws.amazon.com. You will need to sign up with both Ec2 and S3. It does not cost anything up front but you will need a credit card for them to draw funds from once you start using the service. One thing that took me a little while to get use to was the extensive use of certificates for authentication. Beyond signing in to your AWS account nearly everything else with the Ec2 service uses certificates or private keys. You use them to start your instances, as well as gain remote root access to an instance that you have started. It really makes things more secure. So lets get started....btw I recently switched from PC to Mac so all of the instructions will be for the Mac but they translate easily to the PC if you are familiar with java.

1. Log into your AWS account, I am assuming you signed up with Ec2 and S3 already.
2. After you are signed click on the "You Web Services Account" button and you will find the "AWS Access Identifiers" link.
3. Select X.509 certificates link.
4. When you click on the "create new" link you will be asked to confirm, click yes and the two files will be generated. You will find the two following files. These are the certificates I mentioned above that are used to authenticate you when any commands are issued to Ec2. There will be an additional cert that we create later to launch your instances.
   1. X.509 certificate named cert-xxxxxxx.pem
   2. RSA private key named pk-xxxxxxx.pem
5. Next you will need to download the Amazon Ec2 command line tools.
6. Now it is time to setup your machine to use the Ec2 tools.
7. Open the terminal and go to your Mac home directory and create a new folder named ~/.ec2
8. Copy the cert-xxxxxxx.pem and pk-xxxxxxx.pem into your ~/.ec2 directory from above.
9. Unzip the tools into the ~./ec2 directory and move out the bin and lib directories to this directory as well. It should look like the following
   1. cert-xxxxxxx.pem file
   2. pk-xxxxxxx.pem file
   3. The bin directory
   4. The lib directory
10. Next you will need to set a few environmental variables. To make things easier you can place these changes in your ~/.bash_profile file. If this file does not exist in your home directory you can create it then add the following:
    # Amazon Ec2 tools
    export EC2_HOME=~/.ec2
    export PATH=$PATH:$EC2_HOME/bin
    export EC2_PRIVATE_KEY=`ls $EC2_HOME/pk-*.pem`
    export EC2_CERT=`ls $EC2_HOME/cert-*.pem`
    export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/
11. After making the changes you will need to reload your ~/.bash by running the command
    source ~/.bash_profile
12. Now you are ready to start issuing commands to Ec2, list instances and start them. The first step is finding the instance that is appropriate for your needs. You can test with the amazon images that are available and customize them to your needs. To list all of the Amazon instances type the following command.
    $ ec2-describe-images -o amazon
    IMAGE ami-20b65349 ec2-public-images/fedora-core4-base.manifest.xml amazon available public
    IMAGE ami-22b6534b ec2-public-images/fedora-core4-mysql.manifest.xml amazon available public
    IMAGE ami-23b6534a ec2-public-images/fedora-core4-apache.manifest.xml amazon available public
    IMAGE ami-25b6534c ec2-public-images/fedora-core4-apache-mysql.manifest.xmlamazon available public
    IMAGE ami-26b6534f ec2-public-images/developer-image.manifest.xml amazon available public
    IMAGE ami-2bb65342 ec2-public-images/getting-started.manifest.xml amazon available public
    IMAGE ami-36ff1a5f ec2-public-images/fedora-core6-base-x86_64.manifest.xmlamazon available public
    IMAGE ami-bd9d78d4 ec2-public-images/demo-paid-AMI.manifest.xml amazon available public A79EC0DB
13. Out of this bunch you should find at least one suitable to test with, we will use the Fedora Core 4 machine with Apache from above. Before doing this we need a keypair to start the instance. This keypair will be used to gain root access to the instance through SSH after it is up and running.
14. To generate the keypair use the following command, this will create a RSA private key and output it to the screen. You will copy this entire key from ------BEGIN RSA PRIVATE KEY------ TO ------END PRIVATE RSA KEY------. Paste this into a new file named ec2-keypair in your ~/.ec2 directory.
    $ ec2-add-keypair ec2-keypair
15. This step is something that I missed at first and it frustrated me until I figured out what I was doing wrong. Before you can use this key to SSH to a running instance the Ec2 tools require that you set permissions on the file so that only your account has access to the file. You can do that with the command.
    $ chmod 600 ec2-keypair
16. Now we can boot up an ec2 instance. We have chosen the ami-23b6534a instance from above. You will use the following command to start the instance.
    $ ec2-run-instances ami-23b6534a -k ec2-keypair
17. It will take a little while for your instance to start but while you are waiting you can check on the status of the instance with the following command:
    $ ec2-describe-instances
    Once it is up and running you will see "running" as the status. Take note of the server addresses that this command provides since the provide the DNS addresses you will need to access your instance with a web browser or via SSH. They will be in the format of:
    ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com - (Externally accessible DNS address)
    domU-xx-xxx-xxx-xxx.compute-1.internal - (Internally accessible DNS address used from instance to instance)
18. The server instances are locked down pretty tight and you will not have external network access to any of the instances by default. You have control over opening the ports though similar to controlling your own firewall. The network access is not configured uniquely to each instance but instead you control it by groups. You can launch several instances in the same group and provide network access to that group. When you start an instance like we did above it is started as part of the "default" group. We now need to open up network access for web traffic on port 80 and SSH on port 22 with the following commands:
    ec2-authorize default -p 22
    ec2-authorize default -p 80
19. You can now access your instance by opening up your web browser and entering your address http://ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
20. Now you are ready to access the command line of the instance. This is where the private key that you created early comes in. You do not have a root password, instead you use the private key to authenticate yourself. You can access via SSH with the command:
    ssh -i ec2-keypair root@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com

Now you are up and running with your instance. You can change whatever you want and add software to the Linux image. Just remember that it does not persist if you shutdown. If you do a reboot it will persist. After you have made all of the changes you want you can repackage the instance as your own and store it into the Amazon S3 service (LINK TO THESE STEPS)

Challenges of working with Ec2

1. You get a dynamic IP address each time you boot an image. There are solutions with DynDNS that are worth exploring.
2. There is no persistent storage if an instance fails. There are ways to overcome this limitation. So far I have worked with PeristantFS which allows you to mount a bucket from S3 as a directory in your image.
3. You are limited by space in the image to 10GB (I think I need to confirm) if are going to store large files I suggest putting them somewhere in the /mnt directory since that has a lot more space. Also if you save the image anything in the /mnt folder is not saved as part of the image. You can put log files and other content that you don't want saved in this location
4. Databases are a challenge with limited options for persistence. Third parties are popping up offering db hosting on the cloud so you don't have to manage it yourself. I will explore these more in the future.
   

The future of scalable computing....

I really feel like cloud based solutions are the future for hosted solutions. Once you work out some of the limitations you can build a very scalable solution where you have automated scripts that launch new instances as you have a need to scale. In turn you can shut them down as the load decreases. There are overall architecture needs that have to be addressed to utilize an infrastructure like this but it is all doable with a bit of ingenuity. Add in the fact that a small business does not have to invest an significant amount into hardware and software to start running on this type of solution and it is a no brainer. The questions of SLA's come up and I expect that to be an issue for the short term but solvable in the future.

Getting started is easier with RightScale.com

I also used RightScale when I first got started with Ec2, they are a third party that puts a front end onto the managing of ec2 instance. It makes it a lot easier to get started and get your head around Ec2. All you need is an AWS account with Ec2 and S3 and you can get started with RightScale. You do not have to deal with all of the command line stuff above and the Ec2 tools.