The Amazon Elastic Compute Cloud (Amazon EC2) is a web-service allowing companies or individuals to create / run / manage virtual servers (called instances) on their hardware infrastructure. It allows a choice of most operating systems and a range of virtual hardware configurations, so if you wanted lots of processing power you could choose a High CPU instance or for a database server you may choose a High Memory instance. You can of course change instance type with ease. To make this better it allows you to only pay for what you use by the hour and allows creation / destruction of new instances in seconds. It also offers extreme scalability, data storage and fault tolerance. In short it’s a pretty good service!
Until relatively recently (Dec 2009) the instances were not persistent, that is any data you required on a server had to be (re-)created (by scripts) each time a machine was created / rebooted or crashed. This was, and still is, fine for compute intensive batch jobs such as photo processing or number crunching where you create a machine instance, pass it some data and it will give you back the output. That was (essentially) the end of the server’s life. Obviously for a website this kind of arrangement is far from ideal. Thankfully Amazon addressed this issue and created a product called Elastic Block Store (EBS) for this task. A year on and many gremlins have been ironed out and this is now suitable for production website deployment.
Aims of this guide
This guide is not solely dedicated to creating an EBS backed Amazon EC2 instance, that would take a few seconds and wouldn’t really help many people. It is instead designed to guide a relative novice through all the steps required to configure a production Ubuntu 10.04 Lucid Lynx (LTS) server running in the cloud environment. Much of this guide will work on other Linux flavours with certain changes, but I have only tested it on Ubuntu 10.04. Debian based systems will require relatively minor tweaks while RPM / Gentoo / Slackware based distros would require significant code updates (although conceptually it is similar). Likewise it is not required that your server is running in the cloud, but there is some cloud specific content in the guide.
Server Configuration and Scalability
With a small website it is likely that you will only need one server and the size of that will be dictated by your site traffic / server load. As a site grows and more demand is put on the server there eventually becomes a point where you have to make a change. There are a couple of options here:
- Increase the power of the single server
- Separate the server parts i.e. have multiple servers, most commonly mail-server(s), web-server(s), database server(s)
Even if you have a site with low traffic it’s always prudent to plan for the best and have an idea of what to do when the day comes where your server can no longer cope. Amazon EC2 allows an upgrade to a larger instance very quickly, but ultimately you run out of larger instances to use when you hit their Extra Large Instance and become stuck with migration difficulties.
Again Amazon EC2 offers a unique solution to this issue, their EBS storage. It doesn’t just have to be for machine images, EBS volumes can be used essentially as detachable hard disks. This means that you can separate all core (config and data) files for the web-server (e.g. Apache / PHP, site data, vhosts etc), mail-server (e.g. Postfix / Dovecot, e-mails etc) and a database server (e.g. config and data) onto separate “drives” and if the situation ever arises where the server cannot cope you simply detach the required EBS volume from one instance and attach it to another making minimal configuration changes. You can of course copy EBS volumes quickly too and Amazon offers a load balancer if multiple web-servers are used.
Obviously I don’t want to force this choice on anyone (because it does cost a few dollars a month extra) so I have set this up in such a way as it is completely optional to to this. Just follow the relevant section of the guide to your needs.
- Hosting a website on Amazon EC2 - The goals and assumptions of this guide
- Preparing required tools - Create an AWS account, configure Elastic Fox and add an SSH tool
- Customise this guide - Allow all commands to be tailored to you (optional)
- Core software installation - Install some common software to the server image
- Depending upon your chosen configuration there is a choice here:
- Create and attach new EBS volumes - New server that you may want to split in future
- Attach existing EBS volumes - If you have used this guide before and have EBS volumes
- No attached EBS volumes - If you are not using the cloud or don't want to use them
- Depending upon your chosen configuration there is another choice here:
- Software Configuration - Set up the system to work as a multi-function server (from 5a or 5c)
- Software Configuration from existing EBS volumes - Use settings from EBS volumes (from 5b)
- Backing up and clean up - Configure Crons, log rotation etc
Download this guide
To make this guide more useful I've added a feature to allow you to save it offline in a simple HTML format. If you have not customised this guide to your own values you may wish to do so here before you download it. There are a few options here: