The image that we are starting with is extremely minimal, and while it is configured to allow a rapid setup for a certain server type (e.g. a web server / mail server), it is often better to install exactly what you need on a machine and nothing more. This process will install everything needed to create a web-server / database server / e-mail server combo box. Of course you could install only the parts applicable to your server if it is a sole purpose box.
Install some basic apps/utilities
We need the following very basic utilities for a multitude of tasks so they go on first.
sudo aptitude install ssh openssh-server nano
Set a Root User Password
By default the root account has no password, but because it is sometimes advantageous to run as root (not just sudo) it’s a good idea to set one. Just don’t forget it! (I use my IronKey for this kind of information storage)
sudo passwd
Set an Ubuntu User Password
The default system user is called “ubuntu” and has a random password assigned to it. There are arguments for and against changing this, but since I use it as my default user and don’t always want to count on having my AWS key file handy, I set this. Again, write it down somewhere safe.
sudo passwd ubuntu
Create an FTP / SFTP User
It’s often convenient to be able to move data onto a server (especially for a web server), so we’ll add a user just for this.
sudo useradd ftp-user -d /home/ftp-user
sudo passwd ftp-user
sudo adduser ftp-user www-data
Disable root SSH / allow SSH/SFTP with a password
There’s no reason to allow the root user to log in via SSH (even if it is IP locked). We also want our other users to be able to log in with a password.
sudo nano /etc/ssh/sshd_config
Uncomment / update the necessary lines:
[...]
PermitRootLogin no
[...]
PasswordAuthentication yes
[...]
Apply the changes:
sudo /etc/init.d/ssh restart
Configure aptitude (package manager)
Remove the CD as a source and enable main, universe and multiverse support.
sudo nano /etc/apt/sources.list
deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid main universe multiverse
deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid main universe multiverse
deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main universe multiverse
deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main universe multiverse
deb http://security.ubuntu.com/ubuntu lucid-security main universe multiverse
deb-src http://security.ubuntu.com/ubuntu lucid-security main universe multiverse
deb http://archive.canonical.com/ lucid partner
Then update the aptitude database:
sudo aptitude update
Update all installed apps/utilities
sudo aptitude safe-upgrade
IF a new Kernel has been added a reboot will be required.
sudo reboot
Configure the network
We need to tell our machine what IP and domain we are using. Update your hosts file to look like this:
sudo nano /etc/hosts
127.0.0.1 localhost.domain.com localhost
X.X.X.X subdomain.domain.com subdomain
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
sudo hostname subdomain
sudo nano /etc/hostname
subdomain
Restart the service:
sudo /etc/init.d/hostname restart
There are some errors with this service on Ubuntu 10.04 so don’t pay too much attention to any message you receive. To check it has worked run:
sudo hostname
sudo hostname -f
First should read subdomain and the latter subdomain.domain.com.
Set timezone and synchronize the system clock
Always a good idea to set the server to your preferred time zone and have it synchronise itself. There are perhaps neater ways to do this, but this is reliable!
sudo rm -f /etc/localtime
sudo ln -sf /usr/share/zoneinfo/Europe/London /etc/localtime
Make sure our server keeps itself up to date.
sudo aptitude install ntp ntpdate
Disable AppArmor
This is a security application similar to SELINUX but is often the cause of a lot of painful troubleshooting. It is not required to construct a secure system, anyone who tells you otherwise is wrong!
sudo service apparmor stop
sudo update-rc.d -f apparmor remove
sudo aptitude remove apparmor apparmor-utils
Install MySQL
sudo aptitude install mysql-client mysql-server
New password for the MySQL "root" user: (choose)
Repeat password for the MySQL "root" user: (repeat)
Check that it is working:
sudo netstat -tap | grep mysql
tcp 0 0 localhost:mysql *:* LISTEN 7753/mysqld
Install Postfix, Dovecot and Saslauthd
These are all required for our e-mail server (along with MySQL above).
sudo aptitude install postfix postfix-mysql postfix-doc dovecot-common dovecot-postfix dovecot-imapd dovecot-pop3d libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-mysql openssl telnet mailutils
This should also create the root mysql user and set their password. This is for both Local and external.
General type of mail configuration: (internet site)
System mail name: (domain.com)
Install Amavisd-new, SpamAssassin, ClamAv (and related trimmings)
While not essential for an e-mail server, these anti-spam / anti-virus tools are a very, very good idea!
sudo aptitude install amavisd-new spamassassin clamav clamav-daemon zoo unzip bzip2 arj nomarch lzop cabextract apt-listchanges libnet-ldap-perl libauthen-sasl-perl clamav-docs daemon libio-string-perl libio-socket-ssl-perl libnet-ident-perl zip libnet-dns-perl libmail-spf-query-perl pyzor razor arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip zoo rpm2cpio p7zip-full unrar-free ripole
Install PHP / Apache
These form the basis of most unix based webservers.
sudo aptitude install apache2 apache2.2-common apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libapache2-mod-php5 php5 php5-common php5-gd php5-mysql php5-imap php5-cli php5-cgi libapache2-mod-fcgid php-pear php-auth php5-mcrypt mcrypt php5-curl php5-pspell php5-imagick imagemagick flite
There are some errors in the following package comments (the wrong comment identifiers have been used)! Open each and update any comments starting with # (hash / pound) to begin with a ; (semi-colon).
sudo nano /etc/php5/cli/conf.d/imagick.ini
sudo nano /etc/php5/cli/conf.d/imap.ini
sudo nano /etc/php5/cli/conf.d/mcrypt.ini
Configure the basic mods that Apache will use.
sudo a2enmod rewrite ssl actions include dav_fs dav auth_digest headers expires
sudo a2dismod autoindex
Apply these changes:
sudo service apache2 restart
Install PHP Extensions (from PECL) (optional)
If you need to add any PECL extensions there are a few other requirements before installation can take place:
sudo aptitude install php5-dev make
Once they are available to the system any PECL extensions can be added as follows (I’m using Mailparse and Sphinx as my example, you can use any you like!):
sudo pecl install mailparse, sphinx
To enable them a new configuration file is required for each.
sudo nano /etc/php5/conf.d/mailparse.ini
; configuration for php Mailparse module
extension=mailparse.so
sudo aptitude install sphinxsearch
sudo nano /etc/php5/conf.d/sphinx.ini
; configuration for php Sphinx module
extension=sphinx.so
And then restart Apache to apply the changes:
sudo service apache2 restart
Update the locate database
The locate command can be extremely useful. (usage example: locate master.cf)
sudo updatedb
Install the EC2 API command line tools (cloud only)
In order to control the ec2 instances / volumes from the server itself (e.g. for backup crons) we need to add a few tools. Also if you are going to add any EBS volumes to the main EBS backed instance the tools for creation and management of the XFS filesystem are required.
sudo aptitude install sun-java6-jre ec2-api-tools xfslibs-dev xfsdump xfsprogs dmapi
To use these tools the keys that are required must be available to the system. Create a directory to store these X.509 keys. As management will likely be through the ubuntu (default) user we’ll store the keys in their directory.
mkdir /home/ubuntu/ec2-keys
nano /home/ubuntu/ec2-keys/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
Add your certificate data (copy / paste) to this file and save it.
nano /home/ubuntu/ec2-keys/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
Add your private key data (copy / paste) to this file and save it.
We can also add these keys (and other useful constants) to the global shell configuration files so that they are available to the terminal and login shells (called by crons).
sudo nano /etc/bash.bashrc
[...]
# Basic convenience
alias ..='cd ..'
alias ...='cd ...'
alias cd..='cd ..'
alias cd-='cd -'
alias ls='ls -lh --color=auto'
alias dir='ls -lh --color=auto'
alias df="df -h"
alias h=history
alias targz="tar cvfz"
alias untargz="tar xvfz"
alias tarbz2="tar cvfj"
alias untarbz2="tar xvfj"
alias unrar="unrar x"
[...]
export EDITOR=nano
export EC2_CERT=/home/ubuntu/ec2-keys/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_PRIVATE_KEY=/home/ubuntu/ec2-keys/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
export EC2_PRIVATE_IP="`wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4`"
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`"
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`"
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
sudo nano /etc/profile
[...]
# Basic convenience
alias ..='cd ..'
alias ...='cd ...'
alias cd..='cd ..'
alias cd-='cd -'
alias ls='ls -lh --color=auto'
alias dir='ls -lh --color=auto'
alias df="df -h"
alias h=history
alias targz="tar cvfz"
alias untargz="tar xvfz"
alias tarbz2="tar cvfj"
alias untarbz2="tar xvfj"
alias unrar="unrar x"
[...]
export EDITOR=nano
export EC2_CERT=/home/ubuntu/ec2-keys/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_PRIVATE_KEY=/home/ubuntu/ec2-keys/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
export EC2_PRIVATE_IP="`wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4`"
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`"
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`"
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Apply these changes.
rm /home/ubuntu/ec2-keys/*.pem~
source /etc/bash.bashrc
source /etc/profile
source ~/.bashrc
Update the permissions on these files:
sudo chmod 0600 /home/ubuntu/ec2-keys/*
At this point it is likely a good idea to disable your ability to (accidentally) terminate your server!
ec2-modify-instance-attribute $EC2_INSTANCE_ID --disable-api-termination true --region $EC2_REGION
It should tell you that this has been actioned. If you wish to terminate your instance in the future you need to update this setting to false.
Install Subversion (optional)
I like to use Subversion as my version control system because there’s just me, but you may prefer Git, Mercurial, Bazaar or many of the other alternatives. This is not required for the system to run!
sudo aptitude install subversion libapache2-svn
sudo service apache2 restart
Add a Remote Desktop system (optional / large)
This isn’t something that I would bother with, but if it is absolutely required then FreeNX is excellent.
export DEBIAN_FRONTEND=noninteractive
sudo aptitude install ubuntu-desktop
sudo add-apt-repository ppa:freenx-team
sudo aptitude update
sudo aptitude install freenx
cd /usr/lib/nx
sudo wget http://launchpadlibrarian.net/47959420/nxsetup.tar.gz
sudo tar xzvf nxsetup.tar.gz
sudo ./nxsetup --install
Download and launch the NX client and connect to the server via SSH using a user with a password login.
Download this guide
To make this guide more useful I've added a feature to allow you to save it offline in a simple HTML format. If you have not customised this guide to your own values you may wish to do so here before you download it. There are a few options here:
Guide contents
- Hosting a website on Amazon EC2 - The goals and assumptions of this guide
- Preparing required tools - Create an AWS account, configure Elastic Fox and add an SSH tool
- Customise this guide - Allow all commands to be tailored to you (optional)
- Core software installation - Install some common software to the server image
- Depending upon your chosen configuration there is a choice here:
- Create and attach new EBS volumes - New server that you may want to split in future
- Attach existing EBS volumes - If you have used this guide before and have EBS volumes
- No attached EBS volumes - If you are not using the cloud or don't want to use them
- Depending upon your chosen configuration there is another choice here:
- Software Configuration - Set up the system to work as a multi-function server (from 5a or 5c)
- Software Configuration from existing EBS volumes - Use settings from EBS volumes (from 5b)
- Backing up and clean up - Configure Crons, log rotation etc
Hi Paul,
thanks for a very excellent tutorial.
Q: I built a server following your guidelines (except I only added one extra EBS volume for the webserver; I don’t yet need the other 2). I created an A-record to point my godaddy domain to my ec2 elastic ip. I did this on a larger instance for easier development but want to now run it on a reserved micro instance.
I’ve created an AMI from the original c1.medium instance (which performs perfectly); then launch a new micro instance based off this AMI (it launches with a root volume and the attached webserver ebs volume containing all the files for the sites I’m hosting); then take the elastic ip address and point it to the micro instance.
However when I now try to access my websites in a browser, I get either some generic page with php stats of the micro instance (when entering http://www.domain.com) or a ‘not found’ error (when entering http://www.domain.com/somepage).
Do you have any idea what’s going on? (I’m new to this business, so probably something basic.) E.g. are there any actions in your procedure that need to be repeated when starting a new instance?
Thanks very much,
Kevin
Hi Kevin,
Glad you found the guide useful! There shouldn’t be any new actions required to spawn a new instance from a working one – it’s just a copy after all. I just need a touch more information to help you:
- Were the sites operating correctly on the large instance before you created the new micro instance?
- On the micro instance are all of your files correct and available if you visit the mount point for your web-server files (e.g. /webserver)? (I know you’ve said they are)
- Did you configure your vhosts as per this guide or use the standard Ubuntu layout?
- Are you using * and @ rules on GoDaddy or are you testing with individual sub-domains?
Cheers,
Paul
Hi Paul!
Thanks so much for your quick reply. Sorry for the delay on my side – work’s a storm right now and I got distracted by other things.
- The sites were and are operating correctly on the larger instance (I pointed the elastic IP back there for the time being, since I haven’t terminated the larger instance).
- The files are indeed there. I’ve verified file permissions and ownership, too – identical.
- I used your guide.
- I have just the @ and * rules on GoDaddy (nothing else)
- Assigning the elastic IP to the micro definitely works (I can log into it using ssh and the elastic IP). But I still get the purple-and-grey PHP page that seems to display some configuration settings or so when trying to access the webpage with the elastic IP pointed at the micro instance.
Let me repeat that I just boot the new (micro) instance and don’t do anything to it (such as manually starting services etc.).
Thanks so much for your help!
Kevin
* gentle bump *
Hi Kevin,
Sorry for the late reply, afraid I’m away atm so don’t have access to certificates or a computer I trust to test anything. I’m at a loss as to why your files wouldn’t be identical on your new instance if your EBS volume was copied over, but it might be worth checking your mount points (more /etc/fstab) and just visiting a location where the files should exist (e.g. /var/www) to check they are correct.
I can also suggest trying the following method to alter your instance size: http://alestic.com/2011/02/ec2-change-type and see if that is good or not. Unsure if this will play happily with your mount points though – see no reason for it not to…
I’ll update my guide with a few points when I get home.
Cheers,
Paul