EC2 Guide: Core Software Installation (4 / 7)

The image that we are starting with is extremely minimal, and while it is configured to allow a rapid setup for a certain server type (e.g. a web server / mail server), it is often better to install exactly what you need on a machine and nothing more. This process will install everything needed to create a web-server / database server / e-mail server combo box. Of course you could install only the parts applicable to your server if it is a sole purpose box.

Install some basic apps/utilities

We need the following very basic utilities for a multitude of tasks so they go on first.

sudo aptitude install ssh openssh-server nano 

Set a Root User Password

By default the root account has no password, but because it is sometimes advantageous to run as root (not just sudo) it's a good idea to set one. Just don't forget it! (I use my IronKey for this kind of information storage)

sudo passwd 

Set an Ubuntu User Password

The default system user is called "ubuntu" and has a random password assigned to it. There are arguments for and against changing this, but since I use it as my default user and don't always want to count on having my AWS key file handy, I set this. Again, write it down somewhere safe.

sudo passwd ubuntu 

Create an FTP / SFTP User

It's often convenient to be able to move data onto a server (especially for a web server), so we'll add a user just for this.

sudo useradd ftp-user -d /home/ftp-user 
sudo passwd ftp-user 
sudo adduser ftp-user www-data 

Disable root SSH / allow SSH/SFTP with a password

There's no reason to allow the root user to log in via SSH (even if it is IP locked). We also want our other users to be able to log in with a password.

sudo nano /etc/ssh/sshd_config 

Uncomment / update the necessary lines:

[...]
PermitRootLogin no
[...]
PasswordAuthentication yes
[...]

Apply the changes:

sudo /etc/init.d/ssh restart 

Configure aptitude (package manager)

Remove the CD as a source and enable main, universe and multiverse support.

sudo nano /etc/apt/sources.list 
deb http://eu-west-2.ec2.archive.ubuntu.com/ubuntu/ lucid main universe multiverse
deb-src http://eu-west-2.ec2.archive.ubuntu.com/ubuntu/ lucid main universe multiverse
deb http://eu-west-2.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main universe multiverse
deb-src http://eu-west-2.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main universe multiverse
deb http://security.ubuntu.com/ubuntu lucid-security main universe multiverse
deb-src http://security.ubuntu.com/ubuntu lucid-security main universe multiverse
deb http://archive.canonical.com/ lucid partner

Then update the aptitude database:

sudo aptitude update 

Update all installed apps/utilities

sudo aptitude safe-upgrade 

IF a new Kernel has been added a reboot will be required.

sudo reboot 

Configure the network

We need to tell our machine what IP and domain we are using. Update your hosts file to look like this:

sudo nano /etc/hosts 
127.0.0.1       localhost.domain.com   localhost
X.X.X.X   subdomain.domain.com   subdomain

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
sudo hostname subdomain 
sudo nano /etc/hostname 
subdomain

Restart the service:

sudo /etc/init.d/hostname restart 

There are some errors with this service on Ubuntu 10.04 so don't pay too much attention to any message you receive. To check it has worked run:

sudo hostname 
sudo hostname -f 

First should read subdomain and the latter subdomain.domain.com.

Set timezone and synchronize the system clock

Always a good idea to set the server to your preferred time zone and have it synchronise itself. There are perhaps neater ways to do this, but this is reliable!

sudo rm -f /etc/localtime 
sudo ln -sf /usr/share/zoneinfo/Europe/London /etc/localtime 

Make sure our server keeps itself up to date.

sudo aptitude install ntp ntpdate 

Disable AppArmor

This is a security application similar to SELINUX but is often the cause of a lot of painful troubleshooting. It is not required to construct a secure system, anyone who tells you otherwise is wrong!

sudo service apparmor stop 
sudo update-rc.d -f apparmor remove 
sudo aptitude remove apparmor apparmor-utils 

Install MySQL

sudo aptitude install mysql-client mysql-server 
New password for the MySQL "root" user: (choose)
Repeat password for the MySQL "root" user: (repeat)

Check that it is working:

sudo netstat -tap | grep mysql 
tcp 0 0 localhost:mysql *:* LISTEN 7753/mysqld

Install Postfix, Dovecot and Saslauthd

These are all required for our e-mail server (along with MySQL above).

sudo aptitude install postfix postfix-mysql postfix-doc dovecot-common dovecot-postfix dovecot-imapd dovecot-pop3d libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-mysql openssl telnet mailutils 

This should also create the root mysql user and set their password. This is for both Local and external.

General type of mail configuration: (internet site)
System mail name: (domain.com)

Install Amavisd-new, SpamAssassin, ClamAv (and related trimmings)

While not essential for an e-mail server, these anti-spam / anti-virus tools are a very, very good idea!

sudo aptitude install amavisd-new spamassassin clamav clamav-daemon zoo unzip bzip2 arj nomarch lzop cabextract apt-listchanges libnet-ldap-perl libauthen-sasl-perl clamav-docs daemon libio-string-perl libio-socket-ssl-perl libnet-ident-perl zip libnet-dns-perl libmail-spf-query-perl pyzor razor arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip zoo rpm2cpio p7zip-full unrar-free ripole

Install PHP / Apache

These form the basis of most unix based webservers.

sudo aptitude install apache2 apache2.2-common apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libapache2-mod-php5 php5 php5-common php5-gd php5-mysql php5-imap php5-cli php5-cgi libapache2-mod-fcgid php-pear php-auth php5-mcrypt mcrypt php5-curl php5-pspell php5-imagick imagemagick flite 

There are some errors in the following package comments (the wrong comment identifiers have been used)! Open each and update any comments starting with # (hash / pound) to begin with a ; (semi-colon).

sudo nano /etc/php5/cli/conf.d/imagick.ini 
sudo nano /etc/php5/cli/conf.d/imap.ini 
sudo nano /etc/php5/cli/conf.d/mcrypt.ini 

Configure the basic mods that Apache will use.

sudo a2enmod rewrite ssl actions include dav_fs dav auth_digest headers expires 
sudo a2dismod autoindex 

Apply these changes:

sudo service apache2 restart 

Install PHP Extensions (from PECL) (optional)

If you need to add any PECL extensions there are a few other requirements before installation can take place:

sudo aptitude install php5-dev make 

Once they are available to the system any PECL extensions can be added as follows (I'm using Mailparse and Sphinx as my example, you can use any you like!):

sudo pecl install mailparse, sphinx 

To enable them a new configuration file is required for each.

sudo nano /etc/php5/conf.d/mailparse.ini 
; configuration for php Mailparse module
extension=mailparse.so
sudo aptitude install sphinxsearch 
sudo nano /etc/php5/conf.d/sphinx.ini 
; configuration for php Sphinx module
extension=sphinx.so

And then restart Apache to apply the changes:

sudo service apache2 restart 

Update the locate database

The locate command can be extremely useful. (usage example: locate master.cf)

sudo updatedb

Install the EC2 API command line tools (cloud only)

In order to control the ec2 instances / volumes from the server itself (e.g. for backup crons) we need to add a few tools. Also if you are going to add any EBS volumes to the main EBS backed instance the tools for creation and management of the XFS filesystem are required.

sudo aptitude install sun-java6-jre ec2-api-tools xfslibs-dev xfsdump xfsprogs dmapi 

To use these tools the keys that are required must be available to the system. Create a directory to store these X.509 keys. As management will likely be through the ubuntu (default) user we'll store the keys in their directory.

mkdir /home/ubuntu/ec2-keys 
nano /home/ubuntu/ec2-keys/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem 

Add your certificate data (copy / paste) to this file and save it.

nano /home/ubuntu/ec2-keys/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem 

Add your private key data (copy / paste) to this file and save it.

We can also add these keys (and other useful constants) to the global shell configuration files so that they are available to the terminal and login shells (called by crons).

sudo nano /etc/bash.bashrc 
[...]
# Basic convenience
alias ..='cd ..'
alias ...='cd ...'
alias cd..='cd ..'
alias cd-='cd -'
alias ls='ls -lh --color=auto'
alias dir='ls -lh --color=auto'
alias df="df -h"
alias h=history
alias targz="tar cvfz"
alias untargz="tar xvfz"
alias tarbz2="tar cvfj"
alias untarbz2="tar xvfj"
alias unrar="unrar x"
[...]
export EDITOR=nano
export EC2_CERT=/home/ubuntu/ec2-keys/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_PRIVATE_KEY=/home/ubuntu/ec2-keys/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
export EC2_PRIVATE_IP="`wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4`"
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`"
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`"
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
sudo nano /etc/profile 
[...]
# Basic convenience
alias ..='cd ..'
alias ...='cd ...'
alias cd..='cd ..'
alias cd-='cd -'
alias ls='ls -lh --color=auto'
alias dir='ls -lh --color=auto'
alias df="df -h"
alias h=history
alias targz="tar cvfz"
alias untargz="tar xvfz"
alias tarbz2="tar cvfj"
alias untarbz2="tar xvfj"
alias unrar="unrar x"
[...]
export EDITOR=nano
export EC2_CERT=/home/ubuntu/ec2-keys/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export EC2_PRIVATE_KEY=/home/ubuntu/ec2-keys/pk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
export EC2_PRIVATE_IP="`wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4`"
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`"
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`"
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Apply these changes.

rm /home/ubuntu/ec2-keys/*.pem~ 
source /etc/bash.bashrc 
source /etc/profile 
source ~/.bashrc 

Update the permissions on these files:

sudo chmod 0600 /home/ubuntu/ec2-keys/* 

At this point it is likely a good idea to disable your ability to (accidentally) terminate your server!

ec2-modify-instance-attribute $EC2_INSTANCE_ID --disable-api-termination true --region $EC2_REGION 

It should tell you that this has been actioned. If you wish to terminate your instance in the future you need to update this setting to false.

Install Subversion (optional)

I like to use Subversion as my version control system because there's just me, but you may prefer Git, Mercurial, Bazaar or many of the other alternatives. This is not required for the system to run!

sudo aptitude install subversion libapache2-svn 
sudo service apache2 restart 

Add a Remote Desktop system (optional / large)

This isn't something that I would bother with, but if it is absolutely required then FreeNX is excellent.

export DEBIAN_FRONTEND=noninteractive 
sudo aptitude install ubuntu-desktop 
sudo add-apt-repository ppa:freenx-team 
sudo aptitude update 
sudo aptitude install freenx 
cd /usr/lib/nx 
sudo wget http://launchpadlibrarian.net/47959420/nxsetup.tar.gz 
sudo tar xzvf nxsetup.tar.gz 
sudo ./nxsetup --install 

Download and launch the NX client and connect to the server via SSH using a user with a password login.

Download this guide

To make this guide more useful I've added a feature to allow you to save it offline in a simple HTML format. If you have not customised this guide to your own values you may wish to do so here before you download it. There are a few options here:

Guide contents

  1. Hosting a website on Amazon EC2 - The goals and assumptions of this guide
  2. Preparing required tools - Create an AWS account, configure Elastic Fox and add an SSH tool
  3. Customise this guide - Allow all commands to be tailored to you (optional)
  4. Core software installation - Install some common software to the server image
  5. Depending upon your chosen configuration there is a choice here:
    1. Create and attach new EBS volumes - New server that you may want to split in future
    2. Attach existing EBS volumes - If you have used this guide before and have EBS volumes
    3. No attached EBS volumes - If you are not using the cloud or don't want to use them
  6. Depending upon your chosen configuration there is another choice here:
    1. Software Configuration - Set up the system to work as a multi-function server (from 5a or 5c)
    2. Software Configuration from existing EBS volumes - Use settings from EBS volumes (from 5b)
  7. Backing up and clean up - Configure Crons, log rotation etc