Because we changed the paths to our mail logs we need to update our logrotate files. Only update those that have changed:

sudo ##editor## /etc/logrotate.d/rsyslog 

/var/log/postfix/mail.info
/var/log/postfix/mail.warn
/var/log/postfix/mail.err
/var/log/postfix/mail.log
/var/log/dovecot/log
/var/log/dovecot/info
/var/log/dovecot/lda
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                reload rsyslog >/dev/null 2>&1 || true
        endscript
}

Clean up after ourselves:

sudo rm /etc/logrotate.d/rsyslog~ 

Installing a snapshotting tool (cloud only)

One of the great features of EC2 is the ability to take a snapshot of an entire EBS volume (irrespective of size) in about 3-10 seconds. Automating these calls gives some excellent backup potential. I recommend a tool called ec2-consistent-snapshot by Eric Hammond to assist with backups. This is very easy to get set up:

sudo add-apt-repository ppa:alestic 
sudo aptitude update 
sudo aptitude install ec2-consistent-snapshot 

Configure snapshotting (cloud only)

To make our commands simpler and avoid having config files all over the place we can set some environmental variables about our attached EBS volumes. To do this we need to obtain our attached volume IDs from Elasticfox:

• Click on the “Volumes and Snapshots” tab
• Make a note of or copy (right click -> “Copy Volume ID to clipboard”) each of the volume IDs

Back on the server we will set these values into the environmental variables:

sudo ##editor## /etc/bash.bashrc 

[...]
export VOLUME_ID_CORE=##volume_id_core##
export VOLUME_ID_WEBSERVER=##volume_id_webserver##
export VOLUME_ID_EMAIL=##volume_id_email##
export VOLUME_ID_DATABASE=##volume_id_database##
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

sudo ##editor## /etc/profile 

[...]
export VOLUME_ID_CORE=##volume_id_core##
export VOLUME_ID_WEBSERVER=##volume_id_webserver##
export VOLUME_ID_EMAIL=##volume_id_email##
export VOLUME_ID_DATABASE=##volume_id_database##
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

Reload our shell variables:

source /etc/bash.bashrc 
source /etc/profile 
source ~/.bashrc 

We need to make bash script files to run our commands. Firstly we can backup the database volume, which needs to have the MySQL details passed so the snapshot can be taken safely (without losing data).

sudo mkdir /home/ubuntu/snapshotting 
sudo ##editor## /home/ubuntu/snapshotting/database.sh 

#!/bin/bash -l

ec2-consistent-snapshot --description "Database Backup $(date +'%Y-%m-%d %H:%M:%S')" --region "$EC2_REGION" --mysql --mysql-host "localhost" --mysql-socket "/var/run/mysqld/mysqld.sock" --mysql-username "root" --mysql-password "##root_db_password##" --xfs-filesystem /database $VOLUME_ID_DATABASE

The webserver backup is similar, but does not require MySQL to be stopped:

sudo ##editor## /home/ubuntu/snapshotting/webserver.sh 

#!/bin/bash -l

ec2-consistent-snapshot --description "Webserver Backup $(date +'%Y-%m-%d %H:%M:%S')" --region $EC2_REGION --xfs-filesystem /webserver $VOLUME_ID_WEBSERVER

The e-mail backup is almost identical to the webserver and does not require MySQL to be stopped:

sudo ##editor## /home/ubuntu/snapshotting/email.sh 

#!/bin/bash -l

ec2-consistent-snapshot --description "E-mail Backup $(date +'%Y-%m-%d %H:%M:%S')" --region $EC2_REGION --xfs-filesystem /email $VOLUME_ID_EMAIL

These files need to be executable, and we should clean up any temporary files our editor may have left.

sudo chmod +x /home/ubuntu/snapshotting/* 
sudo rm /home/ubuntu/snapshotting/*~ 

In order to freeze / unfreeze the XFS file-system the actual commands need to run as root so we will add our commands to root’s crontab.

su root 

crontab -e 

0 6 * * * /home/ubuntu/snapshotting/database.sh >> /home/ubuntu/backup.log 2>&1
1 6 * * * /home/ubuntu/snapshotting/webserver.sh >> /home/ubuntu/backup.log 2>&1
2 6 * * * /home/ubuntu/snapshotting/email.sh >> /home/ubuntu/backup.log 2>&1

Leave the root user.

exit 

Cleaning up old snapshots (cloud only)

Amazon don’t seem to have a tool to do this neatly, and with a bit of research I found several built in PHP (a language I know) so I decided to roll my own based on a few of these. It’s only a rough draft, but meets my current needs.

sudo ##editor## /var/www/remove_old_snapshots.php 

We need to trigger this as a CLI process so we will wrap its call in a bash wrapper just like the other scripts.

sudo ##editor## /home/ubuntu/snapshotting/remove-old.sh 

#!/bin/bash -l

php /var/www/remove_old_snapshots.php -v $VOLUME_ID_WEBSERVER -v $VOLUME_ID_EMAIL -v $VOLUME_ID_DATABASE -n 7 -r $EC2_REGION -K $EC2_PRIVATE_KEY -C $EC2_CERT

The -n value is the number of snapshots of each volume to keep. If you would like to keep different numbers for each specify -n 3 times (one for each). Again this needs to be executable and our editor may have left a temporary file to clean up:

sudo chmod +x /home/ubuntu/snapshotting/* 
sudo rm /home/ubuntu/snapshotting/*~ 

To remain consistent we will add our commands to root’s crontab.

su root 

crontab -e 

[...]
3 6 * * * /home/ubuntu/snapshotting/remove-old.sh >> /home/ubuntu/backup.log 2>&1

Leave the root user.

exit 

Hopefully that is everything you will need to do to set up a secure server, so I wish you the best of luck with yours! All comments, criticisms and suggestions welcome, but please comment on the relevant pages to make them as useful as possible, thanks!

Configure ClamAV

ClamAV will be our anti-virus system and is very easy to set-up! We just need to update some permissions:

sudo adduser clamav amavis 
sudo adduser amavis clamav 

Configure SpamAssassin

SpamAssassin’s function is given away by it’s name. We will use this for spam filtering our e-mails.

sudo ##editor## /etc/default/spamassassin 

[...]
ENABLED=1
[...]
CRON=1
[...]

We need to create a razor config file for it to work:

sudo razor-admin -create 

Start the service:

sudo service spamassassin start 

Configure Amavisd-New

Amavisd-New will be our interface between our mail transport program (Postfix) and our spam and virus filters (SpamAssassin / ClamAV).

sudo ##editor## /etc/amavis/conf.d/15-content_filter_mode 

Uncomment the spam and anti-virus lines.

use strict;

# You can modify this file to re-enable SPAM checking through spamassassin
# and to re-enable antivirus checking.

#
# Default antivirus checking mode
# Uncomment the two lines below to enable it
#

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

#
# Default SPAM checking mode
# Uncomment the two lines below to enable it
#

@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

1;  # insure a defined return

Some fairly key definitions are missing because they are not “free” – thanks Ubuntu. Enable / uncomment them!

sudo ##editor## /etc/amavis/conf.d/01-debian 

[...]
$unfreeze   = ['unfreeze', 'freeze -d', 'melt', 'fcat'];
[...]
$lha    = 'lha';
[...]

Amavis needs permissions here:

sudo chmod -R 775 /var/lib/amavis/tmp 

sudo service amavis restart 

Configure Mail Logging

Because we mounted parts of our file system through EBS volumes we need to update the mail log locations.

sudo ##editor## /etc/rsyslog.d/50-default.conf 

[...]
mail.*                          -/var/log/postfix/mail.log
[...]
mail.info                       -/var/log/postfix/mail.info
mail.warn                       -/var/log/postfix/mail.warn
mail.err                        /var/log/postfix/mail.err
[..]

sudo service rsyslog restart 

Restart Postfix / Dovecot

Make sure that Amavisd-New is plugged in correctly:

sudo service postfix restart 
sudo service dovecot restart 

ClamAV will be our anti-virus system and is very easy to set-up! We just need to update some permissions:

sudo adduser clamav amavis 
sudo adduser amavis clamav 

Configure SpamAssassin

SpamAssassin’s function is given away by it’s name. We will use this for spam filtering our e-mails.

sudo ##editor## /etc/default/spamassassin 

[...]
ENABLED=1
[...]
CRON=1
[...]

We need to create a razor config file for it to work:

sudo razor-admin -create 

Start the service:

sudo service spamassassin start 

Configure Amavisd-New

Amavisd-New will be our interface between our mail transport program (Postfix) and our spam and virus filters (SpamAssassin / ClamAV).

sudo ##editor## /etc/amavis/conf.d/15-content_filter_mode 

Uncomment the spam and anti-virus lines.

use strict;

# You can modify this file to re-enable SPAM checking through spamassassin
# and to re-enable antivirus checking.

#
# Default antivirus checking mode
# Uncomment the two lines below to enable it
#

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);

#
# Default SPAM checking mode
# Uncomment the two lines below to enable it
#

@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

1;  # insure a defined return

Some fairly key definitions are missing because they are not “free” – thanks Ubuntu. Enable / uncomment them!

sudo ##editor## /etc/amavis/conf.d/01-debian 

[...]
$unfreeze   = ['unfreeze', 'freeze -d', 'melt', 'fcat'];
[...]
$lha    = 'lha';
[...]

Amavis needs permissions here:

sudo chmod -R 775 /var/lib/amavis/tmp 

sudo service amavis restart 

Configure Mail Logging

Because we mounted parts of our file system through EBS volumes we need to update the mail log locations.

sudo ##editor## /etc/rsyslog.d/50-default.conf 

[...]
mail.*                          -/var/log/postfix/mail.log
[...]
mail.info                       -/var/log/postfix/mail.info
mail.warn                       -/var/log/postfix/mail.warn
mail.err                        /var/log/postfix/mail.err
[..]

sudo service rsyslog restart 

Configure MySQL

MySQL set-up largely depends upon your use case and virtual hardware configuration. Shown here is an example, working config.

sudo ##editor## /etc/mysql/my.cnf 

[client]
port                  = 3306
socket                = /var/run/mysqld/mysqld.sock
default-character-set = utf8

[mysqld_safe]
socket                = /var/run/mysqld/mysqld.sock
nice                  = 0

[mysqld]
user                  = mysql
socket                = /var/run/mysqld/mysqld.sock
port                  = 3306
basedir               = /usr
datadir               = /var/lib/mysql
tmpdir                = /tmp

skip-external-locking
default-character-set = utf8
collation_server      = utf8_general_ci
character_set_server  = utf8
bind-address          = 127.0.0.1

key_buffer            = 512M
max_allowed_packet    = 16M
thread_stack          = 192K
thread_cache_size     = 8
myisam-recover        = BACKUP
max_connections       = 200
table_cache           = 1024
thread_concurrency    = 10

query-cache-type      = 1 
query_cache_limit     = 8M
query_cache_size      = 128M

log_error             = /var/log/mysql/error.log

server-id             = 1
expire_logs_days      = 10
max_binlog_size       = 100M

[mysqldump]
quick
quote-names
max_allowed_packet    = 16M

[mysql]
no-auto-rehash
default-character-set = utf8

[isamchk]
key_buffer            = 64M
sort_buffer           = 64M
read_buffer           = 16M
write_buffer          = 16M

[myisamchk]
key_buffer            = 64M
sort_buffer           = 64M
read_buffer           = 16M
write_buffer          = 16M

!includedir /etc/mysql/conf.d/

If you require MySQL to be available from other locations you will need to comment out the “bind-address = 127.0.0.1″ line.

sudo service mysql restart 

Configure PHP

The default installation of PHP on Ubuntu 10.04 is good. There isn’t much wrong with it, but there are always changes that need making. Most notably is the session clean up (which should be exclusively by cron jobs due to potential security risks of giving the web user access) and the configuration of either iconv or mb_string. There are 3 php.ini files, one for Apache (which is the only one I will show here), one for CLI and one for CGI. You should probably make pretty similar changes to all of them.

sudo ##editor## /etc/php5/apache2/php.ini 

engine					= On
short_open_tag				= Off
asp_tags				= Off
precision				= 14
y2k_compliance				= On
output_buffering			= 4096
zlib.output_compression			= Off
implicit_flush				= Off
unserialize_callback_func		=
serialize_precision			= 100
allow_call_time_pass_reference		= Off

safe_mode				= Off
safe_mode_gid				= Off
safe_mode_include_dir			=
safe_mode_exec_dir			=
safe_mode_allowed_env_vars		= PHP_
safe_mode_protected_env_vars		= LD_LIBRARY_PATH

disable_functions			=
disable_classes				=
expose_php				= On
max_execution_time			= 330
max_input_time				= 360
memory_limit				= 128M
error_reporting				= E_ALL & ~E_DEPRECATED
display_errors				= Off
display_startup_errors			= Off
log_errors				= On
log_errors_max_len			= 1024
ignore_repeated_errors			= Off
ignore_repeated_source			= Off
report_memleaks				= On
track_errors				= Off
html_errors				= Off
variables_order				= "GPCS"
request_order				= "GP"
register_globals			= Off
register_long_arrays			= Off
register_argc_argv			= Off
auto_globals_jit			= On
post_max_size				= 18M
magic_quotes_gpc			= Off
magic_quotes_runtime			= Off
magic_quotes_sybase			= Off
auto_prepend_file			=
auto_append_file			=
default_mimetype			= "text/html"
default_charset				= UTF-8
doc_root				=
user_dir				=
enable_dl				= Off
file_uploads				= On
upload_max_filesize			= 12M
max_file_uploads			= 20
allow_url_fopen				= On
allow_url_include			= Off
default_socket_timeout			= 60

pdo_mysql.cache_size			= 2000
pdo_mysql.default_socket		=

define_syslog_variables			= Off

SMTP					= localhost
smtp_port				= 25
mail.add_x_header			= On

sql.safe_mode				= Off

odbc.allow_persistent			= On
odbc.check_persistent			= On
odbc.max_persistent			= -1
odbc.max_links				= -1
odbc.defaultlrl				= 4096
odbc.defaultbinmode			= 1

ibase.allow_persistent			= 1
ibase.max_persistent			= -1
ibase.max_links				= -1
ibase.timestampformat			= "%Y-%m-%d %H:%M:%S"
ibase.dateformat			= "%Y-%m-%d"
ibase.timeformat			= "%H:%M:%S"

mysql.allow_local_infile		= On
mysql.allow_persistent			= On
mysql.cache_size			= 2000
mysql.max_persistent			= -1

mysql.max_links				= -1
mysql.default_port			=
mysql.default_socket			=
mysql.default_host			=
mysql.default_user			=
mysql.default_password			=
mysql.connect_timeout			= 60
mysql.trace_mode			= Off

mysqli.max_persistent			= -1
mysqli.allow_persistent			= On
mysqli.max_links			= -1
mysqli.cache_size			= 2000
mysqli.default_port			= 3306
mysqli.default_socket			=
mysqli.default_host			=
mysqli.default_user			=
mysqli.default_pw			=
mysqli.reconnect			= Off

mysqlnd.collect_statistics		= On
mysqlnd.collect_memory_statistics	= Off

pgsql.allow_persistent			= On
pgsql.auto_reset_persistent		= Off
pgsql.max_persistent			= -1
pgsql.max_links				= -1
pgsql.ignore_notice			= 0
pgsql.log_notice			= 0

sybct.allow_persistent			= On
sybct.max_persistent			= -1
sybct.max_links				= -1
sybct.min_server_severity		= 10
sybct.min_client_severity		= 10

bcmath.scale				= 0

session.save_handler			= files
session.use_cookies			= 1
session.use_only_cookies		= 1
session.name				= PHPSESSID
session.auto_start			= 0
session.cookie_lifetime			= 0
session.cookie_path			= /
session.cookie_domain			=
session.cookie_httponly			=
session.serialize_handler		= php
session.gc_probability			= 0
session.gc_divisor			= 1000
session.gc_maxlifetime			= 2700
session.bug_compat_42			= Off
session.bug_compat_warn			= Off
session.referer_check			=
session.entropy_length			= 0
session.entropy_file			=
session.cache_limiter			= nocache
session.cache_expire			= 180
session.use_trans_sid			= 0
session.hash_function			= 1
session.hash_bits_per_character		= 5

url_rewriter.tags			= "a=href,area=href,frame=src,input=src,form=fakeentry"

mssql.allow_persistent			= On
mssql.max_persistent			= -1
mssql.max_links				= -1
mssql.min_error_severity		= 10
mssql.min_message_severity		= 10
mssql.secure_connection			= Off

mbstring.language			= Neutral
mbstring.internal_encoding		= UTF-8
mbstring.http_input			= auto
mbstring.http_output			= UTF-8
mbstring.encoding_translation		= On
mbstring.detect_order			= auto
mbstring.substitute_character		= none;
mbstring.func_overload			= 0

tidy.clean_output			= Off

soap.wsdl_cache_enabled			= 1
soap.wsdl_cache_dir			= "/tmp"
soap.wsdl_cache_ttl			= 86400
soap.wsdl_cache_limit			= 5

ldap.max_links				= -1

sudo service apache2 graceful  

Configure Apache and a test site

Apache / PHP configuration again largely depends upon your use case. The following will set up a working system and configure a site at “http://www.##domain##” for you.

Create some directories for vhosts and sites first:

sudo mkdir /etc/apache2/vhosts/db-maintenance/ /etc/apache2/vhosts/catch-all/ /etc/apache2/vhosts/redirects/ /var/www/default /var/www/www.##domain## /var/www/db-maintenance.##domain## 

Technically you are not supposed to edit the main apache2.conf file file, but I found some parts of it not to my liking.

sudo ##editor## /etc/apache2/apache2.conf 

ServerRoot "/etc/apache2"
LockFile /var/lock/apache2/accept.lock
PidFile ${APACHE_PID_FILE}
Timeout 300
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 15

<IfModule mpm_prefork_module>
    StartServers          8
    MinSpareServers       5
    MaxSpareServers      20
    MaxClients          256
    MaxRequestsPerChild 4000
</IfModule>

<IfModule mpm_worker_module>
    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0
</IfModule>

<IfModule mpm_event_module>
    StartServers          2
    MaxClients          150
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      25
    MaxRequestsPerChild   0
</IfModule>

User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

AccessFileName .htaccess
<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy all
</Files>

DefaultType text/plain
HostnameLookups Off
ErrorLog /var/log/apache2/error.log
LogLevel warn

Include /etc/apache2/mods-enabled/*.load
Include /etc/apache2/mods-enabled/*.conf
Include /etc/apache2/httpd.conf
Include /etc/apache2/ports.conf

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog /var/log/apache2/access.log vhost_combined

Include /etc/apache2/conf.d/

#Include /etc/apache2/vhosts/db-maintenance/*.conf
Include /etc/apache2/vhosts/*.conf
Include /etc/apache2/vhosts/redirects/*.conf
Include /etc/apache2/vhosts/catch-all/*.conf

This is the user editable file which we will use to set a couple of options and define a default directory.

sudo ##editor## /etc/apache2/httpd.conf 

DocumentRoot "/var/www/default"

<Directory />
    Options FollowSymLinks
    AllowOverride None
</Directory>

<Directory "/var/www/default">
            Options Includes FollowSymLinks MultiViews -Indexes
            AllowOverride AuthConfig
            Order allow,deny
            Allow from all
</Directory>

DirectoryIndex index.php index.html index.htm
AddDefaultCharset UTF-8

Next we will create some default site pages so you can check everything is working. The main (www) site first:

sudo ##editor## /var/www/www.##domain##/index.php 

<?php phpinfo();

Then the database maintenance site:

sudo ##editor## /var/www/db-maintenance.##domain##/index.php

<?php 
echo 'Sorry, this site is currently undergoing scheduled maintenance. 
Please check back soon!';

And a default page to catch incorrect sub-domains:

sudo ##editor## /var/www/default/index.php

<?php 
echo 'We think that you may have mis-typed! 
Were you looking for: <a href="http://www.##domain##">www.##domain##</a>?';

The premise of these files will be that if you visit “http://www.##domain##” you will receive a PHP info screen, but if you visit a none-existent sub-domain (e.g. “http://totally-random.##domain##”) you will end up at the default location (assuming you have set your DNS to a catch all, *, setting in your zone files). If you were to enable the db-maintenance vhosts in the apache2.conf file the idea will be your users will go there instead of your main site. None of this will work yet because we have not set up any vhost files.

The first vhost will be the default (catch-all):

sudo ##editor## /etc/apache2/vhosts/default.conf  

<VirtualHost *:80>
        DocumentRoot /var/www/default/
        ServerName localhost
        <Directory "/var/www/default/">
                Options -Indexes
                AllowOverride AuthConfig
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>

The only key points to note are the choice of document root (you may want the trailing slash, you may not), the exclusion of indexing (i.e. you don’t want anyone able to view all folder contents) and the ability to over-ride the settings using an .htaccess file should you wish to (I prefer to make those changes at this level). The next vhost will be the main site:

sudo ##editor## /etc/apache2/vhosts/www.##domain##.conf  

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        DocumentRoot /var/www/www.##domain##
        ServerName www.##domain##
        <Directory "/var/www/www.##domain##/">
                Options -Indexes FollowSymLinks
                AllowOverride AuthConfig
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>  

Hopefully everything should be fairly obvious and you will note we have used a unique “ServerName” value. Lastly we need the database maintenance vhost:

sudo ##editor## /etc/apache2/vhosts/db-maintenance/www.##domain##.conf  

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        DocumentRoot /var/www/db-maintenance.##domain##
        ServerName www.##domain##
        <Directory "/var/www/db-maintenance.##domain##/">
                Options -Indexes FollowSymLinks
                AllowOverride AuthConfig
                Order allow,deny
                Allow from all
        </Directory>
        <IfModule mod_rewrite.c>
                RewriteEngine On
                RewriteCond %{REQUEST_FILENAME} !-d
                RewriteCond %{REQUEST_FILENAME} !-f
                RewriteRule ^(.*)$ /index.php [L,QSA]
        </IfModule>
</VirtualHost>  

This time we have also included some re-write rules which mean that every address a user tries when maintenance is being carried out (e.g. http://www.##domain##/one-of-your-pages.php) will be handled by just one file, index.php, so you only need to create a single maintenance file. This technique is commonly used in MVC patterns.

Apache needs a restart / reload before these changes are applied:

sudo service apache2 graceful 

Now if you test your site using “http://www.##domain##” you should receive a PHP info screen, but if you visit a none-existent sub-domain (e.g. “http://totally-random.##domain##) you should receive the default message. If you enable database maintenance (by un-commenting the relevant line in the apache2.conf and restarting Apache) and visit your site using “http://www.##domain##/anything.php” you should see the maintenance message. If at this point you don’t see these pages you should check your zone settings with your domain registrar or DNS manager.

Obviously it would be better that all incorrectly typed (non-existing) domains were forwarded on to an actual domain of your choosing. This isn’t hard to achieve with a catch-all vhost:

sudo ##editor## /etc/apache2/vhosts/catch-all/##domain##.conf  

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        ServerName unknown.##domain##
        ServerAlias *.##domain##
        KeepAlive Off
        RewriteEngine On
        RewriteRule ^/(.*)$ http://www.##domain##/$1 [R=301,L]
</VirtualHost>

This time we are using a “ServerAlias” in addition to an arbitrary “ServerName” with a re-write rule to forward everything (including a page name and GET data) to the correct location on the actual site.

It would also be better if the domain without a sub-domain (i.e. http://##domain##) redirected to your main site. Again this isn’t hard to achieve with another vhost:

sudo ##editor## /etc/apache2/vhosts/redirects/##domain##.conf  

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        ServerName ##domain##
        KeepAlive Off
        RewriteEngine On
        RewriteRule ^/(.*)$ http://www.##domain##/$1 [R=301,L]
</VirtualHost>

Nothing new in this one, just another re-write rule. Once again we need to reload our Apche settings:

sudo service apache2 graceful 

Now if you test your site using a none-existent sub-domain (e.g. “http://totally-random.##domain##) or the raw domain “http://##domain##” you should be 301 redirected to “http://www.##domain##.”

Install PhpMyAdmin (optional)

While we are configuring sites it is a good idea to install this very convenient tool to manage MySQL databases. It will need its own vhost:

sudo ##editor## /etc/apache2/vhosts/database.##domain##.conf 

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        DocumentRoot /var/www/database.##domain##
        ServerName database.##domain##
        <Directory "/var/www/database.##domain##/">
                Options -Indexes FollowSymLinks
                AllowOverride AuthConfig
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>

Get the actual program, extract it and edit its settings. You will probably need to customise this link I’m afraid because it changes often!

cd /var/www 
sudo wget "http://sourceforge.net/projects/phpmyadmin/files/phpMyAdmin/##php_my_admin_version##/phpMyAdmin-##php_my_admin_version##-english.tar.gz/download" -O /var/www/phpmyadmin.tar.gz 
sudo tar xvfz /var/www/phpmyadmin.tar.gz 
sudo rm /var/www/phpmyadmin.tar.gz 
sudo mv /var/www/phpMyAdmin-##php_my_admin_version##-english /var/www/database.##domain## 
sudo mv /var/www/database.##domain##/config.sample.inc.php /var/www/database.##domain##/config.inc.php 
sudo ##editor## /var/www/database.##domain##/config.inc.php 

[...]
$cfg['blowfish_secret'] = 'choose a suitably random string here';
[...]
$cfg['Servers'][$i]['extension'] = 'mysqli';
[...]

Clean up any temporary files left by our editor:

sudo rm /var/www/database.##domain##/config.inc.php~ 

It’s also a good idea to password this directory so we’ll create a general password file for use with our restricted sites:

sudo htpasswd -c /var/www/.htpasswd user 

And if you want to add another user:

sudo htpasswd /var/www/.htpasswd another_user 

Now we need to tell the phpMyAdmin directory to use this extra authentication:

sudo ##editor## /var/www/database.##domain##/.htaccess 

AuthUserFile /var/www/.htpasswd
AuthName "Database Management"
AuthType Basic
require valid-user

Restart Apache to check that everything is working:

sudo service apache2 graceful 

If you visit “http://database.##domain##” you should be able to log in and then use your MySQL username (root) and password (##root_db_password##) to manage your MySQL databases.

Install PostfixAdmin

This is the tool that we will use to manage our e-mail domains / users and it must be installed at this stage because it dictates the structure of the database that Postfix and Dovecot will use for e-mail delivery.

We need to create a MySQL database and user for this. This can either be done through phpMyAdmin or via the MySQL command prompt, either way the 4 commands are the same.

mysql -uroot -p##root_db_password##

CREATE USER '##vmail_db_user##'@'%' IDENTIFIED BY '##vmail_db_password##'; 
GRANT USAGE ON * . * TO '##vmail_db_user##'@'%' IDENTIFIED BY '##vmail_db_password##' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0; 
CREATE DATABASE IF NOT EXISTS `##vmail_db##` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; 
GRANT ALL PRIVILEGES ON `##vmail_db##` . * TO '##vmail_db_user##'@'%'; 

exit;

This new site needs a vhost too:

sudo ##editor## /etc/apache2/vhosts/mailadmin.##domain##.conf 

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        DocumentRoot /var/www/mailadmin.##domain##
        ServerName mailadmin.##domain##
        <Directory "/var/www/mailadmin.##domain##/">
                Options -Indexes FollowSymLinks
                AllowOverride AuthConfig
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>

Get the actual program, extract it and edit its settings. You may need to customise this link I’m afraid because it changes occasionally!

cd /var/www 
sudo wget "http://sourceforge.net/projects/postfixadmin/files/postfixadmin/postfixadmin-##postfix_admin_version##/postfixadmin-##postfix_admin_version##.tar.gz/download" -O /var/www/postfixadmin.tar.gz 
sudo tar xvfz /var/www/postfixadmin.tar.gz 
sudo rm /var/www/postfixadmin.tar.gz 
sudo mv /var/www/postfixadmin-##postfix_admin_version## /var/www/mailadmin.##domain## 
sudo ##editor## /var/www/mailadmin.##domain##/config.inc.php 

[...]
$CONF['configured'] = true;
[...]
$CONF['database_type'] = 'mysqli';
$CONF['database_host'] = 'localhost';
$CONF['database_user'] = '##vmail_db_user##';
$CONF['database_password'] = '##vmail_db_password##';
$CONF['database_name'] = '##vmail_db##';
$CONF['database_prefix'] = '';
[...]
$CONF['admin_email'] = 'postmaster@##domain##';
[...]
$CONF['default_aliases'] = array (
    'abuse' => 'abuse@##domain##',
    'hostmaster' => 'hostmaster@##domain##',
    'postmaster' => 'postmaster@##domain##',
    'webmaster' => 'webmaster@##domain##'
);
[...]

Now we need to tell the PostfixAdmin directory to use our general .htpasswd file:

sudo ##editor## /var/www/mailadmin.##domain##/.htaccess 

AuthUserFile /var/www/.htpasswd
AuthName "E-mail Management"
AuthType Basic
require valid-user

Restart Apache to apply the changes:

sudo service apache2 graceful 

You now need to visit “http://mailadmin.##domain##/setup.php” to install the database tables, then enter a password to generate a hash. Please copy this value, but do not proceed with any further prompts. Instead return to the terminal window:

sudo ##editor## /var/www/mailadmin.##domain##/config.inc.php 

[...]
$CONF['setup_password'] = 'the password hash';
[...]

Clean up any temporary files left by our editor:

sudo rm /var/www/database.##domain##/config.inc.php~ 

Now return to “http://mailadmin.##domain##/setup.php”, but don’t refresh, and configure a system admin user using the un-hashed set-up password and whatever details you wish for the account. Then visit “http://mailadmin.##domain##/” and login as your admin user.

Visit the domain list, add ##domain## as a new domain and create any e-mail addresses that you want here (e.g. info@##domain##, your_name@##domain## etc). You should also create the four default accounts (webmaster@##domain##, postmaster@##domain##, abuse@##domain## and hostmaster@##domain##) for it too. These can just be aliases to another account if you wish, but it’s a good idea to have them set up.

Create SSL certificates (for Postfix / Dovecot)

In order to accept secure connections we need to generate some self signed certificates for e-mail use. If you wish you can use official certificates, but that is outside the scope of this guide.

cd /home/ubuntu/smtp-certificates 
sudo openssl req -new -outform PEM -out smtpd.cert -newkey rsa:2048 -nodes -keyout smtpd.key -keyform PEM -days 3650 -x509 

Country Name (2 letter code) [AU]: UK
State or Province Name (full name) [Some-State]: Greater London
Locality Name (eg, city) []: London
Organization Name (eg, company) [Internet Widgits Pty Ltd]: Company Name
Organizational Unit Name (eg, section) []: Trading Name
Common Name (eg, YOUR name) []: ##subdomain##.##domain##
Email Address []: postmaster@##domain##

Create a self-singed root certificate.

sudo openssl req -new -x509 -extensions v3_ca -keyout cakey.pem -out cacert.pem -days 3650 

sudo chmod 0640 /home/ubuntu/smtp-certificates/smtpd.key 
sudo chmod 0640 /home/ubuntu/smtp-certificates/cakey.pem 

For far more detail on this process please visit Ubuntu’s own guide on the subject.

Configure Postfix

Postfix will be our mail server, replacing the ancient Sendmail. Configuration is fairly short, but getting it right is a bit of a black art. To keep our database configuration files contained we’ll put them in a single folder that Postfix owns:

sudo mkdir /etc/postfix/sql 
sudo chown postfix:postfix /etc/postfix/sql 
sudo chmod 0750 /etc/postfix/sql 

What we are essentially doing is using MySQL to determine what happens to the mail. There are 8 different possibilities for this. First up is the relay domains rule, which tells Postfix which domains are to have their mail passed on to another server.

sudo ##editor## /etc/postfix/sql/relay_domains.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT domain FROM domain WHERE domain = '%s' AND backupmx = '1'

We then define a rule for domain forwarding catch-all, e.g. anything@my-domain.com -> anything@mydomain.com:

sudo ##editor## /etc/postfix/sql/virtual_alias_domain_catchall_maps.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT goto FROM alias, alias_domain WHERE alias_domain = '%d' AND address = CONCAT('@', target_domain) AND alias.active = '1' AND alias_domain.active = '1'

We can also define a rule for account forwarding for certain addresses within a domain, e.g. userX@my-domain.com may be forwarded to userX@mydomain.com, but userY@my-domain.com may or may not be forwarded. This query selects the correct mailbox directory for that scenario:

sudo ##editor## /etc/postfix/sql/virtual_alias_domain_mailbox_maps.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT CONCAT(domain, '/', maildir) FROM mailbox, alias_domain WHERE alias_domain = '%d' AND username = CONCAT('%u', '@', alias_domain.target_domain) AND mailbox.active = '1' AND alias_domain.active = '1'

Continuing from the previous rule we also need a query to select the account that a user on an aliased domain will be sent to:

sudo ##editor## /etc/postfix/sql/virtual_alias_domain_maps.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT goto FROM alias, alias_domain WHERE alias_domain = '%d' AND address = CONCAT('%u', '@', target_domain) AND alias.active = '1' AND alias_domain.active = '1'

We can also just define simple aliases on a per address basis, e.g. userX@my-domain.com could be forwarded to anything@yahoo.com, or in fact multiple recipients. This query handles those.

sudo ##editor## /etc/postfix/sql/virtual_alias_maps.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT goto FROM alias WHERE address = '%s' AND active = '1'

We need a simple query to select the mailbox for a certain domain:

sudo ##editor## /etc/postfix/sql/virtual_mailbox_domains.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT domain FROM domain WHERE domain = '%s' AND active = '1'

If the mailboxes are defined with disk space quotas we need to be able to access them:

sudo ##editor## /etc/postfix/sql/virtual_mailbox_limit_maps.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT quota FROM mailbox WHERE username = '%s' AND active = '1'

And finally just get a mailbox location for a certain user on a domain:

sudo ##editor## /etc/postfix/sql/virtual_mailbox_maps.cf 

user		= ##vmail_db_user##
password	= ##vmail_db_password##
hosts		= localhost
dbname		= ##vmail_db##
query		= SELECT CONCAT(domain, '/', maildir) FROM mailbox WHERE username = '%s' AND active = '1'

We can then configure the two main Postfix files:

sudo ##editor## /etc/postfix/main.cf 

smtpd_banner				= $myhostname ESMTP $mail_name
biff					= no
append_dot_mydomain			= no
myhostname				= ##subdomain##.##domain##
alias_maps				= hash:/etc/aliases
alias_database				= hash:/etc/aliases
myorigin				= /etc/mailname
mydestination				= ##subdomain##.##domain##, localhost.##domain##, localhost
relayhost				=
mynetworks				= 127.0.0.0/8
mailbox_size_limit			= 0
recipient_delimiter			= +
inet_interfaces				= all
message_size_limit			= 50000000

# Virtual mail set-up
virtual_mailbox_base			= /home/##vmail_user##
virtual_uid_maps			= static:5000
virtual_gid_maps			= static:5000
virtual_transport			= dovecot
dovecot_destination_recipient_limit	= 1
virtual_create_maildirsize		= yes
virtual_mailbox_extended		= yes
relay_domains				= proxy:mysql:/etc/postfix/sql/relay_domains.cf
virtual_mailbox_domains			= proxy:mysql:/etc/postfix/sql/virtual_mailbox_domains.cf
virtual_alias_maps			= proxy:mysql:/etc/postfix/sql/virtual_alias_maps.cf,
					  proxy:mysql:/etc/postfix/sql/virtual_alias_domain_maps.cf,
					  proxy:mysql:/etc/postfix/sql/virtual_alias_domain_catchall_maps.cf
virtual_mailbox_maps			= proxy:mysql:/etc/postfix/sql/virtual_mailbox_maps.cf,
					  proxy:mysql:/etc/postfix/sql/virtual_alias_domain_mailbox_maps.cf
virtual_mailbox_limit_maps		= proxy:mysql:/etc/postfix/sql/virtual_mailbox_limit_maps.cf
virtual_mailbox_limit_override		= yes
virtual_maildir_limit_message		= Sorry, that user is over their e-mail quota. Please try again later.
virtual_overquota_bounce		= yes

# SASL Related
smtpd_sasl_type				= dovecot
smtpd_sasl_path				= private/auth
smtpd_sasl_local_domain			=
smtpd_sasl_auth_enable			= yes
smtpd_sasl_security_options		= noanonymous
broken_sasl_auth_clients		= yes
smtpd_sasl2_auth_enable			= yes
smtpd_sasl_authenticated_header		= yes
smtpd_sasl_tls_security_options		= noanonymous
#smtpd_sasl_exceptions_networks		= $mynetworks

# TLS Commands
smtpd_tls_auth_only			= no
smtp_tls_security_level			= may
smtpd_tls_security_level		= may
smtp_tls_note_starttls_offer		= yes
smtpd_tls_cert_file			= /home/ubuntu/smtp-certificates/smtpd.cert
smtpd_tls_key_file			= /home/ubuntu/smtp-certificates/smtpd.key
smtpd_tls_CAfile			= /home/ubuntu/smtp-certificates/cacert.pem
smtpd_tls_loglevel			= 1
smtpd_tls_received_header		= yes
smtpd_tls_session_cache_timeout		= 3600s
tls_random_source			= dev:/dev/urandom

# Include Spam settings
disable_vrfy_command			= yes
smtpd_delay_reject			= yes
smtpd_helo_required			= yes
smtpd_helo_restrictions			= permit_mynetworks,
					  reject_non_fqdn_hostname,
					  reject_invalid_hostname,
					  permit

smtpd_recipient_restrictions		= reject_unauth_pipelining,
					  permit_mynetworks,
					  permit_sasl_authenticated,
					  reject_unauth_destination,
					  reject_invalid_hostname,
					  reject_non_fqdn_hostname,
					  reject_non_fqdn_sender,
					  reject_non_fqdn_recipient,
					  reject_unknown_sender_domain,
					  reject_unknown_recipient_domain,
					  reject_rbl_client zen.spamhaus.org,
					  reject_rbl_client dnsbl.sorbs.net,
					  reject_rbl_client dsn.rfc-ignorant.org,
					  reject_rbl_client bl.spamcop.net,
					  permit

smtpd_error_sleep_time			= 1s
smtpd_soft_error_limit			= 10
smtpd_hard_error_limit			= 20

content_filter				= smtp-amavis:[127.0.0.1]:10024

sudo ##editor## /etc/postfix/master.cf 

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       n       -       -       smtpd
smtps     inet  n       -       -       -       -       smtpd
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o broken_sasl_auth_clients=yes
pickup    fifo  n       -       -       60      1       pickup
  -o content_filter=
  -o receive_override_options=no_header_body_checks
cleanup   unix  n       -       -       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       -       -       -       smtp
relay     unix  -       -       -       -       -       smtp
	-o smtp_fallback_relay=
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
retry     unix  -       -       -       -       -       error
discard   unix  -       -       -       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       -       -       -       lmtp
anvil     unix  -       -       -       -       1       anvil
scache    unix  -       -       -       -       1       scache
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix	-	n	n	-	2	pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
  ${nexthop} ${user}
dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient}
smtp-amavis     unix    -       -       -       -       2       smtp
        -o smtp_data_done_timeout=1200
        -o smtp_send_xforward_command=yes
        -o disable_dns_lookups=yes
        -o max_use=20
127.0.0.1:10025 inet    n       -       -       -       -       smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_delay_reject=no
        -o smtpd_client_restrictions=permit_mynetworks,reject
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_data_restrictions=reject_unauth_pipelining
        -o smtpd_end_of_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o smtpd_error_sleep_time=0
        -o smtpd_soft_error_limit=1001
        -o smtpd_hard_error_limit=1000
        -o smtpd_client_connection_count_limit=0
        -o smtpd_client_connection_rate_limit=0
        -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

That is all for Postfix, but before we can use the mail system we need to tweak Dovecot.

Configure Dovecot

Dovecot will be our POP3(S) / IMAP(S) server and handle our SASL authentication. Again, there’s not too much to set up, but if you get a setting or two wrong, it simply won’t work properly!

We are using Dovecot to handle our authentication, and again we will do this from our MySQL tables to ease management:

sudo ##editor## /etc/dovecot/dovecot-sql.conf 

driver			= mysql
connect			= host=127.0.0.1 dbname=##vmail_db## user=##vmail_db_user## password=##vmail_db_password##
user_query		= SELECT concat('/home/##vmail_user##/', domain, '/', maildir) as home, concat('maildir:/home/##vmail_user##/', domain, '/', maildir) as mail, 5000 AS uid, 5000 AS gid, concat('maildir:storage=', quota) AS quota FROM mailbox WHERE username = '%u' AND active = '1'
default_pass_scheme	= MD5
password_query		= SELECT username as user, password FROM mailbox WHERE username = '%u' AND active = '1'

And then we need to configure Dovecot’s main configuration file:

sudo ##editor## /etc/dovecot/dovecot.conf 

protocols		= pop3 pop3s imap imaps
log_path		= /var/log/dovecot/log
info_log_path		= /var/log/dovecot/info
log_timestamp		= "%Y-%m-%d %H:%M:%S "
ssl			= yes
ssl_cert_file		= /home/ubuntu/smtp-certificates/smtpd.cert
ssl_key_file		= /home/ubuntu/smtp-certificates/smtpd.key
disable_plaintext_auth	= no
login_dir		= /var/run/dovecot/login
login_chroot		= yes
login_user		= dovecot
login_greeting		= Dovecot ready.
mail_location		= maildir:/home/##vmail_user##/%d/%u
mail_debug		= no

auth_executable		= /usr/lib/dovecot/dovecot-auth
auth_verbose		= no
auth_debug		= no
auth_debug_passwords	= no

protocol imap {
  login_executable	= /usr/lib/dovecot/imap-login
  mail_executable	= /usr/lib/dovecot/imap
  imap_max_line_length	= 65536
}

protocol pop3 {
  login_executable	= /usr/lib/dovecot/pop3-login
  mail_executable	= /usr/lib/dovecot/pop3
  pop3_uidl_format	= %08Xu%08Xv
}

protocol lda {
  log_path		= /var/log/dovecot/lda
  auth_socket_path	= /var/run/dovecot/auth-master
  postmaster_address	= postmaster@##domain##
}

auth default {
  mechanisms		= plain login

  passdb sql {
    args		= /etc/dovecot/dovecot-sql.conf
  }

  userdb sql {
    args		= /etc/dovecot/dovecot-sql.conf
  }

  socket listen {
    master {
      path		= /var/run/dovecot/auth-master
      mode		= 0600
      user		= ##vmail_user##
      group		= ##vmail_group##
   }

    client {
       path		= /var/spool/postfix/private/auth
       mode		= 0660
       user		= postfix
       group		= postfix
    }
  }
}

There are some files designed to be automatically included. They shouldn’t be, but just in case:

sudo mv /etc/dovecot/conf.d/01-dovecot-postfix.conf 01-dovecot-postfix.conf.bak 
sudo mv /etc/dovecot/auth.d/01-dovecot-postfix.auth 01-dovecot-postfix.auth.bak 

Before we can use the mail system the new settings need to be loaded in.

sudo service dovecot restart 
sudo service postfix restart 

Configure Squirrelmail (optional)

Squirrelmail is a convenient web-mail client. It will need a vhost file:

sudo ##editor## /etc/apache2/vhosts/mail.##domain##.conf 

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        DocumentRoot /var/www/mail.##domain##
        ServerName mail.##domain##
        <Directory "/var/www/mail.##domain##/">
                Options -Indexes FollowSymLinks
                AllowOverride AuthConfig
                Order allow,deny
                Allow from all
        </Directory>
</VirtualHost>

Get the actual program, extract it and edit its settings. You may need to check this link I’m afraid because it may change.

cd /var/www 
sudo wget "http://sourceforge.net/projects/squirrelmail/files/stable/##squirrelmail_version##/squirrelmail-##squirrelmail_version##.tar.gz/download" -O /var/www/squirrelmail.tar.gz 
sudo tar xvfz /var/www/squirrelmail.tar.gz 
sudo rm /var/www/squirrelmail.tar.gz 
sudo mv /var/www/squirrelmail-##squirrelmail_version## /var/www/mail.##domain## 

Squirrelmail needs a private folder, so we need to create and secure it:

sudo mkdir /var/www/private 
sudo mkdir /var/www/private/mail.##domain## 
sudo mkdir /var/www/private/mail.##domain##/data 
sudo mkdir /var/www/private/mail.##domain##/attachments 
sudo chown -R root:www-data /var/www/private/mail.##domain##  
sudo chmod -R 0730 /var/www/private/mail.##domain## 

We can now configure it. There is a Perl script to do this, but I find it faster just to edit the settings manually.

sudo mv /var/www/mail.##domain##/config/config_default.php /var/www/mail.##domain##/config/config.php 
sudo ##editor## /var/www/mail.##domain##/config/config.php 

[...]
$org_name = '##website_name##';
[...]
$domain = '##domain##';
[...]
$smtp_auth_mech = 'plain';
[...]
$data_dir = '/var/www/private/mail.##domain##/data/';
[...]
$attachment_dir = '/var/www/private/mail.##domain##/attachments/';
[...]
$default_charset = 'utf-8';
[...]

Clean up any temporary files left by our editor:

sudo rm /var/www/mail.##domain##/config/config.php~ 

To apply the changes Apache needs to have the data reloaded:

sudo service apache2 graceful 

If you visit “http://mail.##domain##” you will be able to send and receive e-mails. Of course this is no replacement for a mail client, but it’s useful to have on occasion!

Configure Subversion (optional)

We will bring your website (www.##domain##) under version control. This step will create the repositories and set their permissions to allow the web user access.

It’s unlikely that you’ll want anyone to be able to access your files so we can generate a password file for SVN access:

sudo htpasswd -c /svn-repositories/passwd svn-username 

We can then create a folder, a repository within the folder and import our files:

sudo mkdir /svn-repositories/www.##domain## 
sudo svnadmin create /svn-repositories/www.##domain## 
sudo svn import /var/www/www.##domain## file:////svn-repositories/www.##domain## 

This repository needs to be owned by the web user and be granted certain permissions:

sudo chown -R www-data:www-data /svn-repositories/www.##domain## 
sudo chmod -R g+rws /svn-repositories/www.##domain## 

We can then remove our existing site directory and recreate it from our repository:

cd /var/www 
sudo rm -rf /var/www/www.##domain##/ 
sudo svn co file:///svn-repositories/www.##domain## 
sudo chown -R www-data:www-data /var/www/www.##domain##/ 

We can also create a trigger so that every time you commit a change it goes live. This probably wants some modification for production use, i.e. Local -> Staging -> Production with the later step conducted manually.

sudo ##editor## /svn-repositories/www.##domain##/hooks/post-commit 

#!/bin/bash

cd /var/www/www.##domain##
/usr/bin/svn update

This also needs the correct permissions to run:

sudo chown www-data:www-data /svn-repositories/www.##domain##/hooks/post-commit 
sudo chmod +x /svn-repositories/www.##domain##/hooks/post-commit 

We also need to add this repository to a vhost file so that you can check it out. I would suggest a new one for this purpose. Multiple locations can be added, but for now we just have one.

sudo ##editor## /etc/apache2/vhosts/svn.##domain##.conf 

<VirtualHost *:80>
        ServerAdmin webmaster@##domain##
        DocumentRoot /var/www/default/
        ServerName svn.##domain##
        <Directory "/var/www/default/">
                Options -Indexes FollowSymLinks
                AllowOverride FileInfo
                Order allow,deny
                Allow from all
        </Directory>
        <Location /svn/www.##domain##>
        	DAV svn
        	SVNPath /svn-repositories/www.##domain##
        	AuthType Basic
        	AuthName "www.##domain## subversion repository"
        	AuthUserFile /svn-repositories/passwd
        	Require valid-user
        </Location>
</VirtualHost>

Before this can be tested Apache needs to be reloaded:

sudo service apache2 graceful 

You can now check out the repository from “http://svn.##domain##/svn/www.##domain##” using the command line or client (Linux / OSX) or a client such as Tortoise SVN on Windows. You will need to validate using the username / password that you created in this step. If you would rather use the .htpasswd file used for the database and mail admin just update the vhost file with that location.

Required Elements

sudo mkdir /home/ubuntu/smtp-certificates /var/log/dovecot /var/log/postfix /var/spool/##vmail_user## /var/##vmail_user## /svn-repositories 

sudo groupadd -g 5000 ##vmail_group## 
sudo useradd -g ##vmail_group## -u 5000 ##vmail_user##-d /home/##vmail_user## 
sudo chown ##vmail_user##:##vmail_group## /home/##vmail_user## 

sudo chown syslog:##vmail_group## /var/log/dovecot 
sudo chown syslog:##vmail_group## /var/log/postfix 
sudo chown ##vmail_user##:mail /var/spool/##vmail_user##
sudo chmod 0775 /var/spool/##vmail_user## 

Attach the EBS Volumes

We are going to use the same configuration as during initial creation. So, in Elasticfox:

• Click on the “Volumes and Snapshots” tab
• Select each EBS volume individually and click on the green plus button to attach it to your required instance
• Enter a path for each (the range is sdc to sdp – sda and sdb are already in use)
  • For the web-server: /dev/sd##webserver_letter##
  • For the database: /dev/sd##database_letter## – “d” for database may be a good choice
  • For the e-mails: /dev/sd##email_letter## – “e” for e-mails may be a good choice

Then back on the server:

sudo mkdir -m 000 /##webserver_path## 
sudo mkdir -m 000 /##database_path## 
sudo mkdir -m 000 /##email_path## 

echo "/dev/sd##webserver_letter## /##webserver_path## xfs noatime 0 0" | sudo tee -a /etc/fstab 
echo "/dev/sd##database_letter## /##database_path## xfs noatime 0 0" | sudo tee -a /etc/fstab 
echo "/dev/sd##email_letter## /##email_path## xfs noatime 0 0" | sudo tee -a /etc/fstab 

sudo mount -a 

Make MySQL use the EBS volume data

This assumes we have an EBS volume with a mount point called “##database_path##” attached that contains all necessary data / permissions.

sudo service mysql stop 

Remove some existing MySQL parts so they can be mounted with those already available on the EBS volume.

sudo mv /etc/mysql /etc/mysql.ORIG 
sudo mv /var/lib/mysql /var/lib/mysql.ORIG 
sudo mv /var/log/mysql /var/log/mysql.ORIG 

sudo mkdir /etc/mysql /var/lib/mysql /var/log/mysql 
sudo chown mysql:mysql /var/lib/mysql 
sudo chmod 0700 /var/lib/mysql
sudo chown mysql:adm /var/log/mysql 
sudo chmod 0750 /var/log/mysql 
sudo chmod g+s /var/log/mysql 

echo "/##database_path##/etc/mysql /etc/mysql none bind" | sudo tee -a /etc/fstab 
echo "/##database_path##/data /var/lib/mysql none bind" | sudo tee -a /etc/fstab 
echo "/##database_path##/log /var/log/mysql none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

sudo service mysql start 

Make Apache / PHP use the EBS volume data

This assumes we have an EBS volume with a mount point called “##webserver_path##” attached that contains all necessary data / permissions.

sudo service apache2 stop 

Remove some existing Apache parts so they can be mounted with those already available on the EBS volume.

sudo mv /etc/apache2 /etc/apache2.ORIG 
sudo mv /var/log/apache2 /var/log/apache2.ORIG  
sudo mv /var/www /var/www.ORIG 
sudo mv /etc/php5 /etc/php5.ORIG 

sudo mkdir /etc/apache2 /etc/apache2/vhosts /var/log/apache2 /var/www /etc/php5 
sudo chown root:adm /var/log/apache2 

echo "/##webserver_path##/apache2 /etc/apache2 none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/vhosts /etc/apache2/vhosts none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/log /var/log/apache2 none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/sites /var/www none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/php5 /etc/php5 none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

sudo service apache2 start 

Make Postfix / Dovecot use the EBS volume data

This assumes we have an EBS volume with a mount point called “##email_path##” attached that contains all necessary data / permissions.

sudo service postfix stop 
sudo service dovecot stop 

Remove some existing Postfix / Dovecot parts so they can be mounted with those already available on the EBS volume. Because we are going to use a MySQL controlled, virtual mail setup we also need to create a new user and set some permissions.

sudo mkdir /home/ubuntu/smtp-certificates /var/log/dovecot /var/log/postfix /var/spool/##vmail_user## /home/##vmail_user## 

sudo groupadd -g 5000 ##vmail_group## 
sudo useradd -g ##vmail_group## -u 5000 ##vmail_user## -d /home/##vmail_user## 
sudo chown ##vmail_user##:##vmail_group## /home/##vmail_user## 

sudo chown ##vmail_user##:adm /var/log/dovecot 
sudo chown syslog:adm /var/log/postfix 
sudo chown ##vmail_user##:mail /var/spool/##vmail_user## 
sudo chmod 0775 /var/spool/##vmail_user## 

sudo mv /etc/postfix /etc/postfix.ORIG 
sudo mv /etc/dovecot /etc/dovecot.ORIG 
sudo mv /var/spool/postfix /var/spool/postfix.ORIG 

sudo mkdir /etc/postfix /etc/dovecot /var/spool/postfix 

echo "/##email_path##/postfix /etc/postfix none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/dovecot /etc/dovecot none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/spool/postfix /var/spool/postfix none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/spool/##vmail_user## /var/spool/##vmail_user## none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/##vmail_user## /home/##vmail_user## none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/smtp-certificates /home/ubuntu/smtp-certificates none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/log/dovecot /var/log/dovecot none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/log/postfix /var/log/postfix none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

sudo service postfix start 
sudo service dovecot start 

Make Subversion use the EBS volume data (optional)

sudo mkdir /svn-repositories 

echo "/##webserver_path##/svn /svn-repositories none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

Create the EBS volumes

Using Elasticfox you will need to create 3 EBS volumes (Database, Web-server and Mail-server):

• Click on the “Volumes and Snapshots” tab
• Click on the green plus button
• Choose a size and an availability zone for the volume (must be the same as your server instance: ##cloud_region####availability_zone##)
• The size cannot be altered dynamically, but can be changed in the future if required using a snapshot
• Tag each sensibly! (so you can tell which is which)

Attaching the EBS volumes

Using Elasticfox:

• Click on the “Volumes and Snapshots” tab
• Select each EBS volume individually and click on the green plus to attach it to your running instance
• Enter a path for each (the range is sdc to sdp – sda and sdb are already in use)
  • For the web-server: /dev/sd##webserver_letter##
  • For the database: /dev/sd##database_letter## – “d” for database may be a good choice
  • For the e-mails: /dev/sd##email_letter## – “e” for e-mails may be a good choice

Formatting the EBS volumes

Back to the server, where your new volumes are now attached we need to create a file-system on them. XFS is generally used for this:

sudo mkfs.xfs /dev/sd##webserver_letter## 
sudo mkfs.xfs /dev/sd##database_letter## 
sudo mkfs.xfs /dev/sd##email_letter## 

Mounting the EBS volumes in useful locations

Choose descriptive path names for each!

echo "/dev/sd##webserver_letter## /##webserver_path## xfs noatime 0 0" | sudo tee -a /etc/fstab 
sudo mkdir -m 000 /##webserver_path## 
sudo mount /##webserver_path## 

echo "/dev/sd##database_letter## /##database_path## xfs noatime 0 0" | sudo tee -a /etc/fstab 
sudo mkdir -m 000 /##database_path## 
sudo mount /##database_path## 

echo "/dev/sd##email_letter## /##email_path## xfs noatime 0 0" | sudo tee -a /etc/fstab 
sudo mkdir -m 000 /##email_path## 
sudo mount /##email_path## 

Move MySQL to an EBS volume

This assumes we have an EBS volume with a mount point called “##database_path##” attached.

sudo service mysql stop 

Move the existing MySQL parts to run on the EBS volume, but keep their ‘place-holders’ populated through mount points.

sudo mkdir /##database_path##/etc /##database_path##/log /##database_path##/data 

sudo chown mysql:mysql /##database_path##/data 
sudo chmod 0700 /##database_path##/data 
sudo chown mysql:adm /##database_path##/log
sudo chmod 0750 /##database_path##/log 
sudo chmod g+s /##database_path##/log

sudo cp -R /etc/mysql /etc/mysql.ORIG 
sudo cp -R /var/lib/mysql /var/lib/mysql.ORIG 
sudo cp -R /var/log/mysql /var/log/mysql.ORIG 

sudo mv /etc/mysql /##database_path##/etc/ 
sudo mv /var/lib/mysql/* /##database_path##/data/ 
sudo mv /var/log/mysql/* /##database_path##/log/ 

sudo mkdir /etc/mysql

echo "/##database_path##/etc/mysql /etc/mysql none bind" | sudo tee -a /etc/fstab 
echo "/##database_path##/data /var/lib/mysql none bind" | sudo tee -a /etc/fstab 
echo "/##database_path##/log /var/log/mysql none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

sudo service mysql start 

Move Apache / PHP / sites to an EBS volume

This assumes we have an EBS volume with a mount point called “##webserver_path##” attached.

sudo service apache2 stop 

Move some Apache parts to run on the EBS volume, but keep the ‘place-holders’ populated through mount points.

sudo cp -R /etc/apache2 /etc/apache2.ORIG 
sudo cp -R /var/log/apache2 /var/log/apache2.ORIG 
sudo cp -R /var/www /var/www.ORIG 
sudo cp -R /etc/php5 /etc/php5.ORIG 

sudo mv /etc/apache2 /##webserver_path##/ 
sudo mv /var/log/apache2/ /##webserver_path##/log/ 
sudo mv /var/www/ /##webserver_path##/sites/ 
sudo mv /etc/php5 /##webserver_path##/ 

sudo mkdir /##webserver_path##/vhosts /##webserver_path##/apache2/vhosts /etc/apache2 /etc/apache2/vhosts /var/log/apache2 /var/www /etc/php5 

echo "/##webserver_path##/apache2 /etc/apache2 none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/vhosts /etc/apache2/vhosts none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/log /var/log/apache2 none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/sites /var/www none bind" | sudo tee -a /etc/fstab 
echo "/##webserver_path##/php5 /etc/php5 none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

sudo service apache2 start 

Move E-mail / Postfix / Dovecot to an EBS volume

This assumes we have an EBS volume with a mount point called “##email_path##” attached.

sudo service postfix stop 
sudo service dovecot stop 

Move some E-mail related parts to run on the EBS volume, but keep the ‘place-holders’ populated through mount points. Because we are going to use a MySQL controlled, virtual mail set-up we also need to create a new user and set some permissions.

sudo mkdir /##email_path##/##vmail_user## /##email_path##/log /##email_path##/log/postfix /##email_path##/log/dovecot /##email_path##/spool /##email_path##/spool/postfix/ /##email_path##/spool/##vmail_user##/ /##email_path##/smtp-certificates /home/ubuntu/smtp-certificates /var/log/dovecot /var/log/postfix /var/spool/##vmail_user## /home/##vmail_user##

sudo groupadd -g 5000 ##vmail_group## 
sudo useradd -g ##vmail_group## -u 5000 ##vmail_user## -d /home/##vmail_user## 
sudo chown ##vmail_user##:##vmail_group## /home/##vmail_user## 

sudo chown ##vmail_user##:adm /var/log/dovecot 
sudo chown syslog:adm /var/log/postfix 
sudo chown ##vmail_user##:mail /var/spool/##vmail_user##
sudo chmod 0775 /var/spool/##vmail_user## 

sudo chown ##vmail_user##:adm /##email_path##/log/dovecot 
sudo chown syslog:adm /##email_path##/log/postfix 
sudo chown ##vmail_user##:##vmail_group## /##email_path##/##vmail_user## 
sudo chown ##vmail_user##:##vmail_group## /##email_path##/log 
sudo chown ##vmail_user##:##vmail_group## /email/spool/postfix 
sudo chown ##vmail_user##:mail /##email_path##/spool/##vmail_user## 
sudo chmod 0775 /##email_path##/spool/##vmail_user## 

sudo cp -R /etc/postfix /etc/postfix.ORIG 
sudo cp -R /etc/dovecot /etc/dovecot.ORIG 
sudo cp -R /var/spool/postfix /var/spool/postfix.ORIG 

sudo mv /etc/postfix/ /##email_path##/postfix 
sudo mv /etc/dovecot/ /##email_path##/dovecot 
sudo mv /var/spool/postfix/* /##email_path##/spool/postfix 

sudo mkdir /etc/postfix /etc/dovecot /etc/postfix/virtual

echo "/##email_path##/postfix /etc/postfix none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/dovecot /etc/dovecot none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/spool/postfix /var/spool/postfix none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/spool/##vmail_user## /var/spool/##vmail_user## none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/##vmail_user## /home/##vmail_user## none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/smtp-certificates /home/ubuntu/smtp-certificates none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/log/dovecot /var/log/dovecot none bind" | sudo tee -a /etc/fstab 
echo "/##email_path##/log/postfix /var/log/postfix none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

sudo service postfix start 
sudo service dovecot start 

Move Subversion to an EBS Volume (optional)

This assumes we have an EBS volume with a mount point called “##webserver_path##” attached.

We have no subversion repository location yet, so we can just create one:

sudo mkdir /##webserver_path##/svn /svn-repositories 

echo "/##webserver_path##/svn /svn-repositories none bind" | sudo tee -a /etc/fstab 

sudo mount -a 

Install some basic apps/utilities

We need the following very basic utilities for a multitude of tasks so they go on first.

sudo aptitude install ssh openssh-server ##editor## 

Set a Root User Password

By default the root account has no password, but because it is sometimes advantageous to run as root (not just sudo) it’s a good idea to set one. Just don’t forget it! (I use my IronKey for this kind of information storage)

sudo passwd 

Set an Ubuntu User Password

The default system user is called “ubuntu” and has a random password assigned to it. There are arguments for and against changing this, but since I use it as my default user and don’t always want to count on having my AWS key file handy, I set this. Again, write it down somewhere safe.

sudo passwd ubuntu 

Create an FTP / SFTP User

It’s often convenient to be able to move data onto a server (especially for a web server), so we’ll add a user just for this.

sudo useradd ##ftp_username## -d /home/##ftp_username## 
sudo passwd ##ftp_username## 
sudo adduser ##ftp_username## www-data 

Disable root SSH / allow SSH/SFTP with a password

There’s no reason to allow the root user to log in via SSH (even if it is IP locked). We also want our other users to be able to log in with a password.

sudo ##editor## /etc/ssh/sshd_config 

Uncomment / update the necessary lines:

[...]
PermitRootLogin no
[...]
PasswordAuthentication yes
[...]

Apply the changes:

sudo /etc/init.d/ssh restart 

Configure aptitude (package manager)

Remove the CD as a source and enable main, universe and multiverse support.

sudo ##editor## /etc/apt/sources.list 

deb http://##aptitude_update##.archive.ubuntu.com/ubuntu/ lucid main universe multiverse
deb-src http://##aptitude_update##.archive.ubuntu.com/ubuntu/ lucid main universe multiverse
deb http://##aptitude_update##.archive.ubuntu.com/ubuntu/ lucid-updates main universe multiverse
deb-src http://##aptitude_update##.archive.ubuntu.com/ubuntu/ lucid-updates main universe multiverse
deb http://security.ubuntu.com/ubuntu lucid-security main universe multiverse
deb-src http://security.ubuntu.com/ubuntu lucid-security main universe multiverse
deb http://archive.canonical.com/ lucid partner

Then update the aptitude database:

sudo aptitude update 

Update all installed apps/utilities

sudo aptitude safe-upgrade 

IF a new Kernel has been added a reboot will be required.

sudo reboot 

Configure the network

We need to tell our machine what IP and domain we are using. Update your hosts file to look like this:

sudo ##editor## /etc/hosts 

127.0.0.1       localhost.##domain##   localhost
##ip##   ##subdomain##.##domain##   ##subdomain##

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

sudo hostname ##subdomain## 
sudo ##editor## /etc/hostname 

##subdomain##

Restart the service:

sudo /etc/init.d/hostname restart 

There are some errors with this service on Ubuntu 10.04 so don’t pay too much attention to any message you receive. To check it has worked run:

sudo hostname 
sudo hostname -f 

First should read ##subdomain## and the latter ##subdomain##.##domain##.

Set timezone and synchronize the system clock

Always a good idea to set the server to your preferred time zone and have it synchronise itself. There are perhaps neater ways to do this, but this is reliable!

sudo rm -f /etc/localtime 
sudo ln -sf /usr/share/zoneinfo/##timezone## /etc/localtime 

Make sure our server keeps itself up to date.

sudo aptitude install ntp ntpdate 

Disable AppArmor

This is a security application similar to SELINUX but is often the cause of a lot of painful troubleshooting. It is not required to construct a secure system, anyone who tells you otherwise is wrong!

sudo service apparmor stop 
sudo update-rc.d -f apparmor remove 
sudo aptitude remove apparmor apparmor-utils 

Install MySQL

sudo aptitude install mysql-client mysql-server 

New password for the MySQL "root" user: (choose)
Repeat password for the MySQL "root" user: (repeat)

Check that it is working:

sudo netstat -tap | grep mysql 

tcp 0 0 localhost:mysql *:* LISTEN 7753/mysqld

Install Postfix, Dovecot and Saslauthd

These are all required for our e-mail server (along with MySQL above).

sudo aptitude install postfix postfix-mysql postfix-doc dovecot-common dovecot-postfix dovecot-imapd dovecot-pop3d libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-mysql openssl telnet mailutils 

This should also create the root mysql user and set their password. This is for both Local and external.

General type of mail configuration: (internet site)
System mail name: (##domain##)

Install Amavisd-new, SpamAssassin, ClamAv (and related trimmings)

While not essential for an e-mail server, these anti-spam / anti-virus tools are a very, very good idea!

sudo aptitude install amavisd-new spamassassin clamav clamav-daemon zoo unzip bzip2 arj nomarch lzop cabextract apt-listchanges libnet-ldap-perl libauthen-sasl-perl clamav-docs daemon libio-string-perl libio-socket-ssl-perl libnet-ident-perl zip libnet-dns-perl libmail-spf-query-perl pyzor razor arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip zoo rpm2cpio p7zip-full unrar-free ripole

Install PHP / Apache

These form the basis of most unix based webservers.

sudo aptitude install apache2 apache2.2-common apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libapache2-mod-php5 php5 php5-common php5-gd php5-mysql php5-imap php5-cli php5-cgi libapache2-mod-fcgid php-pear php-auth php5-mcrypt mcrypt php5-curl php5-pspell php5-imagick imagemagick flite 

There are some errors in the following package comments (the wrong comment identifiers have been used)! Open each and update any comments starting with # (hash / pound) to begin with a ; (semi-colon).

sudo ##editor## /etc/php5/cli/conf.d/imagick.ini 
sudo ##editor## /etc/php5/cli/conf.d/imap.ini 
sudo ##editor## /etc/php5/cli/conf.d/mcrypt.ini 

Configure the basic mods that Apache will use.

sudo a2enmod rewrite ssl actions include dav_fs dav auth_digest headers expires 
sudo a2dismod autoindex 

Apply these changes:

sudo service apache2 restart 

Install PHP Extensions (from PECL) (optional)

If you need to add any PECL extensions there are a few other requirements before installation can take place:

sudo aptitude install php5-dev make 

Once they are available to the system any PECL extensions can be added as follows (I’m using Mailparse and Sphinx as my example, you can use any you like!):

sudo pecl install mailparse, sphinx 

To enable them a new configuration file is required for each.

sudo ##editor## /etc/php5/conf.d/mailparse.ini 

; configuration for php Mailparse module
extension=mailparse.so

sudo aptitude install sphinxsearch 
sudo ##editor## /etc/php5/conf.d/sphinx.ini 

; configuration for php Sphinx module
extension=sphinx.so

And then restart Apache to apply the changes:

sudo service apache2 restart 

Update the locate database

The locate command can be extremely useful. (usage example: locate master.cf)

sudo updatedb

Install the EC2 API command line tools (cloud only)

In order to control the ec2 instances / volumes from the server itself (e.g. for backup crons) we need to add a few tools. Also if you are going to add any EBS volumes to the main EBS backed instance the tools for creation and management of the XFS filesystem are required.

sudo aptitude install sun-java6-jre ec2-api-tools xfslibs-dev xfsdump xfsprogs dmapi 

To use these tools the keys that are required must be available to the system. Create a directory to store these X.509 keys. As management will likely be through the ubuntu (default) user we’ll store the keys in their directory.

mkdir /home/ubuntu/ec2-keys 

##editor## /home/ubuntu/ec2-keys/##ec2_cert_filename## 

Add your certificate data (copy / paste) to this file and save it.

##editor## /home/ubuntu/ec2-keys/##ec2_pk_filename## 

Add your private key data (copy / paste) to this file and save it.

We can also add these keys (and other useful constants) to the global shell configuration files so that they are available to the terminal and login shells (called by crons).

sudo ##editor## /etc/bash.bashrc 

[...]
# Basic convenience
alias ..='cd ..'
alias ...='cd ...'
alias cd..='cd ..'
alias cd-='cd -'
alias ls='ls -lh --color=auto'
alias dir='ls -lh --color=auto'
alias df="df -h"
alias h=history
alias targz="tar cvfz"
alias untargz="tar xvfz"
alias tarbz2="tar cvfj"
alias untarbz2="tar xvfj"
alias unrar="unrar x"
[...]
export EDITOR=##editor##
export EC2_CERT=/home/ubuntu/ec2-keys/##ec2_cert_filename##
export EC2_PRIVATE_KEY=/home/ubuntu/ec2-keys/##ec2_pk_filename##
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
export EC2_PRIVATE_IP="`wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4`"
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`"
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`"
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
export AWS_ACCESS_KEY_ID=##aws_access_key_id##
export AWS_SECRET_ACCESS_KEY=##aws_secret_access_key##

sudo ##editor## /etc/profile 

[...]
# Basic convenience
alias ..='cd ..'
alias ...='cd ...'
alias cd..='cd ..'
alias cd-='cd -'
alias ls='ls -lh --color=auto'
alias dir='ls -lh --color=auto'
alias df="df -h"
alias h=history
alias targz="tar cvfz"
alias untargz="tar xvfz"
alias tarbz2="tar cvfj"
alias untarbz2="tar xvfj"
alias unrar="unrar x"
[...]
export EDITOR=##editor##
export EC2_CERT=/home/ubuntu/ec2-keys/##ec2_cert_filename##
export EC2_PRIVATE_KEY=/home/ubuntu/ec2-keys/##ec2_pk_filename##
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
export EC2_PRIVATE_IP="`wget -q -O - http://169.254.169.254/latest/meta-data/local-ipv4`"
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`"
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`"
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
export AWS_ACCESS_KEY_ID=##aws_access_key_id##
export AWS_SECRET_ACCESS_KEY=##aws_secret_access_key##

Apply these changes.

rm /home/ubuntu/ec2-keys/*.pem~ 
source /etc/bash.bashrc 
source /etc/profile 
source ~/.bashrc 

Update the permissions on these files:

sudo chmod 0600 /home/ubuntu/ec2-keys/* 

At this point it is likely a good idea to disable your ability to (accidentally) terminate your server!

ec2-modify-instance-attribute $EC2_INSTANCE_ID --disable-api-termination true --region $EC2_REGION 

It should tell you that this has been actioned. If you wish to terminate your instance in the future you need to update this setting to false.

Install Subversion (optional)

I like to use Subversion as my version control system because there’s just me, but you may prefer Git, Mercurial, Bazaar or many of the other alternatives. This is not required for the system to run!

sudo aptitude install subversion libapache2-svn 

sudo service apache2 restart 

Add a Remote Desktop system (optional / large)

This isn’t something that I would bother with, but if it is absolutely required then FreeNX is excellent.

export DEBIAN_FRONTEND=noninteractive 
sudo aptitude install ubuntu-desktop 

sudo add-apt-repository ppa:freenx-team 
sudo aptitude update 
sudo aptitude install freenx 

cd /usr/lib/nx 
sudo wget http://launchpadlibrarian.net/47959420/nxsetup.tar.gz 
sudo tar xzvf nxsetup.tar.gz 
sudo ./nxsetup --install 

Download and launch the NX client and connect to the server via SSH using a user with a password login.

Register with Amazon AWS

I’m not going to go into detail for the actual registration of services, but you need a credit / debit card, a valid e-mail address and a phone(!). It takes a while to get all of it done / activated, and there are several services you can use, but for now only EC2 needs configuring.

There are several pieces of information that are required to manage your instances and they are configured / obtained in the Security Credentials section of your Amazon account. You will need to create and write down / save:

• Your Access Key ID (under Access Keys e.g. ##aws_access_key_id##)
• Your Secret Access Key (under Access Keys e.g. ##aws_secret_access_key##)
• Your X.509 Certificate (under X.509 Certificates e.g. ##ec2_cert_filename##)
• Your Private Key (under X.509 Certificates e.g. ##ec2_pk_filename##)

Install Firefox / Elasticfox

If you don’t use Firefox then please download the latest version. Because Amazon haven’t maintained Elasticfox for quite some time I no longer use the official one, instead I recommend you install this enhancement of Elasticfox which is a simple and bug free tool to manage all things AWS. To add it to Firefox:

• Open Firefox
• Select File -> Open File (Ctrl + O or Cmd + O)
• Browse to the extension and select Open
• Choose Install Now

Configure Elastic Fox

Launch Elasticfox by opening Firefox and selecting Tools -> Elasticfox – N.B. it is not under the Ad-ons section, it has it’s own button. You should be prompted to enter your AWS credentials. You can enter new details or modify existing ones by clicking on the “Credentials” button in the top centre of the Elasticfox window:

• Enter your Account Name, this can be anything you want
• Enter your Access Key (##aws_access_key_id##) and Secret Key (##aws_secret_access_key##)
• Press Add
• Press Close

You now need to create a Key Pair:

• Click on the “KeyPairs” tab in Elasticfox
• Click on the green Key icon at the top of the tab
• Give your Key Pair a name (##ec2_keypair_name##)
• Press Ok
• You will be prompted for a location to save the .pem file
• Press Save

Download PuTTY / PuTTYgen (Windows only)

To manage your instance you will need and SSH tool and for Windows systems that basically means PuTTY, so please grab a copy, it’s free (and legal in most countries – please check yours!).

The Key Pair file that you created earlier (##ec2_keypair_name####ec2_keypair_ext##) is sadly not compatible with PuTTY, but it does have a tool to convert it into a usable format called PuTTYgen. Download this too, run it, load in your key file (you might need to select “All Files” under the “Files of Type” dropdown) and press Generate. Save this new key (##ec2_keypair_name####ec2_keypair_ext_ms##) with your original (##ec2_keypair_name####ec2_keypair_ext##) – don’t delete either!

Configure Elasticfox SSH

Launch Elasticfox and visit the Tools panel in the top right of the Elasticfox window.

Windows

Update the following:

• SSH Command: ##ec2_putty_path##putty.exe
• SSH Key Template: ##ec2_key_path_ms####ec2_keypair_name####ec2_keypair_ext_ms##
• EC2 Private Key Template: ##ec2_key_path_ms####ec2_keypair_name####ec2_keypair_ext_ms##
• SSH User: ubuntu

Linux / OSX

Update the following:

• SSH Key Template: ##ec2_key_path####ec2_keypair_name####ec2_keypair_ext##
• EC2 Private Key Template: ##ec2_key_path####ec2_keypair_name####ec2_keypair_ext##
• SSH User: ubuntu

Choose a Region and Availability Zone

These should be a reasonably easy choice, but still one that needs making. There are at the time of writing 5 main regions each with several availability zones:

• US East (Northern Virginia): us-east-1, with zones: us-east-1a, us-east-1b, us-east-1c, us-east-1d
• US West (Northern California): us-west-1, with zones: us-west-1a, us-west-1b, us-west-1c
• EU (Ireland): eu-west-1, with zones: eu-west-1a, eu-west-1b, eu-west-1c
• Asia Pacific (Singapore): ap-southeast-1, with zones: ap-southeast-1a, ap-southeast-1b
• Asia Pacific (Japan): ap-northeast-1, with zones: ap-northeast-1a, ap-northeast-1b

Now, if you were using this in an enterprise environment you would probably want multiple servers in at least two of these regions so that if one malfunctioned the other could take over sole operation. However, given we are interested in a single server set up for now, it’s best just to choose a region / zone nearest to you / your audience and stick with it. In Elasticfox just choose your option from the top left drop down menu.

Define Security Group(s)

A Security Group is a set of rules applied to an instance defining what ports are available to who. In general this boils down to 5 options per port, it is either: closed, open to the internal Amazon network, open to an IP of your choice, open to everyone or open to a combination of the last 3 options. It is easiest to understand with a couple of examples:

1. Apache (the web-server) allows people to view web pages on port 80 (http://) or port 443 (https://) by default. It is likely that you want both of these options available to everybody, so you would set unrestricted access (0.0.0.0/0) to these two ports.
2. SSH (the secure shell) allows you to connect to your server to manage it directly on port 22 by default. It is likely that you want this option only available to you on your IP only, so you would restrict access to only your IP address (##current_ip##/32) on this port
3. If you had a separate MySQL database server it would accept requests on port 3306 by default. You would need to let your web-server connect to it, but in this instance it would be wise to restrict access to only internal cloud machines (10.0.0.0/8). You could also allow your IP (##current_ip##/32) direct access or of course allow everyone (0.0.0.0/0) if you had a non-cloud based server requiring access.

You only need one security group, but as mentioned previously there may come a time when you would like to separate your server into different parts. So I created 5 security groups which would allow a combo server set up (add all 5 groups) or individualised servers (with only the necessary groups added). My groups were:

Web-server

Database Server

E-mail Server

SSH / SFTP / RDP

FTP (optional)

You can then load these in any combination you like to create servers fit for most purposes.

Assign and Whitelist an Elastic IP

Your machine will need an ip address (##ip##) so that you can configure your zone files and point your domain to it. This is very easy to do in Elasticfox:

• Visit the “Elastic IPs” tab
• Click the green plus button to allocate a new IP address
• When the IP address appears right click it and choose “Copy Elastic IP to Clipboard”

Until relatively recently Amazon had real trouble with ip blacklisting (especially bad for e-mails), but they now have a process in place which whitelists a new IP for you within a few days. For this to happen you need to fill out a use case request form to remove sending restrictions. It only takes a minute or two to inform them of your intentions and get your IP whitelisted.

Choose an Instance Size and AMI file

Before you can launch an instance you must first decide which type you need, there are many available and all have different costs associated with them so it’s best not to be greedy! One key matter to consider is that some are 32 bit and some are 64 bit. This will dictate which AMI you choose to run.

Once you have made your choice you can visit Ubuntu’s AMI list and choose the right one for you. Choose the correct (either 64 bit or 32 bit) EBS backed version for your chosen region and copy the AMI reference down.

Launch an AMI

Launching an AMI file is quite straightforward. In Elasticfox:

• Select the “Images” tab
• Type (or paste) your chosen Ubuntu AMI reference into the search box
• Right click the one remaining file and choose “Launch Instance(s) of this AMI”
• A popup box will appear
• In the KeyPair field, select the KeyPair you created earlier (##ec2_keypair_name##)
• In the Security Groups section, select the appropriate group and click on the right arrow to move it into the Launch in box. Do this from bottom to top because Elasticfox has a bug here where groups appear as undefined!
• Press Launch

Associate your Elastic IP

• Select the “Instances” tab (your new instance should appear there)
• Wait for the status to update to “running”
• Right click the instance row
• Select “Associate Elastic IP with Instance”
• Select your IP from the dropdown
• Press “Associate”
• Wait for 30 seconds or so

Connect to your Instance

• Right click the instance row again
• Select “Connect to Instance”
• You may need to point to your key file (Windows: ##ec2_key_path_ms####ec2_keypair_name####ec2_keypair_ext_ms##, Linux: ##ec2_key_path####ec2_keypair_name####ec2_keypair_ext##)

You should now see a terminal window open and be logged in to your server!

Until relatively recently (Dec 2009) the instances were not persistent, that is any data you required on a server had to be (re-)created (by scripts) each time a machine was created / rebooted or crashed. This was, and still is, fine for compute intensive batch jobs such as photo processing or number crunching where you create a machine instance, pass it some data and it will give you back the output. That was (essentially) the end of the server’s life. Obviously for a website this kind of arrangement is far from ideal. Thankfully Amazon addressed this issue and created a product called Elastic Block Store (EBS) for this task. A year on and many gremlins have been ironed out and this is now suitable for production website deployment.

Aims of this guide

This guide is not solely dedicated to creating an EBS backed Amazon EC2 instance, that would take a few seconds and wouldn’t really help many people. It is instead designed to guide a relative novice through all the steps required to configure a production Ubuntu 10.04 Lucid Lynx (LTS) server running in the cloud environment. Much of this guide will work on other Linux flavours with certain changes, but I have only tested it on Ubuntu 10.04. Debian based systems will require relatively minor tweaks while RPM / Gentoo / Slackware based distros would require significant code updates (although conceptually it is similar). Likewise it is not required that your server is running in the cloud, but there is some cloud specific content in the guide.

Server Configuration and Scalability

With a small website it is likely that you will only need one server and the size of that will be dictated by your site traffic / server load. As a site grows and more demand is put on the server there eventually becomes a point where you have to make a change. There are a couple of options here:

• Increase the power of the single server
• Separate the server parts i.e. have multiple servers, most commonly mail-server(s), web-server(s), database server(s)

Even if you have a site with low traffic it’s always prudent to plan for the best and have an idea of what to do when the day comes where your server can no longer cope. Amazon EC2 allows an upgrade to a larger instance very quickly, but ultimately you run out of larger instances to use when you hit their Extra Large Instance and become stuck with migration difficulties.

Again Amazon EC2 offers a unique solution to this issue, their EBS storage. It doesn’t just have to be for machine images, EBS volumes can be used essentially as detachable hard disks. This means that you can separate all core (config and data) files for the web-server (e.g. Apache / PHP, site data, vhosts etc), mail-server (e.g. Postfix / Dovecot, e-mails etc) and a database server (e.g. config and data) onto separate “drives” and if the situation ever arises where the server cannot cope you simply detach the required EBS volume from one instance and attach it to another making minimal configuration changes. You can of course copy EBS volumes quickly too and Amazon offers a load balancer if multiple web-servers are used.

Obviously I don’t want to force this choice on anyone (because it does cost a few dollars a month extra) so I have set this up in such a way as it is completely optional to to this. Just follow the relevant section of the guide to your needs.

Requirements

• You use either Ubuntu 10.04 Lucid Lynx (LTS) (non-cloud) or an EBS backed Ubuntu 10.04 Lucid Lynx (LTS) AMI
• You have a basic knowledge of Linux commands (i.e. you can use a terminal and an editor of your choice)
• You are aware of file permissions in Linux
• You have a domain (##domain##) to use
• You understand the difference between a domain (##domain##) and a sub-domain (e.g. www)
• You have your own nameservers (most registrars provide them)
• You understand what DNS is and how to configure your zone file to point to your new server for testing (most registrars offer a GUI tool for this)

If I want to exclude an individual letter or number it’s an easy task, let’s say I wanted to match everything from the start of a string up until either a ‘z’ or a ’7′:

Produces:

Considering: The quick brown fox jumped over the lazy dog
Found: The quick brown fox jumped over the la

Considering: 0 seven 3 four 9 six 7 four 2
Found: 0 seven 3 four 9 six

Now what if I wanted to do the same for the word ‘lazy’ or ‘six’ in the same phrases? It’s not hard either, but it’s not obvious:

Produces:

Considering: The quick brown fox jumped over the lazy dog
Found: The quick brown fox jumped over the

Considering: 0 seven 3 four 9 six 7 four 2
Found: 0 seven 3 four 9

This uses something called a negative lookahead (?!) preceeding the dot (any character) which I’ve told not to be included in a match (?:). Simples.

Hope this saves someone the time it took me to read up on!

Without further ado here is the new PHP Spinner code.

And an example:

Which produces:

The speedy black wolf bounded over the lazy hound
A speedy brown fox skipped over the tired hound
The quick red wolf bounded over the lazy hound
The fast brown fox hopped over the tired dog
The speedy brown fox jumped over the tired dog

and it will work happily as a nested PHP spinner too:

Which produces:

A basic example to illustrate the function
My test case to illustrate the problem
The rudimentary case to illustrate the function
An uncomplicated scenario to illustrate the problem
The fundamental case to illustrate the problem

Improvement

The spinner now allows seed page options to be strings such as ‘daily’, ‘every 2.5 hours’ or ‘every 3 weeks’ rather than just true / false so the spun text will automatically update every set period of time. See the checkSeed (bottom) function for allowed values.

Have fun spinning articles! Any comments, suggestions or improvements are welcome.