Updated version of this tutorial available here : http://blogama.org/node/92

This tutorial provides step-by-step instructions on how to install XEN on an Ubuntu Hardy Heron server (8.04) without compromising on disk I/O and network throughput by using partitions instead of image files. You can find all the software used here in the Ubuntu repositories, so no external files or source compilation are required.

Xen is an open-source para-virtualizing virtual machine monitor (VMM), or 'hypervisor', for the x86 processor architecture. Xen can securely execute multiple virtual machines on a single physical system with close-to-native performance. Xen facilitates enterprise-grade functionality, including:

* Virtual machines with performance close to native hardware.
* Live migration of running virtual machines between physical hosts.
* Up to 32 virtual CPUs per guest virtual machine, with VCPU hotplug.
* x86/32, x86/32 with PAE, and x86/64 platform support.
* Intel Virtualization Technology (VT-x) for unmodified guest operating systems (including Microsoft Windows).
* AMD Virtualization Technology (SVM aka Pacifica) on AM2 and F stepping Opterons (2006H2)
* Excellent hardware support (supports almost all Linux device drivers).

1. Partition And Primary OS Installation

In this example, I will use Ubuntu Hardy Heron (x86_64) for both the host OS (dom0) and the guest operating systems (domU). Prior to this you should have a target system ready with partitions like in the following example:

For example we have 120GB disk space and we want to use 2 guest operating systems on it:

* Take as minimal as required for host OS (dom0), say 10-12GB => so create the first partition accordingly.
* We will divide the rest for the guest OS's (domU's). Each guest OS needs 2 partitions, one ext3 and one swap so create partitions accordingly.

Install the Ubuntu Hardy Heron Server Edition (http://www.ubuntu.com/products/whatisubuntu/serveredition) on the first partition only. Afterwards, on this we will be install the Xen stuff and use this as the host OS (dom0). As per better practice, dom0 should not be used for any production use except for controlling guest OS's. So only choose "ssh" in package selection during OS installation.

After the OS installation the first job is to update/upgrade the OS. Login to the system as root and type following commands:

apt-get update
apt-get upgrade
apt-get install vim-full
/etc/init.d/apparmor stop
update-rc.d -f apparmor remove

2. Host OS And Xen Installation

Install the Xen kernel by typing the following command :

apt-get install ubuntu-xen-server

This will also install the xen-tools package which we use later on to create virtual machines.

The ubuntu-xen-server package installs the Ubuntu kernel 2.6.24-16-xen; unfortunately there's a nasty bug in this kernel which causes networking to not work for virtual machines (see https://bugs.launchpad.net/ubuntu/+source/xen-3.2/+bug/204010 and https://bugs.launchpad.net/ubuntu/+source/linux/+bug/218126). Fortunately Hirano Takahito has patched this kernel and made the patched versions (for i386 and amd64) available on http://www.il.is.s.u-tokyo.ac.jp/~hiranotaka/ (I have created a mirror of these packages at http://downloads.howtoforge.com/ubuntu_8.04_server_xen_from_repositories/ in case the original link doesn't work anymore one day). Download the correct linux-image package for your architecture (i386 or amd64), e.g.:

wget http://www.il.is.s.u-tokyo.ac.jp/~hiranotaka/linux-image-2.6.24-19-xen_2.6.24-19.33~zng1_amd64.deb

Then replace the Ubuntu 2.6.24-16-xen kernel with it like this:

dpkg -i linux-image-2.6.24-16-xen_2.6.24-16.30zng1_i386.deb 

We also need to add the loop module to the kernel every time we boot our system, so we edit /etc/modules. If you already have a loop line in there, make it look like this, otherwise add it :

[...]
loop max_loop=64
[...]

I want to store my virtual machines in the /home/xen directory, therefore I create it now:

mkdir /home/xen

We will use xen-tools to create virtual machines. xen-tools make it very easy to create virtual machines - please read this tutorial to learn more: http://www.howtoforge.com/xen_tools_xen_shell_argo. As mentioned before, the xen-tools package got installed together with the ubuntu-xen-server package.

Now we edit /etc/xen-tools/xen-tools.conf. This file contains the default values that are used by the xen-create-image script unless you specify other values on the command line. I changed the following values and left the rest untouched:

vi /etc/xen-tools/xen-tools.conf

[...]
dir = /home/xen
[...]
dist   = hardy     # Default distribution to install.
[...]
gateway   = 192.168.0.1
netmask   = 255.255.255.0
broadcast = 192.168.0.255
[...]
passwd = 1
[...]
mirror = http://archive.ubuntu.com/ubuntu/
[...]

Then reboot

If you run uname -r you should see Ubuntu using the Xen kernel

3. Creating Virtual Machines (domU)

Run this command

xen-create-image --hostname=xen1.example.com --size=2Gb --swap=256Mb --ide --ip=192.168.2.123 --netmask=255.255.255.0 --gateway=192.168.2.1 --force --dir=/home/xen --memory=256Mb --arch=amd64 --kernel=/boot/vmlinuz-2.6.24-19-xen --initrd=/boot/initrd.img-2.6.24-19-xen --install-method=debootstrap --dist=hardy --mirror=http://archive.ubuntu.com/ubuntu/ --passwd

Modify the options hostname, ip, netmask, gateway, memory to your need. Its important that you get the good files for the options kernel & initrd by looking in your /boot directory. For the moment, the size and swap options are not important, leave them like that.

We are not done yet but you can test the virtual machine with the .img file by running the following command :

xm create /etc/xen/xen1.example.com.cfg
xm console xen1.example.com

to return to Dom0, type CTRL + ]

Don't forget to shut it down :

xm shutdown xen1.example.com

4. Running Virtual Machine with dedicated partition

Assuming you have 2 extra partition : /dev/sda3 and /dev/sda4 do the following :

mkdir /mnt/img
mkdir /mnt/part
mount -o loop /home/xen/domains/xen1.example.com/disk.img /mnt/
mkfs.ext3 /dev/sda3
mkswap /dev/sda4
mount /dev/sda3 /mnt/part
cd /mnt/img
cp -av * /mnt/part

Before we umount the partitions, we need to make a few modifications to fstab :

vi /mnt/part/etc/fstab

Your fstab should look like :

#/etc/fstab: static file system information.
 
<file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/hda1 none swap sw 0 0
/dev/hda2 / ext3 noatime,nodiratime,errors=remount-ro 0 1

Once this is done you can umount :

umount /mnt/img
umount /mnt/part

We are almost done, we need to make a few modifications to /etc/xen/xen1.example.com.cfg. Find the "disk" information and replace with the following :

[...]
#
#  Disk device(s).
#
root        = '/dev/hda2 ro'
disk        = [
                  'phy:/dev/md3,hda1,w',
                  'phy:/dev/md2,hda2,w',
              ]
 
 
#
#  Hostname
#
[...]

You can also change the quantity of vcpus used by adding a line such as :

[...]
vcpus       = '4'
[...]

5. Starting the virtual machine

To start the virtual machine type (dont forget to shut it down if you start it with the .img file with the command "xm shutdown xen1.example.com"):

# xm create /etc/xen/xen1.example.com

Here are the most important Xen commands:

xm create -c /path/to/config - Start a virtual machine.
xm shutdown - Stop a virtual machine.
xm destroy - Stop a virtual machine immediately without shutting it down. It's as if you switch off the power button.
xm list - List all running systems.
xm console - Log in on a virtual machine.
xm help - List of all commands.

5. Little bug fix...

vi /etc/init.d/xendomains

Then find and replace the line :

rest=`echo "$1" | cut cut -d\  -f2-

with

rest=`echo "$1" | cut -d\  -f2-