Intention/AudienceIn this howto I'd like to walk you through the creation of a Xen FreeBSD guest (DomU) with full para-virtualisation (PV). I will assume that you know how to configure Xen DomUs manually, and that you have got at least some basic FreeBSD experience, because you will need to compile a custom kernel later. Furthermore I'm assuming that you don't have any FreeBSD guests running at this point (or don't want to convert any existing ones from HVM to PV). I will show how to
In this short tutorial I chose the HVM-to-PV transition to point out the differences between HVM and PV Xen configurations as well. That may be useful for those of you, who intend to run amd64 HVM guests. Pre-Requisites and Limitations
You will need:
Limitations compared to other guest OS:
Spoiler Alert: I've added a Download section at the end of this tutorial, which contains a fully functional FreeBSD PV guest image with pygrub partition and Xen configuration file. Creating the HVM guestI'll start showing how to create a HVM guest from scratch, with no other FreeBSD instance around. To begin, create a Xen configuration file: /etc/xen/freebsd # HVM loader kernel = "/usr/lib/xen/boot/hvmloader" builder='hvm' device_model = '/usr/lib/xen/bin/qemu-dm' # memory can't be much higher than this; known limitation memory = 850 name = "freebsd" vif = [ 'bridge=xenbr0'] disk = [ 'phy:/dev/vg0/freebsd,hdb,w', 'file:/root/FreeBSD-8.2-RELEASE-i386-disc1.iso,hdc:cdrom,r' ] # make sure we boot from cdrom here boot="d" vcpus = 1 vnc=1 vnclisten="0.0.0.0" vncpasswd="passwd" vncdisplay=5 You may have noticed, that there's no device hda in this configuration. I'm doing this on purpose, because that's where we will mount a pygrub-capabable partition later. The benefit of using hdb here is that we don't need to change anything in the DomU later. Next, you want to create an LVM volume (or empty file) for your DomU. I'll go for 5GB here, although we won't fill all that space. lvcreate -n freebsd -L5G vg0 Then we're ready to start: xm create freebsd Your VNC console can now be connected to port 5905, password is "passwd" obviously. Change your disk configuration: /etc/xen/freebsd
[...]
#remove CDROM here:
disk = [
'phy:/dev/vg0/freebsd,hdb,w'
]
# make sure we do not boot from cdrom any more
# boot="d"
[...]
Now destroy and restart the VM: xm destroy freebsd xm create freebsd The VM will boot up, and at the login prompt you'll not be prompted for a password, as we haven't set one. Do that very soon. At this stage you have a fully working HVM guest, of course lacking stuff like SSH etc, but you could take it from here if you wanted. Optional: Serial Console for HVM guestThis step is absolutely irrelevant if you want to create a PV guest. However for HVM images it might be helpful to note that you can use xm console as well, if you configure the domU correctly. Anyways, it's very easy to get that sorted. In /etc/xen/freebsd add this line: serial='pty' In the FreeBSD guest, the serial console setup is absolutely identical to a normal non-Xen console. The important bits, taken from The FreeBSD Handbook /etc/ttys # change this: ttyu0 "/usr/libexec/getty std.9600" dialup off secure # to this: ttyu0 "/usr/libexec/getty std.9600" vt100 on secure And finally, also in the guest, run this:
echo 'console="comconsole"' >> /boot/loader.conf
That's all. Now your serial console will be activated very early in the boot process and you can watch the (very fast) startup progress.
Transforming the guest from HVM to PVHowever, this tutorial is about PV guests, so let's continue. First you need to get the kernel sources. FreeBSD uses CVSup for that, or csup (minus the 'v') more precisely. We haven't got networking yet, so here we go: ifconfig re0 <IP> up route add default <gateway IP> echo "nameserver <nameserverIP>" > /etc/resolv.conf ping sysconfig.org.uk Now let's get the sources... csup -L2 -h cvsup2.de.freebsd.org /usr/share/examples/cvsup/stable-supfile This will take a while. Depending on where you are, you might want to change the hostname to something closer. After it's completed downloading, we need to build a xen kernel:
cd /usr/src
echo "options EXT2FS" >> sys/i386/conf/XEN
mkdir /root/myboot
make KERNCONF=XEN DESTDIR=/root/myboot kernel
I'm adding ext2 support here for later, so that we can create a partition for pygrub, which we can then add in as hda. Also that will enable us to write new kernels directly into an area, where the DomU can access it. It will become clearer in the next step, why this is convenient. Now that we're done, you need to copy the kernel into the Dom0, for example via SCP: scp /root/myboot/boot/kernel/kernel <domOIP>:/root/kernel-freebsd-pv We'll need that again later. Now let's give some finishing touches. For example you could configure your rc.conf properly to set network details, enable SSH etc, so something like this at the very least (replace appropriate values): /etc/rc.conf hostname="domU" defaultrouter="10.0.0.1" sshd_enable="YES" ifconfig_re0="inet 10.0.0.111/24 up" ifconfig_xn0="inet 10.0.0.111/24 up" Huh? Same IP twice for re0 and xn0? Indeed! re0 is the interface in HVM mode. In PV mode it will be called xn0 (xen network). Due to that fact, only one of the two can be present at any time, and therefore the IPs won't clash. Even better: Should you want to use the VM in HVM again later, for whatever reason, you will have connectivity as well. Now we're almost there. As xen uses a consoled called xc0 (in PV mode only), this needs to be added to the configuration, and all ttyv* consoles need to be disabled to avoid nasty warnings: /etc/ttys xc0 "/usr/libexec/getty Pc" cons25 on secure #ttyv0 "/usr/libexec/getty Pc" cons25 on secure # Virtual terminals #ttyv1 "/usr/libexec/getty Pc" cons25 on secure #ttyv2 "/usr/libexec/getty Pc" cons25 on secure #ttyv3 "/usr/libexec/getty Pc" cons25 on secure #ttyv4 "/usr/libexec/getty Pc" cons25 on secure #ttyv5 "/usr/libexec/getty Pc" cons25 on secure #ttyv6 "/usr/libexec/getty Pc" cons25 on secure #ttyv7 "/usr/libexec/getty Pc" cons25 on secure #ttyv8 "/usr/local/bin/xdm -nodaemon" xterm off secure That's it. You'd probably expect changes to /etc/fstab or other tweaks. Well, there aren't any at this stage. Final touches for the PV configurationLet's change the guest configuration file first: /etc/xen/freebsd kernel="/root/kernel-freebsd-pv" extra = " vfs.root.mountfrom=ufs:ad1s1a,machdep.idle_mwait=0,boot_verbose=1,kern.hz=100" # memory can't be much higher than this; known limitation memory = 850 name = "freebsd" vif = [ 'bridge=xenbr0'] disk = [ 'phy:/dev/vg0/freebsd,hdb,w' ] vcpus = 1 You notice that the HVM-related instructions and VNC have gone. Instead we've added the "evacuated" kernel and its parameters. We'll refine that a bit more later. First let's make sure that our setup works. xm create freebsd -c Note: the -c tells xm to open a console, which is only possible in PV mode, and sadly the only option, because FreeBSD can't use both VNC and console at the same time for some reason. If you get to the login and can actually log in, you can shut down the guest again. We'll do some more cosmetics. Using pygrubUsing pygrub has the advantage that the kernel can reside within the DomU. This step is entirely optional. First, create another LVM volume (or file, if you choose to), and format it with ext2, prepare directories, and throw your kernel in:
lvcreate -L100M -n freebsd-boot vg0
mkfs -t ext2 /dev/vg0/freebsd-boot
mount /dev/vg0/freebsd-boot /mnt
mkdir -p /mnt/boot/{grub,kernel}
cp /root/kernel-freebsd-pv /mnt/boot/kernel
And now, the last step, adding a grub menu file. That's rather unusual for FreeBSD, but pygrub requires it. So here it goes: /mnt/boot/grub/menu.lst
timeout=2
default=0
title FreeBSD
root (hd0,0)
kernel /boot/kernel/kernel-freebsd-pv vfs.root.mountfrom=ufs:ad1s1a,kern.hz=100,boot_verbose=1,machdep.idle_mwait=0
You can omit timeout=2 at this point, but I tend to keep it in there, just to have the option to load other kernels, if I have to. Unmount /mnt and change your guest configuration a last time: /etc/xen/freebsd bootloader="/usr/bin/pygrub" # memory can't be much higher than this; known limitation memory = 850 name = "freebsd" vif = [ 'bridge=xenbr0'] disk = [ 'phy:/dev/vg0/freebsd-boot,hda,w', 'phy:/dev/vg0/freebsd,hdb,w' ] vcpus = 1 You see, the kernel references are gone, and pygrub appeared instead. Also, the new boot volume has been inserted as hda. Now fire it up again. You'll notice the boot menu flashing up (unless you removed the timeout), and apart from that it will boot as it did before. xm create freebsd -c Well done! In the guest you can now use /dev/ad0, which is the boot volume, and update your kernel or grub configuration as you please. mount -t ext2fs /dev/ad0 /mnt NetworkingIf you set up your rc.conf for networking, and chances are that you want to have nice network performance, then you should use this rather undocumented sysctl setting, which unleashes full networking power. Otherwise throughput seems highly limited between PV guests on the same physical box. Strangely it does not affect traffic between different boxes. It's not really documented anywhere, but I stumbled upon it a while ago in the freebsd-xen mailing list: /etc/sysctl.conf [...] net.inet.tcp.tso=0 [...] Why this changes networking performance so drastically, is beyond my knowledge, but at least for me it works. (YMMY) Create a PV guest - Fast TrackIf you have created your first guest like outlined in this tutorial, or are a bit more advanced FreeBSD user, you may want to make your life easier when you create your next guests. Let me very briefly show you how you would go about creating a guest from within a guest. You should have:
Create and Mount Target Image (File)# truncate -s 256M freebsd.img # mdconfig -f freebsd.img # fdisk -BI md0 # bsdlabel -wB md0s1 # newfs -U md0s1a # mount /dev/md0s1a /mnt Create and Mount Target Image (hot-plugged block device: LVM or file from Dom0)As an alternative you can use xm block-attach in Dom0 to use a file or LVM partition as a disk in your DomU. (In this example called ad4) # fdisk -BI /dev/ad4 # bsdlabel -wB /dev/ad4s1 # newfs -U /dev/ad4s1a # mount /dev/ad4s1a Populate the empty ImageIt's extremely easy to install an entire FreeBSD. But it takes time to finish, so get a coffee and cookies and then run this (or the other way round # cd /usr/src; make buildworld && make buildkernel KERNCONF=XEN # setenv DESTDIR /mnt && make installworld && make installkernel KERNCONF=XEN && cd etc && make distribution Then edit /mnt/etc/fstab: /dev/ad1s1a / ufs rw 1 1 Some tutorials suggested to use /dev/xbd0. I found this not to apply any more. Xen 4.1.1 and FreeBSD 8.2 map xbd0 correctly to common block device names. Next, change /mnt/etc/ttys. Add the following line and comment all ttyv* lines:
xc0 "/usr/libexec/getty Pc" cons25 on secure
That's it already. unmount /mnt and you're ready to use your image. The Xen configuration file can easily be derived from the previous sections. Keep in mind that I'm always using ad1 or hdb, in other words the second block device, for the root partition. This allows adding in a pygrub-capable first block device as described earlier, or to temporarily insert a completely different guest, rescue image or whatever without having to tweak your fstab over and over again. At the end of the day it's a matter of taste, but don't complain if it doesn't work for you, because you insisted on doing it your way DownloadIf you'd like to have a copy of the PV DomU, which I created while writing this tutorial, you can find it here: FreeBSD 8.2 32bit PV DomU for Xen 4.1 w/ pygrub - 186.5MB - Aug 10th, 2011 The tar.gz expands to:
References
|
Shortcuts
|
Skip to end of metadata
Go to start of metadata
)
)