George MacKerron: code blog

I recently posted a HOWTO based on my experience moving a Xen domU from Linode to my own Xen Dom0 setup at Hetzner.

Since this machine is only a development server, I more recently decided to turn the same machine into a VMware VM, running locally (in VMware Fusion 4 on my MacBook Pro). Here, I document the steps necessary for that transformation.

The domU in question is running Ubuntu 10.04. I believe some of these steps would need to be amended for later Ubuntu versions (and some other Linux flavours), which use Grub 2.

Retrieve the logical volume

First we need to grab the LVM volume that represents the domU’s disk from the Xen machine. I shut down the domU in preparation for this, but if uptime was important I could have taken an LVM snapshot instead.

On the Xen dom0 I temporarily make the volume world-readable (sudo chmod go+r /dev/vg0/my-domu-root), then retrieve it from my laptop with ssh:

ssh george@my-domu.example -C "dd if=/dev/vg0/my-domu-root bs=1M" | dd of=my-domu.img

Convert to VMDK

What we’ve just downloaded is a raw partition, and VMware wants a disk in VMDK format, so we use qemu-img to convert it.

qemu-img convert -p -O vmdk my-domu-root.img my-vm.vmdk

(I initially tried to do this on my Mac, using brew install qemu to get the qemu-img binary onto my Mac, but for no obvious reason the conversion kept freezing part-way. So in the end I did this step on the Xen dom0 instead, installing the tool with sudo aptitude install qemu-utils, and then copied the converted file back over with SSH).

Create new VMware VM

Now in Fusion, I create a new VM. Amongst other options, I add two hard drives: (1) a new drive, sized large enough to accommodate the domU drive data + some swap space, and (2) the converted drive from the domU.

We can’t boot from the converted domU drive for several reasons. Most fundamental of these is that it’s just a raw partition, so our first job is to copy it into a partition on a complete (virtual) drive.

Boot GParted Live disk

I download the GParted Live disk .iso, and set that as the contents of the VM’s DVD drive. Now I boot the VM.

Partition new drive

Using GParted, I create a partition map on the new, empty disk (keeping the default type, ‘msdos’). I create an ext4 primary partition, matching the filesystem of the domU drive and slightly larger than it. I create a swap primary partition to fill the remaining space on the new disk, and apply.

At this point, the new drive is appearing as /dev/sda, with the new ext4 partition as /dev/sda1 and swap as /dev/sda2. The domU drive is appearing as /dev/sdb and is a partition in itself.

Be sure to use the correct device paths for your system! If I were doing this again, I’d probably use UUID-based paths instead.

Copy domU data

Now I open Terminal and copy the data from the domU partition to the blank ext4 partition on the new drive, and resize the filesystem to fill that new partition.

sudo su

dd if=/dev/sdb of=/dev/sda1 bs=10M

e2fsck -f /dev/sda1
resize2fs /dev/sda1

I then set the boot flag on /dev/sda1 in GParted.

Change root

After making key system directories accessible from the domU partition, we chroot to make that our new root.

mount /dev/sda1 /mnt

mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
mount --bind /dev /mnt/dev

chroot /mnt

Get Grub and a kernel

The domU I am moving in this case already has Grub and a kernel installed, as it was being booted using pygrub on the dom0.

However, for a domU that has been booted with an external kernel — a domU copied directly from Linode, for example — one would need to install both of these at this point. I believe the following command would do the job, assuming you have an Internet connection working, but I haven’t tested it.

aptitude install linux-generic-pae linux-headers-generic-pae grub

Fix drive references

There are several places in the domU configuration that currently refer to /dev/xvda and /dev/xvdb as the root and swap partitions respectively, and these now need changing.

We amend the drive specifications in /etc/fstab, /etc/mtab, and /boot/grub/menu.lst.

sed -i -e 's|/dev/xvda|/dev/sda1|g' -e 's|/dev/xvdb|/dev/sda2|g' /etc/fstab
sed -i -e 's|/dev/xvda|/dev/sda1|g' -e 's|/dev/xvdb|/dev/sda2|g' /etc/mtab
sed -i -e 's|/dev/xvda|/dev/sda1|g' /boot/grub/menu.lst

Notes:

• In spite of it being a file that’s usually dynamically created, it’s important to include /etc/mtab, since that’s where Grub will look to discover our root partition.

• menu.lst is confusing and messy. Parts of it are automatically generated while other parts are not, and various lines that are ostensibly commented out are in fact active, including the kernel options (# kopt) line that’s crucial here.

Install and update Grub

Then we install Grub in the boot record, and get it to update menu.lst.

grub-install /dev/sda
update-grub

TTY

At this point your domU ought to be bootable in VMware but would probably hang, waiting on the Xen hypervisor console hvc0 which doesn’t now exist. So:

cp /etc/init/hvc0.conf /etc/init/tty0.conf
sed -i -e 's|hvc0|tty0|g' /etc/init/tty0.conf
mv /etc/init/hvc0.conf /etc/init/hvc0.conf.disabled

Boot

That should be everything necessary to reboot into your domU on VMware.

shutdown -h now

Remove the original domU disk from the VM, and also the GParted live disk .iso from its virtual DVD drive. Then restart it.

Networking

Assuming your new VM boots up OK, you may now need to tweak networking settings in /etc/network/interfaces and /etc/resolv.conf, and to tweak iptables, nginx or Apache conf files, and so on.