LXC Linux Containers, Ubuntu & udev

I recently started using linux containers instead of xen virtualization. It's not a fully mature setup yet, but I prefer the approach for what my needs are. Plus with the evolving cgroups feature in the kernel it's shaping up to be an efficient way to have multiple independent environments without the overhead of virtualization. For example, IIRC there are fewer context switches required when using LXC to access the network.

I have a base host of Debian Squeeze (currently in testing as of this writing). I have Debian Lenny, Ubuntu Lucid, and Gentoo as guest systems. The Debian squeze installer works well for Lenny and Lucid, but the Ubuntu folks haven't taken the necessary steps to make Ubuntu play nice in a container.

One main glitch I found with Ubuntu Lucid was that during the regular system upgrades I received a new udev package, which started causing problems with dpkg. Essentially we don't want to have udev in the guest since the host deals with the /dev/ filesystem. If your container is set up with a default deny on the dev fs, then you'll have seen the below errors:

Setting up udev (151-12.2) ...
mknod: `/lib/udev/devices/ppp': Operation not permitted
dpkg: error processing udev (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of plymouth:
 plymouth depends on udev (>= 149-2); however:
  Package udev is not configured yet.
dpkg: error processing plymouth (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 udev
 plymouth
E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install.  Trying to recover:
Setting up udev (151-12.2) ...
mknod: `/lib/udev/devices/ppp': Operation not permitted
dpkg: error processing udev (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of plymouth:
 plymouth depends on udev (>= 149-2); however:
  Package udev is not configured yet.
dpkg: error processing plymouth (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 udev
 plymouth

In this error message we see that the udev.postinst script is trying to make a node in /dev/, which we don't want it to do.

There is probably a more graceful way to fix this, but for now I'm quite happy to hack it outta my way by editing /var/lib/dpkg/info/udev.postinst and putting an exit 0 before anything else is done in the script. Once that's done just reconfigure it and it should work:

# dpkg --configure udev
# dpkg --configure plymouth

Debian Sarge from VMWare to Xen DomU

Now I have a good Gentoo Xen instance, I want to migrate my VMWare environments to Xen DomU's.

My mobo is the Via Epia EN12000 with a Via C7 Esther CPU, and only has the cmov CPU flag, so no chance of getting HVN working. I had to build the standard Gentoo Xen Linux Sources at version 2.6.20. That's better than the stock Xen Kernel which is currently 2.6.18, but it still doesn't have specific support for the Via C7. Not a problem though.

I have an old Debian Sarge that hasn't been updated in a while. To get it working I had to add the sarge-backports source to /etc/apt/sources.list. Once that was done it was still a bit of a problem getting a Xen kernel installed. In the end I figured out the dependencies to work.

Aptitude didn't give very helpful details so I had to get the info I needed by using "dpkg -i" on packages. Eventually the following commands got me where I wanted to get:

aptitude install klibc-utils
aptitude -V -f install udev=0.105-4~bpo.1
aptitude -V -f install initramfs-tools
aptitude -V -f install linux-image-2.6-xen-686

The key to this seems to have been the packages removed and installed by klibc-utils. After that you could probably skip to installing linux-image-2.6-xen-686.

For reference, the main error from aptitude that was killing me was:

The following packages have unmet dependencies:
 initramfs-tools: Depends: udev (>= 0.086-1) but it is not installable

even though udev-0.105-4~bpo.1 was installed.

Once the kernels were installed I tried to boot the domain in Xen and faced another problem. The linux-image-2.6.18-4-xen-686 in Sarge Backports now supports PAE, so I had to rebuild my dom0 kernel and apps-emulation/xen with PAE support.