LXC Linux Containers, Ubuntu & udev

I recently started using linux containers instead of xen virtualization. It's not a fully mature setup yet, but I prefer the approach for what my needs are. Plus with the evolving cgroups feature in the kernel it's shaping up to be an efficient way to have multiple independent environments without the overhead of virtualization. For example, IIRC there are fewer context switches required when using LXC to access the network.

I have a base host of Debian Squeeze (currently in testing as of this writing). I have Debian Lenny, Ubuntu Lucid, and Gentoo as guest systems. The Debian squeze installer works well for Lenny and Lucid, but the Ubuntu folks haven't taken the necessary steps to make Ubuntu play nice in a container.

One main glitch I found with Ubuntu Lucid was that during the regular system upgrades I received a new udev package, which started causing problems with dpkg. Essentially we don't want to have udev in the guest since the host deals with the /dev/ filesystem. If your container is set up with a default deny on the dev fs, then you'll have seen the below errors:

Setting up udev (151-12.2) ...
mknod: `/lib/udev/devices/ppp': Operation not permitted
dpkg: error processing udev (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of plymouth:
 plymouth depends on udev (>= 149-2); however:
  Package udev is not configured yet.
dpkg: error processing plymouth (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 udev
 plymouth
E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install.  Trying to recover:
Setting up udev (151-12.2) ...
mknod: `/lib/udev/devices/ppp': Operation not permitted
dpkg: error processing udev (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of plymouth:
 plymouth depends on udev (>= 149-2); however:
  Package udev is not configured yet.
dpkg: error processing plymouth (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 udev
 plymouth

In this error message we see that the udev.postinst script is trying to make a node in /dev/, which we don't want it to do.

There is probably a more graceful way to fix this, but for now I'm quite happy to hack it outta my way by editing /var/lib/dpkg/info/udev.postinst and putting an exit 0 before anything else is done in the script. Once that's done just reconfigure it and it should work:

# dpkg --configure udev
# dpkg --configure plymouth

Gentoo GCC-3.3.6 and mplayer or xine

Got a new VIA EPIA EN12000E mobo over the weekend and plan to set it up as a set top box and file server, among other things. It's a pretty nifty little gadget and best of all has no fan.

I'm building a new gentoo instance on it. So far I've built the core OS on a mirror RAID pair of drives and plan to move 4 SATA drives to it when all the services are completely built and it works with my 1080i HDTV. For now I'll probably just stick to using the VGA interface, hoping that'll be easiest.

I've had a few hurdles to get through during the build. For example, I wanted to do a network boot and install, but the gentoo network boot and install instructions weren't clear. So I reverted to an Ubuntu network boot I set up a while ago and went through the rescue interface.

I set up a RAID1 pair for the root partition, and LVM over RAID1 for a few other partitions. Considering my first gentoo installation is still going strong since being built in 2002, I figure I'll make sure this one lasts too.

It took a while to get going. I generally remember how to get started with the gentoo install since I started doing it years ago when we would start from stage1 (if fanatic enough). Unfortunately the Ubuntu 7.04 rescue boot doesn't provide SSHD so I had to get the stage3 and portage installed and chrooted before I could get SSH to the box.

Once that was done I tweaked the CFLAGS and USE flags for the hardware and did the bootstrap.sh to make sure it's fully optimized for the hardware. I then built a custom kernel (2.6.22.6) to enable most of the embeded hardware that I'll be using to start with.

When I booted the new kernel none of the LVM partitions mounted. Duh! I forgot to emerge the lvm2 package. Luckily LVM was compiled in the kernel, so I just copied the static vgscan and vgchange binaries from my other box and started the LVM partitions up.

Next I diff'd the /var/lib/portage/world file to see what I want from my old box and started working through the USE flags. I noticed a dependancy for gcc-3.3.6. The core system already came with gcc-4.1.2 so why should i put an old gcc-3 on it? It turns out that all that is needed is to 'emerge libstdc++-v3'. Note that the package is not simply libstc++.

I'm not sure why libstdc++-v3 isn't included in the 2007.0 stages since they're packaged with gcc-4. I eventually found the answer on an IRC log (http://dbforms.oxxus.net/dependency-question-appemulationwine-world-corefonts/index.php), which references the Gentoo GCC Upgrade Guide.

Next I have ccache and distcc set up to speed up the builds and I must wait a couple of days for everything to install. I hope the configs won't be too complicated and I'm looking forward to seeing the performance of this system.