CD-Building again

More fun and frustrations while rolling up BlueOnyx ISO images.

It has been know for a while that BlueOnyx (and it's sister project Aventurin{e}) have issues when installed on newer Dell servers: The OS installs fine, but after the first reboot you have no network cards anymore.

This is the case, because the nutters at Dell insist on shipping their hardware with BIOSDEVNAME support. Basically it's an arbitrary renaming of network interfaces. So on Linux you no longer see eth0, eth1, eth3 and so on, but maybe em1, em2, em3 or crazy stuff like p1m1 (or similar). And it's enabled by default and there is no BIOS option to turn it off.

Software wise the only way to get clear of this mess is to add the kernel parameter biosdevname=0 both at install time when you boot the OS installer from the CD. AND to add biosdevname=0 to the kernel parameters in grub.conf.

It gets even worse: If you do so, you also must have the RPM kernel-firmware installed, or the NIC cards on a Dell PowerEdge 710 (for example) will still be "invisible" and unuseable after the first reboot of the freshly installed OS.

So I went off to rebuild the ISO images for BlueOnyx 5106R (on CentOS-5.8), BlueOnyx 5107R (SL-6.3 and CentOS-6.3), BlueOnyx 5108R (SL-6.3 and CentOS-6.3) and Aventurin{e} 6106R. All which use biosdevname=0 in the installer and also set it as default kernel parameter in grub.conf.

So that's six different updated ISO's. All rolled up on different build boxes (a necessity due to the architecture and demands of Revisor) and all with different kinks and oddities by themselves.

And it required testing of each ISO image. A procedure which took three whole days. And which was only possible because a client borrowed me a Dell server for testing this mess. Thanks Colin! Much appreciated!

But it sure doesn't help to run into Anaconda oddities again, which is always the same. The Anaconda on SL-6.3 is still broken an chokes on finding an existing /etc/localtime in the installed OS. Additionally it builds an initrd image with missing kernel modules, so the install fails because even the most basic hardware is no longer supported due to missing driver kernel modules. That could only be bypassed by using the last good initrd and CD kernel from the last working ISO.

And then Anaconda on CentOS-5.8 doesn't honour the "clearpart --all" command and fails to clear the partition table if it finds any existing LVM's on the drives. Apparently "clearpart --initlable --all" is supposed to do the trick, but we'll see.

But hey, it gets worse: In our kickstart scripts we use "cat", "grep" and "sort". Which now produce the error message "command not found". Why? Because the bloody stage2 image doesn't contain the BusyBox binary that we need for those commands! This is a two year old bug mentioned here in the CentOS bugtracker! Which just fuels my rage at the CentOS maintainers who always kick those in the nuts who want to spin their own CDs. It's bloody ridiculous.

In the end I still had to go in, take the CD's stage2.img apart, had to fix Anaconda and some BusyBox stuff to get it all working again. Then I had to piece stage2.img back together and needed to be really careful to use the right initrd.img to get it all working.