From Earlham Cluster Department
This page is now stale
Your mileage may vary, and we are not updating this page any longer. We use it internally for reference, but we are now working on Liberating the BCCD (see the main Cluster Computing Group page).
Fossilizing the BCCD
This section outlines the steps required to disassemble a BCCD ISO, manifest it on a hard disk drive, and boot from that hard drive. Most or all of this must be done as root.
Mount the Images
These scripts, used for the lnx-bbc project, might prove to be helpful in working with the BCCD images: FossilScripts
The Basic Images
cd /mnt # or where ever mkdir bccd mount -t iso9660 -o loop bccd-ppc-2005-08-30T00-0500.iso bccd # on PPC mkdir initrd gunzip < bccd/boot/root.bin > initrd.ext2 mount -t ext2 -o loop initrd.ext2 initrd # on x86 mkdir lnx mount -o loop bccd/lnx.img lnx mkdir root gunzip < lnx/root.bin > root.ext2 mount -o loop root.ext2 root
First, decompress the singularity with the cloop utility
wget http://developer.linuxtag.net/knoppix/sources/cloop_0.66-1.tar.gz tar xzf cloop_0.66-1.tar.gz cd cloop-0.66 vim Makefile # add APPSONLY=1 at the top make zcode make extract_compressed_fs ./extract_compressed_fs ../bccd/singularity > ../singularity.romfs cd ..
The latest currently-available version of cloop (2.01) doesn't work for this purpose; others might (I didn't experiment), but 0.66 definitely does.
Next, mount the singularity (you must have romfs support compiled into the kernel):
mkdir singularity mount -t romfs -o loop singularity.romfs singularity
Extract the singularity
cd singularity tar cf - . | (cd /path/to/destination/partition;tar xvf -)
Create a working initrd
Create an initrd for fossilized booting with the linuxrc at http://ppckernel.org/~tobias/bccd/linuxrc:
cd /mnt/root # or where ever you mounted root.ext2 (from root.bin) wget http://ppckernel.org/~tobias/bccd/linuxrc # replace the existing linuxrc chmod a+x linuxrc cd .. umount root gzip < root.ext2 > /path/to/destination/partition/boot/root.bin
Add / remount read-write hook
/sbin/singularity-init to remount / read-write during init, using the following command:
debug "Remounting / read-write..." mount -o rw,remount /dev/root /
This can be placed somewhere around the proc mount command.
Prepare for Fossilization of /mnt/rw
Comment out lines concerning /mnt/rw
# mount -n -t tmpfs none /mnt/rw
Add network setup to singularity-init
ifconfig eth0 inet 192.168.10.1 netmask 255.255.255.0 broadcast 192.168.10.255 up route add default gw 192.168.10.1 eth0
Configure the bootloader
Configure your bootloader (e.g., yaboot, lilo, or grub) as follows:
- boot the kernel
/boot/vmlinuxon PowerPC or
- use the initrd
- execute the init script
Here is a sample lilo.conf.
Setup Compatibility Nodes
Add the following to /linuxrc:
- /sbin/devfsd /dev
The deal is that the BCCD is now on a different (read/writeable) medium: a harddisk. Let's un-obfuscate some of the workings. An ls -l on / will reveal a few symlinks: /etc, /home, /local, /tmp, and /var. All of these point to an appropriate directory in /mnt/rw. What happens is that since the CD is not writeable, it creates a ramdisk, copies files from /etc.ro/ to /mnt/rw/etc/ (change etc accordingly), and then the /etc symlink becomes a writeable medium.
Here's the works:
rm /etc /home /local /tmp /var mkdir /etc /home /local /tmp /var cd /etc.ro && tar cf - . | (cd /etc/; tar vf -) cd /home.ro && tar cf - . | (cd /home/; tar vf -) cd /local.ro && tar cf - . | (cd /local/; tar vf -) cd /tmp.ro && tar cf - . | (cd /tmp/; tar vf -) cd /var.ro && tar cf - . | (cd /var/; tar vf -)
You're almost done, except you should remove the place in the scripts where the bootup copies the files from /<dir>.ro/. Just comment out the lines in /sbin/singularity-init that do the copying (around line 105):
# cp -a /etc.ro /mnt/rw/etc # cp -a /var.ro /mnt/rw/var
While you're editing /sbin/singularity-init, also comment out these lines:
# rsync -plarv /lib/mozilla-1.6/plugins.ro/ /mnt/rw/plugins/ # chmod 1777 /mnt/rw/tmp # debug "Making /mnt/rw/tmp/build links" # mkdir -p /mnt/rw/tmp/build/ # mkdir -p /mnt/rw/tmp/build/staging # mkdir -p /mnt/rw/tmp/build/staging/singularity # mkdir -p /mnt/rw/tmp/build/staging/singularity/image # ln -s /lib /mnt/rw/tmp/build/staging/singularity/image/lib
Configure gcc Environment
Though the BCCD is now fossilized onto the harddrive, the gcc environment does not know this as it was compiled for the CD. It will look for files in (effectively) /tmp/build/staging/singularity/image/lib ... the directories and symlink creation that we just commented out. Since /tmp is a fossilized directory, just create a symlink inside of it:
mkdir -p /tmp/build/staging/singularity/image cd /tmp/build/staging/singularity/image/ ln -s /lib
- fix the mounting commands so that / is only mounted once (?)
- decide how to handle directories like /etc that are mounted in ram at /dev/rw/etc and populated with items from /etc.ro (leave as is, or create a script to simplify the setup for hard disk booting?)
- Kevin's done this, we just need to document
- Kevin's done this, we just need to document
- modify init scripts to make them appropriate for hard disk booting (e.g., remove the "Enter a password for the default user" prompt)
- This appears to be done
- finish setting up networking
- create a patch against the original singularity image for /sbin/singularity-init and other modified configuration files for automating the fossilize process
- package up any binary additions with list-packages (see the package instructions in the wiki)
- last but not least, keep track of all the changes we make!
Good luck! Direct questions and comments to email@example.com.