Diskless Cluster Setup

From Earlham Cluster Department

Jump to: navigation, search

This page shows how to setup a diskless cluster that uses UnionFS to consolidate system and cluster software in a single disk image. It is a work in progress.


Choose an operating system

The first version of this project used Debian GNU/Linux. It's intended to be general enough to be deployed on other platforms; but, as Charlie says, "your mileage may vary."

Setup the shared environment

Install a kernel

You must build the kernel locally, otherwise the unionfs install will fail later.

cd /usr/src/linux-
make oldconfig
cp arch/ppc/boot/images/zImage.chrp /boot/vmlinuz-
make modules_install

Install unionfs

You must build unionfs with the same gcc compiler as you used to build the kernel above, otherwise you'll get an error about "Invalid module format."

Now you should be able to modprobe unionfs

See http://www.unionfs.org/ for usage examples.

Setup the server environment

PXE (i386 only)

This section is only required for i386-based machines, not PowerPCs. Download the latest syslinux bootloader: http://www.kernel.org/pub/linux/utils/boot/syslinux/

cp ./pxelinux.0 /boot/
mkdir /boot/pxelinux.cfg
vi /boot/pxelinux.cfg/default

add a line similar to this, but with a reference to a kernel from /boot/:

DEFAULT vmlinuz- init=/linuxrc root=/dev/nfs ip=dhcp


Install a tftp server (Debian's tftpd-hpa is good) and add a line like this to /etc/inetd.conf (or the equivalent in whatever internet "super-server" you use):

tftp           dgram   udp     wait    root  /usr/sbin/in.tftpd /usr/sbin/in.tftpd -v -s /boot

This runs tftpd and sets its root directory to /boot


Install ISC's DHCPD (e.g., Debian's dhcp3-server package), and setup dhcpd.conf like so:

option domain-name "little-fe-ppc.net";
option domain-name-servers;
option subnet-mask;
default-lease-time 604800;
max-lease-time 604800;
allow booting;
allow bootp;

subnet netmask {
    option broadcast-address;
    option routers;


    # for client-side union setup
    #   for powerpc:
    filename "vmlinuz- init=/linuxrc root=/dev/nfs ip=dhcp console=ttyS1,115200n1";
    #   for i386:
    filename "pxelinux.0"

    option root-path "/client/setup";

    # to use server-side union
    # filename "vmlinuz- root=/dev/nfs ip=dhcp console=ttyS1,115200n1";
    # option root-path "/client/root";

    use-host-decl-names on;

host lf1 { hardware ethernet 00:0B:2F:43:61:EB; fixed-address; }
host lf2 { hardware ethernet 00:0b:2f:4f:03:b5; fixed-address; }
#host lf3 { hardware ethernet ; fixed-address; }

subnet netmask {
    option broadcast-address;

    use-host-decl-names on;

host lf1-eth1 { hardware ethernet 00:0b:2f:62:49:d1; fixed-address; }
host lf2-eth1 { hardware ethernet 00:0b:2f:6e:6b:7b; fixed-address; }
#host lf3-eth1 { hardware ethernet ; fixed-address; }


If you're using Debian, use the nfs-kernel-server package (not the nfs-user-server package). (Knowledge about how this translates to other platforms would be helpful here.)

Setup /etc/exports like so:

/     ,insecure_locks,no_root_squash,sync)
/root ,insecure_locks,no_root_squash,sync)
/home ,insecure_locks,no_root_squash,sync)

Setup the client environment


Download and build Busybox:

make defconfig
make menuconfig #make a statically linked binary
make install #this creates the _install directory

Now create some necessary directories:

cd _install
mkdir dev proc sys mnt
cd ..

Now, put the _install directory somewhere that it can be NFS mounted, e.g., /client/setup

Backing up the environment

Personal tools
this semester