Diskless Cluster Setup

From Earlham Cluster Department

(Redirected from Unified Root Setup)
Jump to: navigation, search

This page shows how to setup a diskless cluster that uses UnionFS to consolidate system and cluster software in a single disk image. It is a work in progress.

Contents

Choose an operating system

The first version of this project used Debian GNU/Linux. It's intended to be general enough to be deployed on other platforms; but, as Charlie says, "your mileage may vary."

Setup the shared environment

Install a kernel

You must build the kernel locally, otherwise the unionfs install will fail later.

cd /usr/src/linux-2.6.15.6
make oldconfig
make
cp arch/ppc/boot/images/zImage.chrp /boot/vmlinuz-2.6.15.6
make modules_install

Install unionfs

You must build unionfs with the same gcc compiler as you used to build the kernel above, otherwise you'll get an error about "Invalid module format."

Now you should be able to modprobe unionfs

See http://www.unionfs.org/ for usage examples.

Setup the server environment

PXE (i386 only)

This section is only required for i386-based machines, not PowerPCs. Download the latest syslinux bootloader: http://www.kernel.org/pub/linux/utils/boot/syslinux/

make
cp ./pxelinux.0 /boot/
mkdir /boot/pxelinux.cfg
vi /boot/pxelinux.cfg/default

add a line similar to this, but with a reference to a kernel from /boot/:

DEFAULT vmlinuz-2.6.15.7 init=/linuxrc root=/dev/nfs ip=dhcp

TFTPD

Install a tftp server (Debian's tftpd-hpa is good) and add a line like this to /etc/inetd.conf (or the equivalent in whatever internet "super-server" you use):

tftp           dgram   udp     wait    root  /usr/sbin/in.tftpd /usr/sbin/in.tftpd -v -s /boot

This runs tftpd and sets its root directory to /boot

DHCPD

Install ISC's DHCPD (e.g., Debian's dhcp3-server package), and setup dhcpd.conf like so:

option domain-name "little-fe-ppc.net";
option domain-name-servers 192.168.1.100;
option subnet-mask 255.255.255.0;
default-lease-time 604800;
max-lease-time 604800;
allow booting;
allow bootp;

subnet 192.168.1.0 netmask 255.255.255.0 {
    range 192.168.1.200 192.168.1.254;
    option broadcast-address 192.168.1.255;
    option routers 192.168.1.1;

    next-server 192.168.1.100;
    server-identifier 192.168.1.100;

    # for client-side union setup
    #   for powerpc:
    filename "vmlinuz-2.6.15.6 init=/linuxrc root=/dev/nfs ip=dhcp console=ttyS1,115200n1";
    #   for i386:
    filename "pxelinux.0"

    option root-path "/client/setup";

    # to use server-side union
    # filename "vmlinuz-2.6.15.6 root=/dev/nfs ip=dhcp console=ttyS1,115200n1";
    # option root-path "/client/root";

    use-host-decl-names on;
}

host lf1 { hardware ethernet 00:0B:2F:43:61:EB; fixed-address 192.168.1.101; }
host lf2 { hardware ethernet 00:0b:2f:4f:03:b5; fixed-address 192.168.1.102; }
#host lf3 { hardware ethernet ; fixed-address 192.168.1.103; }

subnet 192.168.10.0 netmask 255.255.255.0 {
    range 192.168.10.200 192.168.10.254;
    option broadcast-address 192.168.10.255;

    use-host-decl-names on;
}

host lf1-eth1 { hardware ethernet 00:0b:2f:62:49:d1; fixed-address 192.168.10.101; }
host lf2-eth1 { hardware ethernet 00:0b:2f:6e:6b:7b; fixed-address 192.168.10.102; }
#host lf3-eth1 { hardware ethernet ; fixed-address 192.168.10.103; }

NFSD

If you're using Debian, use the nfs-kernel-server package (not the nfs-user-server package). (Knowledge about how this translates to other platforms would be helpful here.)

Setup /etc/exports like so:

/               192.168.1.0/24(ro,insecure_locks,no_root_squash,sync)
/client/setup   192.168.1.0/24(ro,insecure_locks,no_root_squash,sync)
/client/overlay 192.168.1.0/24(ro,insecure_locks,no_root_squash,sync)
/root           192.168.1.0/24(rw,insecure_locks,no_root_squash,sync)
/home           192.168.1.0/24(rw,insecure_locks,no_root_squash,sync)

Setup the client environment

Busybox

Download and build Busybox:

make defconfig
make menuconfig #make a statically linked binary
make
make install #this creates the _install directory

Now create some necessary directories:

cd _install
mkdir dev proc sys mnt
cd ..

Now, put the _install directory somewhere that it can be NFS mounted, e.g., /client/setup

Backing up the environment

Personal tools
Namespaces
Variants
Actions
websites
wiki
this semester
Toolbox