Running Timer Code

From Earlham Cluster Department

(Difference between revisions)
Jump to: navigation, search
(update instructions -- toby & alex)
 
Line 3: Line 3:
===Setup a client and server with the tp_timer kernel===
===Setup a client and server with the tp_timer kernel===
-
If you already have a client and server running the proper kernel, skip to the [[#Check_out_and_run_the_test_script|next section]].  tp_timer should be compiled in module format (simply copy the -mod config below, as indicated).
 
'''1. Build and install the kernel.'''
'''1. Build and install the kernel.'''
To compile the kernel, check out <code>llk-sources</code> from CVS and use a .config file from llk-sources/ccg-configs. Make sure that <code>CONFIG_TP_TIMER=y</code> (for tp_timer built in) or <code>CONFIG_TP_TIMER=m</code> (for a module version of tp_timer). Note that there is a significant chance that this compilation will crash the build host is running tp_timer (having the module unloaded is okay). This is probably a problem with NFS, so building it locally may work (but watch out for distcc and ccache that like to touch files in your home directory).  In short, build on a machine that isn't running a tp_timer kernel.
To compile the kernel, check out <code>llk-sources</code> from CVS and use a .config file from llk-sources/ccg-configs. Make sure that <code>CONFIG_TP_TIMER=y</code> (for tp_timer built in) or <code>CONFIG_TP_TIMER=m</code> (for a module version of tp_timer). Note that there is a significant chance that this compilation will crash the build host is running tp_timer (having the module unloaded is okay). This is probably a problem with NFS, so building it locally may work (but watch out for distcc and ccache that like to touch files in your home directory).  In short, build on a machine that isn't running a tp_timer kernel.
 +
 +
You probably want the module config (<code><machine>-mod</code> below).  The resulting kernel version will be '2.6.9-llk-mod.'  After building, <code>./ikl.py</code> in llk-sources will install the kernel locally for you.  2.6.9-llk-mod is the default kernel for both grub (on bazaar) and yaboot
 +
(on cairo).
<pre>
<pre>
cvs co llk-sources
cvs co llk-sources
 +
cvs co llk
cd llk-sources
cd llk-sources
cp ccg-configs/<machine>-mod .config # where machine is 'cairo' or 'bazaar'
cp ccg-configs/<machine>-mod .config # where machine is 'cairo' or 'bazaar'
Line 17: Line 20:
</pre>
</pre>
-
'''2.  Setup the boot-loader'''
+
'''2.  Copy to another machine.'''
 +
If you need to copy the kernel to another machine, run:
 +
<pre>
 +
llk/scripts/distribute.sh 2.6.9-llk-mod <host name>
 +
</pre>
 +
 
 +
'''3.  Manage your kernels.'''
 +
We will probably accumulate a large number of kernels over the next couple days.  These scripts will aid in managing that proliferation:
 +
*<code>llk/scripts/kernel_store.sh</code> saves the currently running kernel binary + associated files to a local directory (default: /usr/local/kernels/<kernel name specified>).
 +
*<code>llk/scripts/kernel_load.sh</code> restores one of the previously saved kernels to /boot (under the name of the currently running kernel).
 +
 
 +
'''Boot-loader info,''' incase it's needed (you shouldn't have to change this!):
<pre>
<pre>
# on cairo:
# on cairo:
Line 27: Line 41:
grub < /etc/grub.conf
grub < /etc/grub.conf
</pre>
</pre>
-
 
-
'''3.  Reboot.'''
 
-
<pre>
 
-
reboot
 
-
</pre>
 
-
 
===Check out and run the test script===
===Check out and run the test script===
This must be done on a non-testing node, e.g., hopper.  If you run this on a node you're using to test and accidently touch NFS, the machine will crash.
This must be done on a non-testing node, e.g., hopper.  If you run this on a node you're using to test and accidently touch NFS, the machine will crash.

Latest revision as of 10:43, 21 February 2006

Contents

Obtaining Kernel Timing Data

Testing is now radically simplified with the run_test.sh script.

Setup a client and server with the tp_timer kernel

1. Build and install the kernel. To compile the kernel, check out llk-sources from CVS and use a .config file from llk-sources/ccg-configs. Make sure that CONFIG_TP_TIMER=y (for tp_timer built in) or CONFIG_TP_TIMER=m (for a module version of tp_timer). Note that there is a significant chance that this compilation will crash the build host is running tp_timer (having the module unloaded is okay). This is probably a problem with NFS, so building it locally may work (but watch out for distcc and ccache that like to touch files in your home directory). In short, build on a machine that isn't running a tp_timer kernel.

You probably want the module config (<machine>-mod below). The resulting kernel version will be '2.6.9-llk-mod.' After building, ./ikl.py in llk-sources will install the kernel locally for you. 2.6.9-llk-mod is the default kernel for both grub (on bazaar) and yaboot (on cairo).

cvs co llk-sources
cvs co llk
cd llk-sources
cp ccg-configs/<machine>-mod .config # where machine is 'cairo' or 'bazaar'
make oldconfig
make -jN CC="ccache gcc" # where N is the number of hosts in DISTCC_HOSTS * 4
./ikl.py # ikl = install kernel locally

2. Copy to another machine. If you need to copy the kernel to another machine, run:

llk/scripts/distribute.sh 2.6.9-llk-mod <host name>

3. Manage your kernels. We will probably accumulate a large number of kernels over the next couple days. These scripts will aid in managing that proliferation:

Boot-loader info, incase it's needed (you shouldn't have to change this!):

# on cairo:
vim /etc/yaboot.conf
ybin -v

# on bazaar:
vim /boot/grub/menu.lst
grub < /etc/grub.conf

Check out and run the test script

This must be done on a non-testing node, e.g., hopper. If you run this on a node you're using to test and accidently touch NFS, the machine will crash.

cvs co llk
cd llk/scripts
./run_test.sh <client> <server> <protocol> <table name> <test set>

Where <client> and <server> are short host names like 'b10' and the protocol is 'udp' or 'tcp'. The script will load the module on both hosts, calibrate the test, run the test, fetch the data, unload the module, and insert the data into the database with the specified table name and test set label.

Older info

Basic Flow:

  1. make sure the tp_timer module is installed and working on the client and server nodes
  2. clear /proc/tp_timer on both client and server machines (just read its contents; cat /proc/tp_timer > /dev/null)
  3. run pacgen on client and server to amass new data.
  4. read data into log files on both server and client
  5. load data from the log files into database
  6. look at data via php scripts

Machines setup with llk timing: b10,b11,c10,c11

Need to check out from CVS:

pacgen sample commands:

./server tcp <port number>
./client -t <protocol> -s <source IP> -m <source port> -d <dest IP> -p <dest port>

Each window is a specific ID for the packet. This ID is used as the payload for the packets. Useful for checking for packet fragmentation (educated guess via Alex). Windows are 0xffffffff, 0xffffffff (2 unsigned ints worth of 1's), an unsigned int for socket/thread id (We don't use it; that part of the code was removed), and an unsigned int holding the window number (we don't know what this is for).

After pacgen is ran with on a tp timer enabled machine, the timing results will be available in /proc/tp_timer. Note that after you read this proc entry, the information will be cleared.


To enter the data into our database:

  1. Collect the /proc/tp_timer from the server node and the client node:
    1. cat /proc/tp_timer > tp_timer_client.log
  2. Execute the create-table-llk.pl script found in the the llk cvs module in the scripts sub directory.
    1. Usage: ./create-table-llk.pl <tablename> <label> <datafile> <client|server>
    2. If the table given by name to the scripts doesn't exist, the script creates the table.
Personal tools
Namespaces
Variants
Actions
websites
wiki
this semester
Toolbox