List of Figures

Chapter 1: So You Want to Use a Cluster

Figure 1.1: Sample decomposition of a 3-D mesh. The upper right corner box has been pulled out to show that the mesh has been subdivided along the x, y, and z axes.

Figure 2.1: Block diagram of a motherboard chipset. The chipset consists of the entire diagram excluding the processor and memory.

Figure 5.1: Layering of network protocols
Figure 5.2: Diagram showing the configuration of our simple example cluster.
Figure 5.3: Diagram showing compute nodes with multiple interfaces on multiple networks. Notice that the Myrinet network is entirely internal to the cluster, a common design point since the dedicated network is typically much higher performing than networks outside the cluster.
Figure 5.4: Above are shown some possible locations one may wish to place a firewall, denoted by the curved dotted lines.
Figure 5.5: Above are shown some of the interesting points through the Linux kernel where network packets are affected. The letters are points in kernel space where routing decisions are made. Numbered locations are some of the places where netfilters exist that will determine the fate of packets passing through. A.) incoming packet routing decision. B.) local machine process space. C.) postrouting decision. 1.) FORWARD netfilter table. 2.) INPUT netfilter table. 3.) OUTPUT netfilter table.

Figure 6.1: Cable bundles. Wire ties make 8 power cables into a neat and managable group
Figure 6.2: The back of a rack, showing the clean organization of the cables. Note that the fans are unobstructed.
Figure 6.3: Basic RedHat Kickstart file. The RedHat Installer, Anaconda, interprets the contents of the kickstart file to build a node
Figure 6.4: Description (Kickstart) Graph. This graph completely describes all of the appliances of a Rocks Cluster.
Figure 6.5: Description Graph Detail. This illustrates how two modules 'standalone.xml' and 'base.xml' share base configuration and also differ in other specifics
Figure 6.6: The ssh.xml module includes the ssh packages and configures the service in the Kickstart post section.
Figure 6.7: The 'base.xml' module configures the main section of the Kickstart file.

Figure 7.1: Schematic of a general manager-worker system
Figure 7.2: A simple server in C
Figure 7.3: A simple client in C
Figure 7.4: A simple server in Python
Figure 7.5: A simple client in Python
Figure 7.6: A simple server in Perl
Figure 7.7: A simple client in Perl
Figure 7.8: A Python server that uses select
Figure 7.9: A Python client
Figure 7.10: Matrix-matrix multiply program
Figure 7.11: Manager for parameter study
Figure 7.12: Two code fragments for parallelizing the Poisson problem with the Jacobi iteration
Figure 7.13: Two code fragments for parallelizing the Poisson problem with the Jacobi iteration, including the communication of ghost points. Note the changes in the declarations for U and UNEW.
Figure 7.14: LU Factorization code. The factors L and U are computed in-place; that is, they are stored over the input matrix a.

Figure 8.1: Simple "Hello World" program in MPI.
Figure 8.2: A more interesting version of "Hello World".
Figure 8.3: A more complex "Hello World" program in MPI. Only process 0 writes to stdout; each process sends a message to process 0.
Figure 8.4: Using MPI_Probe to find the size of a message before receiving it.
Figure 8.5: Framework of the matrix-vector multiply program.
Figure 8.6: The matrix-vector multiply program, manager code.
Figure 8.7: The matrix-vector multiply program, worker code.
Figure 8.8: Domain and 9 ? 9 computational mesh for approximating the solution to the Poisson problem.
Figure 8.9: A simple version of the neighbor exchange code. See the text for a discussion of the limitations of this routine.
Figure 8.10: A better version of the neighbor exchange code.
Figure 8.11: Computing π using collective operations.
Figure 8.12: — Computing π using the Monte Carlo method.
Figure 8.13: A parallel Poisson solver that exploits two libraries written with MPI.
Figure 8.14: The main program in a high-level program to solve a nonlinear partial differential equation using PETSc.
Figure 8.15: Jumpshot displaying message traffic.

Figure 9.1: Dynamic process matrix-vector multiply program, manager part.
Figure 9.2: Dynamic process matrix-vector multiply program, worker part.
Figure 9.3: Fault-tolerant manager.
Figure 9.4: Nonblocking exchange code for the Jacobi example.
Figure 9.5: A 12 x 12 computational mesh, divided into 4?4 domains, for approximating the solution to the Poisson problem using a two-dimensional decomposition.
Figure 9.6: Locations of mesh points in ulocal for a two-dimensional decomposition.
Figure 9.7: Nonblocking exchange code for the Jacobi problem for a two-dimensional decomposition of the mesh.
Figure 9.8: Two possible message-matching patterns when MPI_ANY_SOURCE is used in the MPI_Recv calls (from [48]).
Figure 9.9: Schematic representation of collective data movement in MPI.
Figure 9.10: Using MPI_Allgather and MPI_Allgatherv.
Figure 9.11: Parallel I/O of Jacobi solution. Note that this choice of file view works only for a single output step; if output of multiple steps of the Jacobi method are needed, the arguments to MPI_File_set_view must be modified.
Figure 9.12: C program for writing a distributed array that is also noncontiguous in memory because of a ghost area (derived from an example in [50]).
Figure 9.13: Neighbor exchange using MPI remote memory access.
Figure 9.14: Simple MPI program in C++.