Myrinet Clusters Research Articles

Using benchmarking to determine efficient usage of nodes in a cluster

PurposeThe purpose of this paper is to evaluate how to use nodes in a cluster efficiently by studying the NAS Parallel Benchmarks (NASPB) on Intel Xeon and AMD Opteron dual CPU Linux clusters.Design/methodology/approachThe performance results of the NASPB are presented both with one MPI process per node (1 ppn) and with two MPI processes per node (2 ppn). These benchmark results were analyzed by considering the impact of cache effects, code scalability, memory bandwidth within nodes, and the impact of MPI and the MPI communication network. Memory bandwidth was benchmarked using MPI versions of the Streams benchmarks. The impact of MPI and the MPI communication network are evaluated by benchmarking the performance of MPI sends and receives, MPI broadcast, and the MPI all‐to‐all routines.FindingsThe performance results from running the NASPB and from the memory bandwidth benchmarks show that better performance can sometimes be achieved using 1 ppn. Performance results show that the AMD Opteron/Myrinet cluster is able to achieve significantly better utilization of the second processor than the Intel Xeon/Myrinet cluster.Practical implicationsMost Linux clusters are purchased with two processors per node. One would like to run all applications on a cluster with two processors per node using 2 ppn instead of 1 ppn in order to utilize the second processor on each node. However, our results show that this is not always the best choice. Users should always assess their program performance with both 1 ppn and 2 ppn before running production calculations. This issue becomes even more important with the emergence of multi‐core processors.Originality/valueTo the authors' best knowledge, this is the only detailed comparison of AMD Opteron and Intel Xeon dual processor node parallel performance on large Myrinet clusters. The paper should be of value to everybody considering running on or purchasing AMD or Intel‐based Linux cluster.

Multiprogrammed non-blocking checkpoints in support of optimistic simulation on myrinet clusters

CCL (checkpointing and communication library) is a software layer in support of optimistic parallel discrete event simulation (PDES) on myrinet-based COTS clusters. Beyond classical low latency message delivery functionalities, this library implements CPU offloaded, non-blocking (asynchronous) checkpointing functionalities based on data transfer capabilities provided by a programmable DMA engine on board of myrinet network cards. These functionalities are unique since optimistic simulation systems conventionally rely on checkpointing implemented as a synchronous, CPU-based data copy. Releases of CCL up to v2.4 only support monoprogrammed non-blocking checkpoints. This forces re-synchronization between CPU and DMA activities, which is a potential source of overhead, each time a new checkpoint request must be issued at the simulation application level while the last issued one is still being carried out by the DMA engine. In this paper we present a redesigned release of CCL (v3.0) that, exploiting hardware capabilities of more advanced myrinet clusters, supports multiprogrammed non-blocking checkpoints. The multiprogrammed approach allows higher degree of concurrency between checkpointing and other simulation specific operations carried out by the CPU, with benefits on performance. We also report the results of the experimental evaluation of those benefits for the case of a Personal Communication System (PCS) simulation application, selected as a real world test-bed.

Modeling and optimization of non-blocking checkpointing for optimistic simulation on myrinet clusters

Checkpointing-and-Communication Library (CCL) is a recently developed software which implements CPU offloaded, non-blocking checkpointing functionalities in support of optimistic parallel simulation on myrinet clusters. This is achieved by exploiting data transfer capabilities provided by a programmable DMA engine on board of myrinet network cards. Re-synchronization between CPU and DMA activities must sometimes be employed for several reasons, such as the maintenance of data consistency, thus adding overhead to (otherwise CPU cost-free) non-blocking checkpoint operations. In this paper we present a detailed cost model for non-blocking checkpointing and derive a performance effective re-synchronization semantic which we call minimum cost re-synchronization. With this semantic, an occurrence of re-synchronization either commits an on-going DMA based checkpoint operation (causing suspension of CPU activities) or aborts the operation (with possible increase in the expected rollback cost due to a reduced amount of committed checkpoints) on the basis of a minimum overhead expectation evaluated through the cost model.

High performance cluster computing with 3-D nonlinear diffusion filters

This paper deals with parallelization and implementation aspects of partial differential equation (PDE)-based image processing models for large cluster environments with distributed memory. As an example we focus on nonlinear diffusion filtering which we discretize by means of an additive operator splitting (AOS). We start by decomposing the algorithm into small modules that shall be parallelized separately. For this purpose image partitioning strategies are discussed and their impact on the communication pattern and volume is analyzed. Based on the results we develop an algorithmic implementation with excellent scaling properties on massively connected low-latency networks. Test runs on two different high-end Myrinet clusters yield almost linear speedup factors up to 209 for 256 processors. This results in typical denoising times of 0.4 s for five iterations on a 256×256×128 data cube.

Understanding performance of SMP clusters running MPI programs

Clusters of multiprocessors (CLUMPs) have an hybrid memory model, with message passing between nodes and shared memory inside nodes. We examine the performance of Myrinet clusters of SMP PCs when using a single memory model (SMM) based on the MPICH-PM/CLUMP library of the RWCP, which can directly use the MPI programs written for a cluster of uniprocessors. The specificities of the communication patterns with the SMM approach are detailed. PC clusters with 2-way and 4-way nodes are considered and compared.