Symmetric Multiprocessor Research Articles

An educational tool for testing caches on symmetric multiprocessors

In this article, we present a simulator for cache memory systems on symmetric multiprocessors. This simulator is called SMPCache. It has a full graphic and user-friendly interface, and it operates on PC systems with Windows. The simulator has been conceived as a tool for the teaching of cache memories on multiprocessors systems. This tool is very useful to evaluate and understand different design alternatives: the number of processors, the cache coherence protocols, schemes for bus arbitration, mapping, replacement policies, cache size, memory block size, etc. Our experiences in the last three years have demonstrated to us the benefits of the simulator for teaching purposes.

PARALLEL COMPUTATION OF TURBULENT COMBUSTION AND FLAME SPREAD IN FIRES

A parallel procedure based on a single-program, multiple-data (SPMD) algorithm is presented for parallel computing of turbulent combustion and flame spread in fires. The computation is based on modeling of radiative turbulent reacting flow and pyrolysis of solid fuel. With angular domain decomposition applied to the parallel computing of radiation and spatial domain decomposition to the computation of nonradiative turbulent reacting flow and solid fuel pyrolysis, the whole computation is distributed among a group of concurrent tasks, which communicate with each other through a message-passing interface library. Using this procedure, a self-developed computational combustion code has been parallelized on both a multiprocessor PC and a symmetric multiprocessor (SMP) system, SGI Origin 2000. The parallelization was verified by comparing the parallel results with sequential results. The performance of the parallel procedure was evaluated using various test cases. As expected, the efficiency of parallelism varies with both computer architecture and case scenario. In general, good efficiency was obtained.

A simple process for migrating server applications to SMP:s

A strong focus on quality attributes such as maintainability and flexibility has resulted in a number of new methodologies, e.g., object-oriented and component-based design, which can significantly limit the application performance. A major challenge is to find solutions that balance and optimize the quality attributes, e.g., symmetric multiprocessor (SMP) performance contra maintainability and flexibility. We have studied three large real-time telecommunication server applications developed by Ericsson. In all these applications maintainability is strongly prioritized. The applications are also very demanding with respect to performance due to real-time requirements on throughput and response time. SMP:s and multithreading are used in order to give these applications a high and scalable performance. Our main finding is that dynamic memory management is a major bottleneck in these types of applications. The bottleneck can, however, be removed using memory allocators optimized for SMP:s or by reducing the number of allocations. We found that the number of allocations can be significantly reduced by using alternative design strategies for maintainability and flexibility. Based on our experiences we have defined a simple guideline-based process with the aim of helping designers of server applications to establish a balance between SMP performance, maintainability and flexibility.

Benchmarking MILC code with OpenMP and MPI

A trend in high performance computers that is becoming increasingly popular is the use of symmetric multi-processing (SMP) rather than the older paradigm of MPP. MPI codes that ran and scaled well on MPP machines can often be run on an SMP machine using the vendor's version of MPI. However, this approach may not make optimal use of the (expensive) SMP hardware. More significantly, there are machines like Blue Horizon, an IBM SP with 8-way SMP nodes at the San Diego Supercomputer Center that can only support 4 MPI processes per node (with the current switch). On such a machine it is imperative to be able to use OpenMP parallelism on the node, and MPI between nodes. We describe the challenges of converting MILC MPI code to using a second level of OpenMP parallelism, and benchmarks on IBM and Sun computers.

Achieving 60 GFLOP/s on the production CFD code OVERFLOW-MLP

NASA Ames has developed a new methodology for achieving very high levels of parallel efficiency on the new NUMA based shared memory symmetric Multi-Processing (SMP) computing systems available today. This methodology is simple, general, and widely applicable to production CFD codes in use at NASA and elsewhere. The new methodology is formally called shared memory Multi-Level Parallelism (MLP), and is based on shared memory access to global data while invoking two levels of parallelism for scaling efficiency. During the past year, this technique has been refined and improved and inserted into the OVERFLOW production CFD code. Executions of the new code on an SGI Origin R12K system with 512 CPUs have demonstrated over 60 GFLOP/s of sustained performance for customer driven real problems. A detailed discussion of the MLP technique, OVERFLOW-MLP code optimizations, and performance results are presented.

Prefix Computations on Symmetric Multiprocessors

We introduce a new prefix computation algorithm on linked lists which builds upon the sparse ruling set approach of Reid-Miller and Blelloch. Besides being somewhat simpler and requiring nearly half the number of memory accesses, we can bound our complexity with high probability instead of merely on average. Moreover, whereas Reid-Miller and Blelloch targeted their algorithm for implementation on a vector multiprocessor architecture, we develop our algorithm for implementation on the symmetric multiprocessor architecture (SMP). These symmetric multiprocessors dominate the high-end server market and are currently the primary candidate for constructing large scale multiprocessor systems. Our prefix computation algorithm was implemented in C using POSIX threads and run on four symmetric multiprocessors: the HP-Convex Exemplar (S-Class), the IBM SP-2 (High Node), the SGI Power Challenge, and the DEC AlphaServer. We ran our code using a variety of benchmarks which we identified to examine the dependence of our algorithm on memory access patterns. For some problems, our algorithm actually matched or exceeded the performance of the standard sequential solution using only a single thread. Moreover, in spite of the fact that the processors must compete for access to main memory, our algorithm still achieved scalable performance with up to 16 processors, which was the largest platform available to us.

Accelerating shared virtual memory via general-purpose network interface support

Clusters of symmetric multiprocessors (SMPs) are important platforms for high-performance computing. With the success of hardware cache-coherent distributed shared memory (DSM), a lot of effort has also been made to support the coherent shared-address-space programming model in software on clusters. Much research has been done in fast communication on clusters and in protocols for supporting software shared memory across them. However, the performance of software virtual memory (SVM) is still far from that achieved on hardware DSM systems. The goal of this paper is to improve the performance of SVM on system area network clusters by considering communication and protocol layer interactions. We first examine what are the important communication system bottlenecks that stand in the way of improving parallel performance of SVM clusters; in particular, which parameters of the communication architecture are most important to improve further relative to processor speed, which ones are already adequate on modern systems for most applications, and how will this change with technology in the future. We find that the most important communication subsystem cost to improve is the overhead of generating and delivery interrupts for asynchronous protocol processing. Then we proceed to show, that by providing simple and general support for asynchronous message handling in a commodity network interface (NI) and by altering SVM protocols appropriately, protocol activity can be decoupled from asynchronous message handling, and the need for interrupts or polling can be eliminated. The NI mechanisms needed are generic, not SVM-dependent. We prototype the mechanisms and such asynchronous home-basedLRCprotocol, calledGeNIMA(GEneral-purpose Network Interface support for shared Memory Abstractions), on a cluster of SMPs with a programmable NI. We find that the performance improvements are substantial, bringing performance on a small-scale SMP cluster much closer to that of hardware-coherent shared memory for many applications, and we show the value of each of the mechanisms in different applications.

Analysis and measurement of the effect of kernel locks in SMP systems

AbstractThis article reports the use of case studies to evaluate the performance degradation caused by the kernel‐level lock. We define the lock ratio as a ratio of the execution time for critical sections to the total execution time of a parallel program. The kernel‐level lock ratio determines how effective programs work on symmetric multiprocessor (SMP) systems. We have measured the lock ratios and the performance of three types of parallel programs on SMP systems with Linux 2.0: matrix multiplication, parallel make, and WWW server programs. Experimental results show that the higher the lock ratio of parallel programs, the worse their performance becomes. Copyright © 2001 John Wiley & Sons, Ltd.

Fast Bit-Reversals on Uniprocessors and Shared-Memory Multiprocessors

In this paper, we examine different methods using techniques of blocking, buffering, and padding for efficient implementations of bit-reversals. We evaluate the merits and limits of each technique and its application and architecture-dependent conditions for developing cache-optimal methods. Besides testing the methods on different uniprocessors, we conducted both simulation and measurements on two commercial symmetric multiprocessors (SMP) to provide architectural insights into the methods and their implementations. We present two contributions in this paper: (1) Our integrated blocking methods, which match cache associativity and translation-lookaside buffer (TLB) cache size and which fully use the available registers, are cache-optimal and fast. (2) We show that our padding methods outperform other software-oriented methods, and we believe they are the fastest in terms of minimizing both CPU and memory access cycles. Since the padding methods are almost independent of hardware, they could be widely used on many uniprocessor workstations and multiprocessors.

Timestamp snooping

Symmetric multiprocessor (SMP) servers provide superior performance for the commercial workloads that dominate the Internet. Our simulation results show that over one-third of cache misses by these applications result in cache-to-cache transfers, where the data is found in another processor's cache rather than in memory. SMPs are optimized for this case by using snooping protocols that broadcast address transactions to all processors. Conversely, directory-based shared-memory systems must indirectly locate the owner and sharers through a directory, resulting in larger average miss latencies.This paper proposes timestamp snooping, a technique that allows SMPs to i) utilize high-speed switched interconnection networks and ii) exploit physical locality by delivering address transactions to processors and memories without regard to order. Traditional snooping requires physical ordering of transactions. Timestamp snooping works by processing address transactions in a logical order. Logical time is maintained by adding a few bits per address transaction and having network switches perform a handshake to ensure on-time delivery. Processors and memories then reorder transactions based on their timestamps to establish a total order.We evaluate timestamp snooping with commercial workloads on a 16-processor SPARC system using the Simics full-system simulator. We simulate both an indirect (butterfly) and a direct (torus) network design. For OLTP, DSS, web serving, web searching, and one scientific application, timestamp snooping with the butterfly network runs 6-28% faster than directories, at a cost of 13-43% more link traffic. Similarly, with the torus network, timestamp snooping runs 6-29% faster for 17-37% more link traffic. Thus, timestamp snooping is worth considering when buying more interconnect bandwidth is easier than reducing interconnect latency.

Exploiting Application Tunability for Efficient, Predictable Resource Management in Parallel and Distributed Systems

Parallel and distributed computing is becoming increasingly mainstream, driven both by the widespread availability of commodity small-scale symmetric multiprocessors and high-performance cluster platforms, as well as the growing use of parallelism and distribution in networked applications such as image recognition, media processing, virtual reality, and telepresence. However, many of these applications impose soft timeliness and output quality constraints on top of the traditional performance requirements, necessitating efficient, predictable management of system resources. Existing techniques are inadequate to simultaneously support these twin requirements of efficiency and predictability. In this paper, we propose a novel approach for increasing system efficiency while meeting application timeliness and quality constraints. Our approach exploits the application tunability found in many general-purpose computations. Tunability refers to an application's ability to trade off resource requirements over several dimensions including time, quality, and resource type; the resulting flexibility enables the underlying resource management system to choose an application operating point best suited to available resource characteristics. We describe language and scheduler extensions to support tunability in the MILAN metacomputing environment and then systematically characterize performance benefits of tunability using a parameterizable task system. Our results show that application tunability is easily expressible and can significantly improve resource utilization.

Parallel molecular dynamics simulations of alkane/hydroxylated α -aluminum oxide interfaces

In this paper we describe a practical implementation of parallel computation for the molecular dynamics (MD) simulation of an alkane/aluminum oxide interface. A serial MD program was converted into a parallel code utilizing the message passing interface (MPI). This code was evaluated on a twelve processor symmetrical multiprocessor as well as on a cluster of four processor SMPs. A maximum speedup of 5.25 was achieved with twelve processors on the large shared memory machine. The cluster performance saturated at a speedup of 4.5 with two nodes. High communication costs and considerable load imbalance in the system were identified as areas that need further investigation for obtaining better performance.

Cacheminer: A runtime approach to exploit cache locality on SMP

Exploiting cache locality of parallel programs at runtime is a complementary approach to a compiler optimization. This is particularly important for those applications with dynamic memory access patterns. We propose a memory-layout oriented technique to exploit cache locality of parallel loops at runtime on Symmetric Multiprocessor (SMP) systems. Guided by application-dependent and targeted architecture-dependent hints, our system, called Cacheminer, reorganizes and partitions a parallel loop using the memory-access space of its execution. Through effective runtime transformations, our system maximizes the data reuse in each partitioned data region assigned in a cache, and minimizes the data sharing among the partitioned data regions assigned to all caches. The executions of tasks in the partitions are scheduled in an adaptive and locality-presented way to minimize the execution time of programs by trading off load balance and locality. We have implemented the Cacheminer runtime library on two commercial SMP servers and an SimCS simulated SMP. Our simulation and measurement results show that our runtime approach can achieve comparable performance with the compiler optimizations for programs with regular computation and memory-access patterns, whose load balance and cache locality can be well optimized by the tiling and other program transformations. However, our experimental results show that our approach is able to significantly improve the memory performance for the applications with irregular computation and dynamic memory access patterns. These types of programs are usually hard to optimize by static compiler optimizations.

Gigantic clusters: where are they and what are they doing

Clusters are like skyscrapers: mammoth structures towering above their smaller counterparts, except that skyscrapers have longer lives. The race to build ever bigger clusters is on, largely driven by emerging scientific and engineering applications. Several research efforts are underway at various universities and US research laboratories. The author examines some of the largest clusters in the world, providing some recent news and opinions from the experts. A cluster is a collection of complete computers (nodes) interconnected by a high-speed network. Typically, each node is a workstation, PC, or symmetric multiprocessor (SMP). Cluster nodes work collectively as a single computing resource and fill the conventional role of using each node as an independent machine. A cluster computing system is a compromise between a massively parallel processing system and a distributed system. An MPP system node typically cannot serve as a standalone computer; a cluster node usually contains its own disk and a complete operating system, and therefore, also can handle interactive jobs. In a distributed system, nodes can serve only as individual resources while a cluster presents a single system image to the user.

Architectural approach to the role of optics in monoprocessor and multiprocessor machines

The relevance of introducing optical interconnects (OI's) in monoprocessors and multiprocessors is studied from an architectural point of view. We show that perhaps the major explanation for why optical technologies have nearly been unable to penetrate into computers is that OI's generally do not shorten the memory-access time, which is the most critical issue for today's stored-program machines. In monoprocessors the memory-access time is dominated by the electronic latency of the memory itself. Thus implementing OI's inside the memory hierarchy without changing the memory architecture cannot dramatically improve the global performance. In strongly coupled multiprocessors the node-bypass latency dominates. Therefore the higher the connectivity (possibly with optics), the shorter the path to another node, but the more expensive the network and the more complex the structure of electronic nodes. This relation leaves the choice of the best network open in terms of simplicity and latency reduction. The bottlenecks resulting from and the benefits of implementing OI's are discussed with respect to symmetric multiprocessors, rings, and distributed shared-memory supercomputers.

Modeling and Simulative Performance Analysis of SMP and Clustered Computer Architectures

The performance cliaracteris tics of several classes of parallel computing systems are analyzed and com pared using high-fidelity modeling and execution- driven simulation. Processor, bus and network mod els are used to construct and simulate the architec tures of symmetric multiprocessors (SMPs), clusters of uniprocessors, and clusters of SMPs. To demon strate a typical use, the performance of ten systems is evaluated using a parallel matrix-multiplication algorithm. Because the performance of a parallel algo rithm on an architecture depends on its communica tion-to-computation ratio, an analysis of communica tion latencies for bus transactions, cache coherence, and network transactions is used to quantify each system's communication overhead. While low-level performance attributes are difficult to measure on experimental testbed systems, and are difficult to accurately represent in purely analytical models, with high fidelity simulative models they can be readily and accurately obtained. This level of detail allows the designer to rapidly prototype and evaluate the performance of parallel and distributed systems.

Concurrent simulation of neutral flow in the GEC reference cell

This paper describes computational techniques for concurrent Direct Simulation Monte Carlo (DSMC) of neutral flow inside three-dimensional plasma reactors. These techniques are designed to reduce the overall time to obtain realistic three-dimensional results. A simulation problem is first defined by automatically generating a tetrahedral grid. This grid is statically partitioned in order to achieve scalable memory utilization. Parallel algorithms allow high-performance multicomputer architectures to be leveraged to reduce simulation time. Adaptive gridding techniques are employed to maintain the desired accuracy in the presence of dynamic flow variations. Load balancing algorithms, based on heat diffusion, are used to maximize efficiency. Finally, automatic granularity control is used to ensure appropriate granularity for load balancing and for maximum processor utilization. Simulation results are presented for a simulation of the GEC reference cell in a flow configuration of industrial relevance. Performance results are presented for large-scale multicomputers, symmetric multiprocessors and networks of PCs. Copyright © 2000 John Wiley & Sons, Ltd.

Concurrent simulation of neutral flow in the GEC reference cell

This paper describes computational techniques for concurrent Direct Simulation Monte Carlo (DSMC) of neutral flow inside three-dimensional plasma reactors. These techniques are designed to reduce the overall time to obtain realistic three-dimensional results. A simulation problem is first defined by automatically generating a tetrahedral grid. This grid is statically partitioned in order to achieve scalable memory utilization. Parallel algorithms allow high-performance multicomputer architectures to be leveraged to reduce simulation time. Adaptive gridding techniques are employed to maintain the desired accuracy in the presence of dynamic flow variations. Load balancing algorithms, based on heat diffusion, are used to maximize efficiency. Finally, automatic granularity control is used to ensure appropriate granularity for load balancing and for maximum processor utilization. Simulation results are presented for a simulation of the GEC reference cell in a flow configuration of industrial relevance. Performance results are presented for large-scale multicomputers, symmetric multiprocessors and networks of PCs. Copyright © 2000 John Wiley & Sons, Ltd.

The Same-Source Parallel MM5

Beginning with the March 1998 release of the Penn State University/NCAR Mesoscale Model (MM5), and continuing through eight subsequent releases up to the present, the official version has run on distributed -memory (DM) parallel computers. Source translation and runtime library support minimize the impact of parallelization on the original model source code, with the result that the majority of code is line-for-line identical with the original version. Parallel performance and scaling are equivalent to earlier, hand-parallelized versions; the modifications have no effect when the code is compiled and run without the DM option. Supported computers include the IBM SP, Cray T3E, Fujitsu VPP, Compaq Alpha clusters, and clusters of PCs (so-called Beowulf clusters). The approach also is compatible with shared-memory parallel directives, allowing distributed-memory/shared-memory hybrid parallelization on distributed-memory clusters of symmetric multiprocessors.

HIGHLY SCALABLE PARALLEL COMPUTATIONAL MODELS FOR LARGE-SCALE RTM PROCESS MODELING SIMULATIONS, PART 3: VALIDATION AND PERFORMANCE RESULTS

In Part 3 of this article, we illustrate the performance results and scalability obtained by implementing the traditional explicit control-volume (CV)/finite-element (FE) and a recently developed and new, implicit, pure finite-element approach in symmetric multiprocessor machines as illustrated in the preceding Part 1 and Part 2. The results indicate that the current-generation symmetric multiprocessors (SMPs) with a medium to large number of processors can be effectively used as a supercomputer (Massively Parallel Platforms, MPPs) for large-scale complex geometries in scientific and engineering problems. The recently developed and new implicit pure FE methodology is shown to be physically accurate, computationally superior, and applicable to practical large-scale problems compared to the traditional explicit CV-FE on different SMP platforms (SGI Power Challenge and SGI Origin2000). For large-scale problems, the explicit CV-FE-based modeling/analysis becomes impossible to analyze within reasonable time or realistically impossible even with parallel processing for large-scale problems. In view of such considerations, as an illustration, a large, complex finite-element mesh of 809,505 elements and 405,327 nodes has been successfully analyzed using the implicit pure FE methodology and 40 processors within a reasonable time of 4.72 hours. The proposed parallel approaches have excellent parallel efficiency and optimal scalability when compared to other relevant results in computational sciences published in the literature, with the highest degree of portability of the software code to a wide range of parallel architectures.