Clusters Of Symmetric Multiprocessors Research Articles

Multilayered grouping parallel algorithm for multiple‐level multiscale analyses

SUMMARYMultiscale analysis technique became a successful remedy to complicated problems in which nonlinear behavior is linked with microscopic damage mechanisms. For efficient parallel multiscale analyses, hierarchical grouping algorithms (e.g., the two‐level ‘coarse‐grained’ method) have been suggested and proved superior over a simple parallelization. Here, we expanded the two‐level algorithm to give rise to a multilayered grouping parallel algorithm suitable for large‐scale multiple‐level multiscale analyses. With practical large‐scale applications, we demonstrated the superior performance of multilayered grouping over the coarse‐grained method. Notably, we show that the unique data transfer rates of the symmetric multiprocessor cluster system can lead to the seemingly ‘super‐linear speedup’ and that there appears to exist the optimal number of subgroups of three‐tiered multiscale analysis. Copyright © 2014 John Wiley & Sons, Ltd.

A high‐productivity task‐based programming model for clusters

SUMMARYProgramming for large‐scale, multicore‐based architectures requires adequate tools that offer ease of programming and do not hinder application performance. StarSs is a family of parallel programming models based on automatic function‐level parallelism that targets productivity. StarSs deploys a data‐flow model: it analyzes dependencies between tasks and manages their execution, exploiting their concurrency as much as possible.This paper introduces Cluster Superscalar (ClusterSs), a new StarSs member designed to execute on clusters of SMPs (Symmetric Multiprocessors). ClusterSs tasks are asynchronously created and assigned to the available resources with the support of the IBM APGAS runtime, which provides an efficient and portable communication layer based on one‐sided communication.We present the design of ClusterSs on top of APGAS, as well as the programming model and execution runtime for Java applications. Finally, we evaluate the productivity of ClusterSs, both in terms of programmability and performance and compare it to that of the IBM X10 language. Copyright © 2012 John Wiley & Sons, Ltd.

Hybrid Message-Passing and Shared-Memory Programming in a Molecular Dynamics Application On Multicore Clusters

Hybrid programming, whereby shared-memory and message-passing programming techniques are combined within a single parallel application, has often been discussed as a method for increasing code performance on clusters of symmetric multiprocessors (SMPs). This paper examines whether the hybrid model brings any performance benefits for clusters based on multicore processors. A molecular dynamics application has been parallelized using both MPI and hybrid MPI/OpenMP programming models. The performance of this application has been examined on two high-end multicore clusters using both Infiniband and Gigabit Ethernet interconnects. The hybrid model has been found to perform well on the higher-latency Gigabit Ethernet connection, but offers no performance benefit on low-latency Infiniband interconnects. The changes in performance are attributed to the differing communication profiles of the hybrid and MPI codes.

Efficient shared memory and RDMA based collectives on multi-rail QsNetII SMP clusters

Clusters of Symmetric Multiprocessors (SMP) are more commonplace than ever in achieving high-performance. Scientific applications running on clusters employ collective communications extensively. S...

Parallelization methods for implementation of a magnetic induction tomography forward model in symmetric multiprocessor systems

This paper describes four parallelization approaches used in a finite-difference-based electromagnetic modeller for application in magnetic induction tomography (MIT) and suitable for implementation on computer systems with symmetric multiprocessor (SMP) architecture. The approaches include: (i) splitting by coils using a distributed memory approach, (ii) splitting by physical domain using a distributed memory approach, (iii) splitting by physical domain using hybrid distributed/shared memory approach and (iv) splitting by both coils and physical domain using multi-level distributed and shared memory approaches respectively. All four approaches were implemented and tested on an IBM SP supercomputer. Coil parallelization was the most efficient method due to low inter-processor communication requirements but was limited by the number of coils in the MIT system. Approaches (ii) and (iii) allowed a larger number of processors to be employed but the efficiency versus number of processors was found to drop at a faster rate in comparison to (i). The fourth approach both allowed a larger number of processors to be employed and was found to provide higher efficiency than the parallelization by physical domain only. This multi-level hybrid approach therefore appears to offer an effective parallelization method for implementation of the MIT forward model on SMP clusters.

Optimal broadcast for fully connected processor-node networks

We develop and implement an optimal broadcast algorithm for fully connected processor networks under a bidirectional communication model in which each processor can simultaneously send a message to one processor and receive a message from another, possibly different processor. For any number of processors p the algorithm requires N − 1 + ⌈ log p ⌉ communication rounds to broadcast N blocks of data from a root processor to the remaining processors, meeting the lower bound in the model. For data of size m , assuming that sending and receiving data of size m ′ takes time α + β m ′ , the best running time that can be achieved by the division of m into equal-sized blocks is ( ( ⌈ log p ⌉ − 1 ) α + β m ) 2 . The algorithm uses a regular, circulant graph communication pattern, and degenerates into a binomial tree broadcast when the number of blocks to be broadcast is one. The algorithm is furthermore well suited to fully connected clusters of SMP ( Symmetric Multi-Processor) nodes. The algorithm is implemented as part of an MPI ( Message Passing Interface) library. We demonstrate significant practical bandwidth improvements of up to a factor 1.5 over several other, commonly used broadcast algorithms on both a small SMP cluster and a 72 node NEC SX vector supercomputer.

VCluster: a thread‐based Java middleware for SMP and heterogeneous clusters with thread migration support

AbstractClusters, composed of symmetric multiprocessor (SMP) machines and heterogeneous machines, have become increasingly popular for high‐performance computing. Message‐passing libraries, such as message‐passing interface (MPI) and parallel virtual machine (PVM), are de facto parallel programming libraries for clusters that usually consist of homogeneous and uni‐processor machines. For SMP machines, MPI is combined with multithreading libraries like POSIX Thread and OpenMP to take advantage of the architecture. In addition to existing parallel programming libraries that are in C/C++ and FORTRAN programming languages, the Java programming language presents itself as another alternative with its object‐oriented framework, platform neutral byte code, and ever‐increasing performance. This paper presents a new parallel programming model and a library, VCluster, which implements this model. VCluster is based on migrating virtual threads instead of processes to support clusters of SMP machines more efficiently. The implementation uses thread migration, which can be used in dynamic load balancing. VCluster was developed in pure Java, utilizing the portability of Java to support clusters of heterogeneous machines. Several applications are developed to illustrate the use of this library and compare the usability and performance of VCluster with other approaches. Copyright © 2007 John Wiley & Sons, Ltd.

Stabilizing large‐scale generalized systems on parallel computers using multithreading and message‐passing

AbstractWe discuss the parallelization of an efficient algorithm for the partial stabilization of large‐scale linear control systems in generalized state‐space form. The algorithm is composed of highly parallel iterative schemes that appear in the computation of certain matrix functions. Here we evaluate different approaches to exploit parallelism at two levels, based on threads and processes. Our experimental results on a cluster of symmetric multiprocessors and a CC‐NUMA platform show that the efficiency of the matrix operations underlying the iterative schemes carry over to the parallel implementation of the stabilization algorithm. Copyright © 2006 John Wiley & Sons, Ltd.

High performance noise reduction for biomedical multidimensional data

Anisotropic nonlinear diffusion (AND) is one of the most powerful noise reduction techniques in the field of image processing. This method is based on a partial differential equation (PDE) tightly coupled with a massive set of eigensystems. Denoising large 3D images in biomedicine and structural cellular biology by AND is extremely expensive from a computational point of view, with huge memory needs. In this work, high performance computing techniques have been applied to AND. An strategy for optimal memory usage has been designed, which has allowed a remarkable reduction of the memory requirements. Parallel implementations of AND have been developed with special focus on clusters of symmetric multiprocessors (SMPs), currently a dominant platform in high performance computing. Different programming models have been used for the parallelization of AND: (1) Message-passing paradigm using MPI and (2) a hybrid paradigm that uses message passing among nodes plus the shared address space paradigm between the processors within the nodes. The parallel approaches have been evaluated on a cluster of dual-Xeon nodes, a representative example of clusters of SMPs. The conclusion drawn is that the hybrid approach is the most suitable for the parallelization of AND for this kind of computing platforms.

OpenMP extension to SMP clusters

This article discusses the approaches to apply the OpenMP programming model to symmetric multiprocessor (SMP) clusters using software distributed shared memory (SDSM). The major focus of this article is on the challenges that the prior studies faced and on their solution techniques. Exploiting message-passing primitives explicitly for the openMP synchronization and work-sharing directives enables light interprocess synchronizations. The studies on loop scheduling for SMP clusters will promise significant improvement in system performance. Finally, OpenMP is considered as promising programming model

Task Pool Teams: a hybrid programming environment for irregular algorithms on SMP clusters

Clusters of symmetric multiprocessors (SMPs) are popular platforms for parallel programming since they provide large computational power for a reasonable price. For irregular application programs with dynamically changing computation and data access behavior, a flexible programming model is needed to achieve efficiency. In this paper we propose Task Pool Teams as a hybrid parallel programming environment to realize irregular algorithms on clusters of SMPs. Task Pool Teams combine task pools on single cluster nodes by an explicit message passing layer. They offer load balance together with multi-threaded, asynchronous communication. Appropriate communication protocols and task pool implementations are provided and accessible by an easy-to-use application programmer interface. As application examples we present a branch and bound algorithm and the hierarchical radiosity algorithm. Copyright © 2006 John Wiley & Sons, Ltd.

Parallel distributed numerical simulations in aeronautic applications

The numerical simulation plays a key role in industrial design because it enables to reduce the time and the cost to develop new products. Because of the international competition, it is important to have a complete chain of simulation tools to perform efficiently some virtual prototyping. In this paper, we describe two components of large aeronautic numerical simulation chains that are extremely consuming of computer resource. The first is involved in computational fluid dynamics for aerodynamic studies. The second is used to study the wave propagation phenomena and is involved in acoustics. Because those softwares are used to analyze large and complex case studies in a limited amount of time, they are implemented on parallel distributed computers. We describe the physical problems addressed by these codes, the main characteristics of their implementation. For the sake of re-usability and interoperability, these softwares are developed using object-oriented technologies. We illustrate their parallel performance on clusters of symmetric multi-processors. Finally, we discuss some challenges for the future generations of parallel distributed numerical software that will have to enable the simulation of multi-physic phenomena in the context of virtual organizations also known as the extended enterprise.

Simulation tools to study a distributed shared memory for clusters of symmetric multiprocessors

Distributed shared memory (DSM) systems have become popular as a means of utilizing clusters of computers for solving large applications. We have developed a high-performance DSM, Strings. In addition, to improve the performance of our DSM, a memory hierarchy simulator has been developed that allows us to compare various techniques very quickly and with much less effort. This paper describes our simulator, DSM Sim. We show that the simulator's performance closely matches the real system and demonstrate potential performance gains of up to 60% after adding optimization features to the simulator. The simulator also accepts the same code as the software distributed shared memory.

Distributed Shared Arrays: An Integration of Message Passing and Multithreading on SMP Clusters

This paper presents a Distributed Shared Array runtime system to support Java-compliant multithreaded programming on clusters of symmetric multiprocessors (SMPs). As a hybrid of message passing and shared address space programming models, the DSA programming model allows programmers to explicitly control data distribution so as to take advantage of the deep memory hierarchy, while relieving them from error-prone orchestration of communication and synchronization at run-time. The DSA system is developed as an integral component of mobility support middleware for grid computing so that DSA-based virtual machines can be reconfigured to adapt to the varying resource supplies or demand over the course of a computation. The DSA runtime system also features a directory-based cache coherence protocol in support of replication of user-defined sharing granularity and a communication proxy mechanism for reducing network contention. We demonstrate the programmability of the model in a number of parallel applications and evaluate its performance on a cluster of SMP servers, in particular, the impact of the coherence granularity.

Memory management for multi-threaded software DSM systems

When software distributed shared memory (SDSM) systems provide multithreading to exploit cluster of symmetric multiprocessors (SMPs), a challenge is how to preserve memory consistency in a thread-safe way, which is known as “atomic page update problem”. In this paper, we show that this problem can be solved by creating two independent access paths to a physical page and by assigning different access permissions to them. Especially, we propose three new methods using System V shared memory inter-process communication (IPC), a new mdup() system call, and a fork() system call in addition to a known method using file mapping. The main contribution of this paper is to introduce various solutions to the atomic page update problem and to compare their characteristics extensively. Experiments carried out on a Linux-based cluster of SMPs and an IBM SP Night Hawk system show that the proposed methods overcome the drawbacks of the file mapping method such as high initialization cost and buffer cache flushing overhead. In particular, the method using a fork() system call preserves the whole address space to the application.

Implementation of the EARTH programming model on SMP clusters: a multi‐threaded language and runtime system

AbstractThis paper describes the design and implementation of an Efficient Architecture for Running THreads (EARTH) runtime system for a multi‐processor/multi‐node cluster. The (EARTH) model was designed to support the efficient execution of parallel (multi‐threaded) programs with irregular fine‐grain parallelism using off‐the‐shelf computers. Implementing an EARTH runtime system requires an explicitly threaded runtime system. For portability, we built this runtime system on top of Pthreads under Linux and used sockets for inter‐node communication. Moreover, in order to make the best use of the resources available on a cluster of symmetric multi‐processors (SMP), this implementation enables the overlapping of communication and computation.We used Threaded‐C, a language designed to implement the programming model supported by the EARTH architecture. This language allows the expression of various levels of parallelism and provides the primitives needed to manage the required communication and synchronization. The Threaded‐C programming language supports irregular fine‐grain parallelism through a two‐level hierarchy of threads and fibers. It also provides various synchronization and communication constructs that reflect the nature of EARTH's fibers—non‐preemptive execution with data‐driven scheduling—as well as the extensive use of split‐phase transactions on EARTH to execute long‐latency operations. Copyright © 2003 John Wiley & Sons, Ltd.

A Performance-Prediction Model for PIC Applications on Clusters of Symmetric MultiProcessors: Validation with Hierarchical HPF+OpenMP Implementation

A performance-prediction model is presented, which describes different hierarchical workload decomposition strategies for particle in cell (PIC) codes on Clusters of Symmetric MultiProcessors. The devised workload decomposition is hierarchically structured: a higher-level decomposition among the computational nodes, and a lower-level one among the processors of each computational node. Several decomposition strategies are evaluated by means of the prediction model, with respect to the memory occupancy, the parallelization efficiency and the required programming effort. Such strategies have been implemented by integrating the high-level languages High Performance Fortran (at the inter-node stage) and OpenMP (at the intra-node one). The details of these implementations are presented, and the experimental values of parallelization efficiency are compared with the predicted results.

Implementation of 3D lattice Monte Carlo simulation on a cluster of symmetric multiprocessors

This paper presents a new approach to parallelize 3D lattice Monte Carlo algorithms used in the numerical simulation of polymer on ZiQiang 2000-a cluster of symmetric multiprocessors (SMPs). The combined load for cell and energy calculations over the time step is balanced together to form a single spatial decomposition. Basic aspects and strategies of running Monte Carlo calculations on parallel computers are studied. Different steps involved in porting the software on a parallel architecture based on ZiQiang 2000 running under Linux and MPI are described briefly. It is found that parallelization becomes more advantageous when either the lattice is very large or the model contains many cells and chains.

Combining Shared and Distributed Memory Programming Models on Clusters of Symmetric Multiprocessors: Some Basic Promising Experiments

This note presents some experiments on different clusters of SMPs, where both distributed and shared memory parallel programming paradigms can be naturally combined. Although the platforms exhibit the same macroscopic memory organization, it appears that their individual overall performance is closely dependent on the ability of their hardware to efficiently exploit the local shared memory within the nodes. In that context, cache blocking strategy appears to be very important not only to get good performance out of each individual processor but mainly good performance out of the overall computing node since sharing memory locally might become a severe bottleneck. On a very simple benchmark, representative of many large simulation codes, we show through numerical experiments that mixing the two programming models enables us to get attractive speed-ups that compete with a pure distributed memory approach. This opens promising perspectives for smoothly moving large industrial codes developed on distributed vector computers with a moderate number of processors on these emerging platforms for intensive scientific computing that are the clusters of SMPs.

A Hybrid Mpi-Openmp Implementation of an Implicit Finite-Element Code on Parallel Architectures

Summary The hybrid MPI-OpenMP model is a natural parallel programming paradigm for emerging parallel architectures that are based on symmetric multiprocessor (SMP) clusters. This paper presents a hybrid implementation adapted for an implicit finite-element code developed for groundwater transport simulations. The original code was parallelized for distributed memory architectures using MPI (Message Passing Interface) using a domain decomposition strategy. OpenMP directives were then added to the code (a straightforward loop-level implementation) to use multiple threads within each MPI process. To improve the OpenMP performance, several loop modifications were adopted. The parallel performance results are compared for four modern parallel architectures. The results show that for most of the cases tested, the pure MPI approach outperforms the hybrid model. The exceptions to this observation were mainly due to a limitation in the MPI library implementation on one of the architectures. A general conclusion is that while the hybrid model is a promising approach for SMP cluster architectures, at the time of this writing, the payoff may not be justified for converting all existing MPI codes to hybrid codes. However, improvements in OpenMP compilers combined with potential MPI limitations in SMP nodes may make the hybrid approach more attractive for a broader set of applications in the future.