Single Program Multiple Data Research Articles

Armadillo generation distributed system with geranium Cadcam cluster for solving 2D telegraphic problem

In this paper, the parallel implementation of the iterative alternating direction explicit method by D'Yakonov (IADE-DY) and the Mitchell and Fairweather double sweep method (DS-MF) compared with the alternative direction implicit (ADI) method for solving the 2D telegraphic problem on a distributed system of Armadillo cluster and geranium Cadcam cluster using the parallel virtual machine and message passing interface is presented. We implement the scheduling of n tridiagonal system of equations with the above methods to show improvement on speedup and efficiency with parallel strategies across the two platforms. The single program multiple data model was employed for the implementation. The IADE-DY, DS-MF and ADI are developed by the splitting of the implicit equation using the finite difference discretization on the 2D telegraphic problem. The implementation is discussed in relation to means of the performance parallel strategies and analysis. The model enhances overlap communication and computation to avoid unnecessary synchronization, hence, the method yields significant speedup by the use of the non-blocking communication. We present some analyses that are helpful for speedup and efficiency.

Collective Asynchronous Remote Invocation (CARI): A High-Level and Effcient Communication API for Irregular Applications

The Message Passing Interface (MPI) standard continues to dominate the landscape of parallel computing as the de facto API for writing large-scale scientific applications. But the critics argue that it is a low-level API and harder to practice than shared memory approaches. This paper addresses the issue of programming productivity by proposing a high-level, easy-to-use, and effcient programming API that hides and segregates complex low-level message passing code from the application specific code. Our proposed API is inspired by communication patterns found in Gadget-2, which is an MPI-based parallel production code for cosmological N-body and hydrodynamic simulations. In this paper—we analyze Gadget-2 with a view to understanding what high-level Single Program Multiple Data (SPMD) communication abstractions might be developed to replace the intricate use of MPI in such an irregular application—and do so without compromising the effciency. Our analysis revealed that the use of low-level MPI primitives—bundled with the computation code—makes Gadget-2 diffcult to understand and probably hard to maintain. In addition, we found out that the original Gadget-2 code contains a small handful of—complex and recurring—patterns of message passing. We also noted that these complex patterns can be reorganized into a higherlevel communication library with some modifications to the Gadget-2 code. We present the implementation and evaluation of one such message passing pattern (or schedule) that we term Collective Asynchronous Remote Invocation (CARI). As the name suggests, CARI is a collective variant of Remote Method Invocation (RMI), which is an attractive, high-level, and established paradigm in distributed systems programming. The CARI API might be implemented in several ways—we develop and evaluate two versions of this API on a compute cluster. The performance evaluation reveals that CARI versions of the Gadget-2 code perform as well as the original Gadget-2 code but the level of abstraction is raised considerably.

Parallelization of a Lagrangian–Eulerian DEM/CFD code for application to fluidized beds

The coupled Lagrangian–Eulerian DEM/CFD code developed over several years at Aston University and the University of Birmingham has been parallelized using a single program multiple data (SPMD) strategy. Initial implementation on a high-performance compute cluster indicated good scalability up to 32 processors, beyond which speedup gains are swamped by the all-processor communication overhead inherent in the fluid flow algorithm. A Geldart group A powder bed comprising 1million particles was fluidized in the bubbling regime using the parallelized code.

Parallel Implementation of 2-D Telegraphic Equation on MPI/PVM Cluster

In this paper, a parallel implementation of the Iterative Alternating Direction Explicit method by D’Yakonov (IADE-DY) to solve 2-D telegraphic problem on a distributed system using Message Passing Interface (MPI) and Parallel Virtue Machine (PVM) are presented. The parallelization of the program is implemented by a domain decomposition strategy. A Single Program Multiple Data (SPMD) model is employed for the implementation. The implementation is discussed in relation to means of the parallel performance strategies and analysis. The model enhances overlap communication and computation to avoid unnecessary synchronization, hence, the method yields significant speedup. The level of speedup observed from tables as the mesh increases are in the range of 5–10%. Improvement has been achieved by numbers of tables and figures in our experiment. We present some analyses that are helpful for speedup and efficiency. It is concluded that the efficiency is strongly dependent on the grid size, block numbers and the number of processors for both MPI and PVM. Different strategies to improve the computational efficiency are proposed.

Method of communication flow dependence analysis for shared-memory SPMD program

The traditional analysis methods of data flow dependence cannot recognize the processing nodes to which the statements for accessing shared data should belong and also cannot avoid the impact caused by the control flow whose sequence is non-deterministic while dealing with shared memory Single Program Multiple Data(SPMD)program,thus generating the results with a lower accuracy when they analyze shared data dependence.A scalable analysis method of shared data communication-flow dependence was presented according to the alias feature of shared data in shared-memory SPMD program,which was applied in a prototype of back-end analyzer.The experimental results show that the new method can find more accurate shared data communication-flow dependence than the traditional method.

Modeling and Evaluating Non-shared Memory CELL/BE Type Multi-core Architectures for Local Image and Video Processing

Local processing, which is a dominant type of processing in image and video applications, requires a huge computational power to be performed in real-time. However, processing locality, in space and/or in time, allows to exploit data parallelism and data reusing. Although it is possible to exploit these properties to achieve high performance image and video processing in multi-core processors, it is necessary to develop suitable models and parallel algorithms, in particular for non-shared memory architectures. This paper proposes an efficient and simple model for local image and video processing on non-shared memory multi-core architectures. This model adopts a single program multiple data approach, where data is distributed, processed and reused in an optimal way, regarding the data size, the number of cores and the local memory capacity. The model was experimentally evaluated by developing video local processing algorithms and programming the Cell Broadband Engine multi-core processor, namely for advanced video motion estimation and in-loop deblocking filtering. Furthermore, based on these experiences it is also addressed the main challenges of vectorization, and the reduction of branch mispredictions and computational load imbalances. The limits and advantages of the regular and adaptive algorithms are also discussed. Experimental results show the adequacy of the proposed model to perform local video processing, and that real-time is achieved even to process the most demanding parts of advanced video coding. Full-pixel motion estimation is performed over high resolution video (720×576 pixels) at a rate of 30 frames per second, by considering large search areas and five reference frames.

Communication Set Generation for a Special Case of Irregular Parallel Applications

Irregular computing significantly influences the performance of large scale parallel applications.How to generate local memory access sequence and communication set efficiently for irregular array reference is an important issue in compiling a data-parallel language into a single program multiple data(SPMD) code for distributed-memory machines.By far,many researches have focused on the problem of communication set generation under regular array references in parallel loops.However,little researches give attentions to generating communication set for irregular array accesses in loop nests.In general case of irregular accesses,Inspector/Executor model is adopted to scan over array elements at Inspector phase of run time such that communication set can be constructed.This paper proposes an approach to derive an algebraic solution of communication set enumeration at compile time for the situation of irregular array references in nested loops.And this paper introduces integer lattice into alignment and cyclic(k)-distribution for global-to-local and local-to-global index translations.Then it also presents the algorithm for the corresponding SPMD code generation.When the parameters of alignments and cyclic(k) and array references are known,the SPMD code can then be completely derived at compile time such that no inspector-like run-time code will be needed to construct enumeration of the communication set.

Using Graphics Processors for High-Performance Computation and Visualization of Plasma Turbulence

Direct numerical simulation (DNS) of turbulence is computationally intensive and typically relies on some form of parallel processing. The authors present techniques to map DNS computations to modern graphics processing units (GPUs), which are characterized by very high memory bandwidth and hundreds of SPMD (single-program-multiple-data) processors.

Parallel three‐dimensional simulation of the injection molding process

AbstractThis paper describes the development of a parallel three‐dimensional unstructured non‐isothermal flow solver for the simulation of the injection molding process. The numerical model accounts for multiphase flow in which the melt and air regions are considered to be a continuous incompressible fluid with distinct physical properties. This aspect avoids the complex reconstruction of the interface. A collocated finite volume method is employed, which can switch between first‐ and second‐order accuracy in both space and time. The pressure implicit with splitting of operators algorithm is used to compute the transient flow variables and couple velocity and pressure. The temperature equation is solved using a transport equation with convection and diffusion terms. An upwind differencing scheme is used for the discretization of the convection term to enforce a bounded solution. In order to capture the sharp interface, a bounded compressive high‐resolution scheme is employed. Parallelization of the code is achieved using the PETSc framework and a single program multiple data message passing model. Predicted numerical solutions for several example problems are considered. The first case validates the solution algorithm for moderate Reynolds number flows using a structured mesh. The second case employs an unstructured hybrid mesh showing the capability of the solver to describe highly viscous flows closer to realistic injection molding conditions. The final case presents the non‐isothermal filling of a thick cavity using three mesh sizes and up to 80 processors to assess parallel performance. The proposed algorithm is shown to have good accuracy and scalability. Copyright © 2008 John Wiley & Sons, Ltd.

SU‐GG‐I‐101: A Parallel Implementation for Fast Deformable Image Registration of Large Data Sets

Purpose: One major obstacle for using deformable image registration for online or near real‐time applications is its high computational cost. The goal of this study is to implement a practical parallel computing technique for image intensity‐based deformable registration of relatively large data sets. Method and Materials: Taking advantage of the locality characteristic of the deformed images, deformable image registration can be parallelized by dividing the entire volume into smaller sub‐volumes. We implemented a parallel solution with the single‐program‐multiple‐data (SPMD) programming model in a cluster. The cluster is composed of one lead computer and 12 nodes. To ensure continuity at boundaries of sub‐volumes, we added an overlapping volume, which became a computational overhead. We proposed two techniques to reduce the overlapping volume by applying pre‐registration during the coarse levels of a multi‐resolution approach, and by handling the boundary condition carefully when applying the smoothing filter during registration. The baseline data for comparison was the deformation result from one CPU. Results were compared in three difficult cases with large deformation and large data set. The scalability was also analyzed. Results: 8‐1 nodes appears to be enough to handle each case. Four overlapping slices combined with pre‐registration at coarse level were able to get the correct deformation field at the boundary of each sub‐volumes even in the presence of relatively large deformations. Computing times were 20.5s, 44.4s, and 76.5s for a prostate case (matrix size: 294*237*62), a head & neck case (361*414*87), and a breast case (465*371*157), respectively. The same data sets required 54.5s, 150.2s, and 331.6s to compute on a single CPU, respectively. Conclusion: We have implemented an effective parallel solution for deformable image registration which can handle large data sets in a short time.

Using cellular automata for parallel simulation of laser dynamics with dynamic load balancing

We present an analysis of the feasibility of executing a parallel bioinspired model of laser dynamics, based on cellular automata (CA), on the usual target platform of this kind of applications: a heterogeneous non-dedicated cluster. As this model employs a synchronous CA, using the single program, multiple data (SPMD) paradigm, it is not clear in advance if an appropriate efficiency can be obtained on this kind of platform. We have evaluated its performance including artificial load to simulate other tasks or jobs submitted by other users. A dynamic load balancing strategy with two main differences from most previous implementations of CA based models has been used. First, it is possible to migrate load to cluster nodes initially not belonging to the pool. Second, a modular approach is taken in which the model is executed on top of a dynamic load balancing tool – the Dynamite system – gaining flexibility. Very satisfactory results have been obtained, with performance increases from 60% to 80%.

Parallel Languages and Compilers: Perspective From the Titanium Experience

We describe the rationale behind the design of key features of Titanium—an explicitly parallel dialect of Java for high-performance scientific programming—and our experiences in building applications with the language. Specifically, we address Titanium's partitioned global address space model, single program multiple data parallelism support, multi-dimensional arrays and array-index calculus, memory management, immutable classes (class-like types that are value types rather than reference types), operator overloading, and generic programming. We provide an overview of the Titanium compiler implementation, covering various parallel analyses and optimizations, Titanium runtime technology and the GASNet network communication layer. We summarize results and lessons learned from implementing the NAS parallel benchmarks, elliptic and hyperbolic solvers using adaptive mesh refinement, and several applications of the immersed boundary method.

Development of parallel density functional program using distributed matrix to calculate all‐electron canonical wavefunction of large molecules

We developed a new parallel density-functional canonical molecular-orbital program for large molecules based on the resolution of the identity method. In this study, all huge matrices were decomposed and saved to the distributed local memory. The routines of the analytical molecular integrals and numerical integrals of the exchange-correlation terms were parallelized using the single program multiple data method. A conventional linear algebra matrix library, ScaLAPACK, was used for matrix operations, such as diagonalization, multiplication, and inversion. Anderson's mixing method was adopted to accelerate the self-consistent field (SCF) convergence. Using this program, we calculated the canonical wavefunctions of a 306-residue protein, insulin hexamer (26,790 orbitals), and a 133-residue protein, interleukin (11,909 orbitals) by the direct-SCF method. In regard to insulin hexamer, the total parallelization efficiency of the first SCF iteration was estimated to be 82% using 64 Itanium 2 processors connected at 3.2 GB/s (SGI Altix3700), and the calculation successfully converged at the 17-th SCF iteration. By adopting the update method, the computational time of the first and the final SCF loops was 229 min and 156 min, respectively. The whole computational time including the calculation before the SCF loop was 2 days and 17 h. This study put the calculations of the canonical wavefunction of 30,000 orbitals to practical use.

Strategies for Parallel Genetic Computation of Optimization Problems

ABSTRACTThis paper explores the efficiency of various strategies for parallel genetic computation of optimization problems on multicomputer platforms. The strategies for designing parallel genetic algorithms on multicomputer platforms are investigated considering the correlations of the algorithmic and the architectural spaces. Two parallel genetic computational models are considered based on the manager/workers and Single Program Multiple Data parallel paradigms. Parallelism profiling and analysis of parallel system performance have been made for the different parallel computational models in respect to the scalability of the application and the scalability of the parallel machine size.

분산 환경에서 CFD 분석 프로그램 수행을 위한 그리드 시스템 META 설계 및 구현

본 논문에서는 분산 환경 상에서 CFD(Computational Fluid Dynamics) 분석 프로그램을 편리하게 수행할 수 있도록 하는 그리드 시스템 META(Metacomputing Environment using Test-un of Application)의 설계 및 구현에 관하여 기술한다. 그리드 시스템 META는 CFD 프로그램 개발자들이 네트워크에 분산된 계산 자원들을 단일 시스템처럼 사용할 수 있도록 한다. 그리드 컴퓨팅과 관련하여 연구주제로는 고장허용, 자원 선택, 사용자 인터페이스 설계 등이 있다. 본 논문에서는 MPI(Message Passing Interface)로 작성된 SPMD(Single Program, Multiple Data) 구조의 병렬프로그램을 실행시키기 위한 자동 자원 선택방법을 활용하였다. 본 논문에서 제안한 자원 관리기법은 네트워크상의 전송지연 시간과 시험수행을 통해 얻어진 핵심루프의 경과시간을 이용한다. 전송지연시간은 병렬 프로그램이 복수의 시스템에 분산되어 수행될 때 수행 성능에 큰 영향을 주는 요인이다. CFD 프로그램들의 공통적인 특성 때문에 핵심루프 경과시간은 전체 수행시간을 예측할 수 있는 지표가 된다. 핵심루프는 CFD 프로그램의 전체 수행시간 중 90% 이상을 차지한다. This paper describes the design and implementation of a grid system META (Metacomputing Environment using Test-run of Application) which facilitates the execution of a CFD (Computational Fluid Dynamics) analysis program on distributed environment. The grid system META allows the CFD program developers can access the computing resources distributed over the network just like one computer system. The research issues involved in the grid computing include fault-tolerance, computing resource selection, and user-interface design. In this paper, we exploits an automatic resource selection scheme for executing the parallel SPMD (Single Program Multiple Data) application written in MPI (Message Passing Interface). The proposed resource selection scheme is informed from the network latency time and the elapsed time of the kernel loop attained from test-run. The network latency time highly influences the executional performance when a parallel program is distributed and executed over several systems. The elapsed time of the kernel loop can be used as an estimator of the whole execution time of the CFD Program due to a common characteristic of CFD programs. The kernel loop consumes over 90% of the whole execution time of a CFD program.

An single program multiple data strategy for calculation of anharmonic vibrations

This paper presents a general perturbational and variational scheme to calculate solutions of the spectral problem for the vibrational molecular Hamiltonian. A parallel strategy in the ongoing development of our software P_Anhar is presented, in order to calculate the vibrational spectrum of medium sized molecules. The efficiency of this approach is checked on ethylene oxide.

Asynchronous Distributed Calibration for Scalable and Reconfigurable Multi-Projector Displays

Centralized techniques have been used until now when automatically calibrating (both geometrically and photometrically) large high-resolution displays created by tiling multiple projectors in a 2D array. A centralized server managed all the projectors and also the camera(s) used to calibrate the display. In this paper, we propose an asynchronous distributed calibration methodology via a display unit called the plug-and-play projector (PPP). The PPP consists of a projector, camera, computation and communication unit, thus creating a self-sufficient module that enables an asynchronous distributed architecture for multi-projector displays. We present a single-program-multiple-data (SPMD) calibration algorithm that runs on each PPP and achieves a truly scalable and reconfigurable display without any input from the user. It instruments novel capabilities like adding/removing PPPs from the display dynamically, detecting faults, and reshaping the display to a reasonable rectangular shape to react to the addition/removal/faults. To the best of our knowledge, this is the first attempt to realize a completely asynchronous and distributed calibration architecture and methodology for multi-projector displays.

Scalable Architecture for SoC Video Encoders

Evolving video coding standards demand functional flexibility for implementations, not only at design time but also after fabrication. This paper presents a System-on-Chip design approach with a feasible combination of performance, scalability, programmability, area efficiency, and design time effort for a video encoder. The encoder is based on a homogeneous master-slave processor architecture. Each slave encodes a part of the frame in the Single Program Multiple Data (SPMD) data parallel model. Both shared and distributed memory architectures are presented. Design effort is reduced by identical program codes, automated assembly of software and hardware modules independent of the number and type of processors, as well as our flexible on-chip communication network called Heterogeneous IP Block Interconnection (HIBI). A case study implementation with two to ten simple ARM7 processors, 32-bit HIBI bus and non-optimized processor-independent software gives the performance from 6 to 53 fps for QCIF. The whole encoder area ranges from 173 to 770 kgates excluding the memories. The relation scales reasonably well to systems with more powerful processors and optimized code. The optimization of the communication network shows that with more than six slaves even a serial HIBI connection with 100 MHz speed is feasible. HIBI and the parallelization approach allow exploration and optimization of the communication both at the application and architecture layers.

Case study of gate-level logic simulation on an extremely fine-grained chip multiprocessor

Explicit-multi-threading (XMT) is a parallel programming approach for exploiting on-chip parallelism. Its fine-grained single program multiple data (SPMD) programming model is suitable for many com...

Discretization and parallel performance of an unstructured finite volume Navier–Stokes solver

AbstractThe discretization, parallelization and performance of an implicit, unstructured, time‐dependent Computational Fluid Dynamics code is described. A detailed description is provided of the improvements made on second‐order accurate tools for spatial interpolation and gradient calculation to discretize the Navier–Stokes equations in an unstructured framework. The main goal in the development of the discretization tools was to ensure a scalable and accurate parallel code. The performance of the discretization tools has been validated using standard bench‐mark problems for non‐uniform, non‐orthogonal grids. Parallelization of the code is done within the PETSc framework using a single‐program‐multiple‐data (SPMD) parallelization model. The resulting parallel code is shown to scale linearly within the limit of the available number of processors. Copyright © 2006 John Wiley & Sons, Ltd.