Parallel Scientific Computing Research Articles

Teaching Practice about Parallel Scientific Computing for Graduate Students Majored in Physics

With the rapid developments of computer hardware and software, nowadays parallel computing is playing more and more important roles in the scientific research in physics related fields. The broad use of supercomputers and computer clusters makes it possible to solve much more efficiently various scientific problems by computations and simulations. Therefore, to gain basic knowledge and programming skills of parallel computing is very necessary for junior graduate students majored in physics, while the teaching of these knowledge and skills has been lacking in the traditional course of computational physics. Here, we present a teaching practice of parallel computing for junior graduate students in physics, which has been shown to be successful by the homework presentation of the students

Chapel: a versatile language for teaching parallel programming

Chapel is a new programming language geared towards parallel programming and scientific computing. The syntax borrows heavily from Python, Java, and C/C++, but introduces keywords to ease creation ...

Scientific computations on multi-core systems using different programming frameworks

Numerical linear algebra is one of the most important forms of scientific computation. The basic computations in numerical linear algebra are matrix computations and linear systems solution. These computations are used as kernels in many computational problems. This study demonstrates the parallelisation of these scientific computations using multi-core programming frameworks. Specifically, the frameworks examined here are Pthreads, OpenMP, Intel Cilk Plus, Intel TBB, SWARM, and FastFlow. A unified and exploratory performance evaluation and a qualitative study of these frameworks are also presented for parallel scientific computations with several parameters. The OpenMP and SWARM models produce good results running in parallel with compiler optimisation when implementing matrix operations at large and medium scales, whereas the remaining models do not perform as well for some matrix operations. The qualitative results show that the OpenMP, Cilk Plus, TBB, and SWARM frameworks require minimal programming effort, whereas the other models require advanced programming skills and experience. Finally, based on an extended study, general conclusions regarding the programming models and matrix operations for some parameters were obtained.

A Hybrid Hypergraph Partitioning Algorithm for Scientific Computing

Hypergraph partitioning is an increasingly important and widely studied research topic in parallel scientific computing. In this paper, we present a multiway hypergraph partitioning algorithm, mixed simulated annealing algorithm for global optimization and tabu search algorithm for local optimization. Experiments on the benchmark suite of several unstructured meshes show that, for 2-, 4-, 8-, 16-and 32-way partitioning, the quality of partition produced by our algorithm are on the average 6% and the maximum 17% better than those produced by partitioning software hMETIS in term of the cutsize metric.

Parallel algebraic domain decomposition solver for the solution of augmented systems

We consider the parallel iterative solution of indefinite linear systems given as augmented systems. Our numerical technique is based on an algebraic non-overlapping domain decomposition technique that only exploits the graph of the sparse matrix. This approach to high-performance, scalable solution of large sparse linear systems in parallel scientific computing, is to combine direct and iterative methods. We report numerical and parallel performance of the scheme on large matrices arising from the finite element discretization of linear elasticity in structural mechanics problems.

Load-balancing spatially located computations using rectangular partitions

Distributing spatially located heterogeneous workloads is an important problem in parallel scientific computing. We investigate the problem of partitioning such workloads (represented as a matrix of non-negative integers) into rectangles, such that the load of the most loaded rectangle (processor) is minimized. Since finding the optimal arbitrary rectangle-based partition is an NP-hard problem, we investigate particular classes of solutions: rectilinear, jagged and hierarchical. We present a new class of solutions called m-way jagged partitions, propose new optimal algorithms for m-way jagged partitions and hierarchical partitions, propose new heuristic algorithms, and provide worst case performance analyses for some existing and new heuristics. Moreover, the algorithms are tested in simulation on a wide set of instances. Results show that two of the algorithms we introduce lead to a much better load balance than the state-of-the-art algorithms. We also show how to design a two-phase algorithm that reaches different time/quality tradeoffs.

Design and Implementation of a High Productivity Language with Communication Aggregation for Parallel Scientific Computation

Design and Implementation of a High Productivity Language with Communication Aggregation for Parallel Scientific Computation

Design and Implementation of a High Productivity Language with Communication Aggregation for Parallel Scientific Computation

Design and Implementation of a High Productivity Language with Communication Aggregation for Parallel Scientific Computation

Overcoming Communication Latency Barriers in Massively Parallel Scientific Computation

Anton, a massively parallel special-purpose machine that accelerates molecular dynamics simulations by orders of magnitude, uses a combination of specialized hardware mechanisms and restructured software algorithms to reduce and hide communication latency. Anton delivers end-to-end internode latency significantly lower than any other large-scale parallel machine, and its critical-path communication time for molecular dynamics simulations is less than 3 percent that of the next-fastest platform.

Multi-GPU accelerated multi-spin Monte Carlo simulations of the 2D Ising model

A Modern Graphics Processing unit (GPU) is able to perform massively parallel scientific computations at low cost. We extend our implementation of the checkerboard algorithm for the two-dimensional Ising model [T. Preis et al., Journal of Chemical Physics 228 (2009) 4468–4477] in order to overcome the memory limitations of a single GPU which enables us to simulate significantly larger systems. Using multi-spin coding techniques, we are able to accelerate simulations on a single GPU by factors up to 35 compared to an optimized single Central Processor Unit (CPU) core implementation which employs multi-spin coding. By combining the Compute Unified Device Architecture (CUDA) with the Message Parsing Interface (MPI) on the CPU level, a single Ising lattice can be updated by a cluster of GPUs in parallel. For large systems, the computation time scales nearly linearly with the number of GPUs used. As proof of concept we reproduce the critical temperature of the 2D Ising model using finite size scaling techniques.

The BPS preconditioner on Beowulf cluster

This work presents the implementation on a Linux Cluster of a parallel preconditioner for the solution of the linear system resulting from the finite element discretization of a 2D second order elliptic boundary value problem. The numerical method, proposed by Bramble, Pasciak and Schatz, is developed using Domain Decomposition techniques, which are based on the splitting of the computational domain into subregionsof smaller size, enforcing suitable compatibility conditions. The Fortran code is implemented using PETSc: a suite of data structures and routines devoted to the scientific parallel computing and based on the MPI standard for all message-passing communications. The main interest of the paper is to present an efficient and portable code for the solution of large-scale linear systems and to investigate how the architecturalaspects of the cluster influence the performance of the considered algorithm. We provide an analysis of the execution times as well as of the scalability, using as test case the classical Poisson equation with Dirichlet boundary conditions.

PyTrilinos

PyTrilinos is a collection of Python modules that are useful for serial and parallel scientific computing. This collection contains modules that cover serial and parallel dense linear algebra, serial and parallel sparse linear algebra, direct and iterative linear solution techniques, domain decomposition and multilevel preconditioners, nonlinear solvers, and continuation algorithms. Also included are a variety of related utility functions and classes, including distributed I/O, coloring algorithms, and matrix generation. PyTrilinos vector objects are integrated with the popular NumPy Python module, gathering together a variety of high-level distributed computing operations with serial vector operations. PyTrilinos is a set of interfaces to existing, compiled libraries. This hybrid framework uses Python as front-end, and efficient precompiled libraries for all computationally expensive tasks. Thus, we take advantage of both the flexibility and ease of use of Python, and the efficiency of the underlying C++, C, and FORTRAN numerical kernels. Out numerical results show that, for many important problem classes, the overhead required by the Python interpreter is negligible. To run in parallel, PyTrilinos simply requires a standard Python interpreter. The fundamental MPI calls are encapsulated under an abstract layer that manages all interprocessor communications. This makes serial and parallel scripts using PyTrilinos virtually identical.

Parallel Computing Algorithms and Applications

The author reviews a book by L.R. Scott, T. Clark, and B. Bagheri, called Scientific Parallel Computing, Princeton Univ. Press, 2005, ISBN: 978-0691119359, 416 pages.

Effectively utilizing global cluster memory for large data-intensive parallel programs

Large scientific parallel applications demand large amounts of memory space. Current parallel computing platforms schedule jobs without fully knowing their memory requirements. This leads to uneven memory allocation in which some nodes are overloaded. This, in turn, leads to disk paging, which is extremely expensive in the context of scientific parallel computing. To solve this problem, we propose a new peer-to-peer solution called parallel network RAM. This approach avoids the use of disk, better utilizes available RAM resources, and will allow larger problems to be solved while reducing the computational, communication, and synchronization overhead typically involved in parallel applications. We proposed several different parallel network RAM designs and evaluated the performance of each under different conditions. We discovered that different designs are appropriate in different situations.

Resource-Aware Scientific Computation on a Heterogeneous Cluster

Although researchers can develop software on small, local clusters and move it later to larger clusters and supercomputers, the software must run efficiently in both environments. Two efforts aim to improve the efficiency of scientific computation on clusters through resource-aware dynamic load balancing. The popularity of cost-effective clusters built from commodity hardware has opened up a new platform for the execution of software originally designed for tightly coupled supercomputers. Because these clusters can be built to include any number of processors ranging from fewer than 10 to thousands, researchers in high-performance scientific computation at smaller institutions or in smaller departments can maintain local parallel computing resources to support software development and testing, then move the software to larger clusters and supercomputers. As promising as this ability is, it has also led to the need for local expertise and resources to set up and maintain these clusters. The software must execute efficiently both on smaller local clusters and on larger ones. These computing environments vary in the number of processors, speed of processing and communication resources, and size and speed of memory throughout the memory hierarchy as well as in the availability of support tools and preferred programming paradigms. Software developed and optimized using a particular computing environment might not be as efficient when it's moved to another one. In this article, we describe a small cluster along with two efforts to improve the efficiency of parallel scientific computation on that cluster. Both approaches modify the dynamic load-balancing step of an adaptive solution procedure to tailor the distribution of data across the cooperating processes. This modification helps account for the heterogeneity and hierarchy in various computing environments.

On the Performance of the Python Programming Language for Serial and Parallel Scientific Computations

This article addresses the performance of scientific applications that use the Python programming language. First, we investigate several techniques for improving the computational efficiency of serial Python codes. Then, we discuss the basic programming techniques in Python for parallelizing serial scientific applications. It is shown that an efficient implementation of the array‐related operations is essential for achieving good parallel performance, as for the serial case. Once the array‐related operations are efficiently implemented, probably using a mixed‐language implementation, good serial and parallel performance become achievable. This is confirmed by a set of numerical experiments. Python is also shown to be well suited for writing high‐level parallel programs.

Parallel scientific computation: a structured approach using BSP and MPI

1. Introduction 2. LU decomposition 3. The fast Fourier transform 4. Sparse matrix-vector multiplication A. Auxiliary BSPedupack functions B. A quick reference guide to BSPlib C. Programming in BSP style using MPI References Index

Parallel and distributed implementation of large industrial applications

Parallelization of high performance computing applications has been a field of active research for quite some time now. Most projects that have parallelized industrial software packages have focused on the specific application and did not attempt to document and generalize their lessons learned. We report on results of a project that has parallelized a state of the art industrial computational fluid dynamics (CFD) packages and that explicitly aimed at establishing software engineering guidelines for future similar projects. Thanks to the consequent application of the software engineering guidelines defined for the project, the parallel CFD code has proven excellent efficiency and scalability on a large number of parallel hardware platforms. The project also addressed software engineering issues such as object orientation and resource management. The CFD package has been redesigned as an object oriented program and implemented in C++ and Java. The object oriented CFD program has shown reasonable efficiency in preliminary benchmark tests. We expect that further optimizations to the OO code and advances in compiler technology will make the performance gap (relative to the fortran version) almost disappear in the near future. A resource manager has been developed that allows production runs of parallel scientific computing software to execute in batch mode on networks of workstations by dynamically allocating resources to parallel batch jobs that currently are not claimed by interactive users. The resource manager has been successfully tested with the parallel CFD code as workload.

Titanium: a high-performance Java dialect

Titanium is a language and system for high-performance parallel scientific computing. Titanium uses Java as its base, thereby leveraging the advantages of that language and allowing us to focus attention on parallel computing issues. The main additions to Java are immutable classes, multidimensional arrays, an explicitly parallel SPMD model of computation with a global address space, and zone-based memory management. We discuss these features and our design approach, and report progress on the development of Titanium, including our current driving application: a three-dimensional adaptive mesh refinement parallel Poisson solver. © 1998 John Wiley & Sons, Ltd.

Application specification and software reuse in parallel scientific computing

The authors have developed a new approach to support simple software design based on reusable software components. They demonstrate their work through prototype problem solving environments for developing stencil- and numerical linear algebra based applications.