Large Number Of Processors Research Articles

Orchestration by approximation

We present a novel 2-approximation algorithm for deploying stream graphs on multicore computers and a stream graph transformation that eliminates bottlenecks. The key technical insight is a data rate transfer model that enables the computation of a "closed form", i.e., the data rate transfer function of an actor depending on the arrival rate of the stream program. A combinatorial optimization problem uses the closed form to maximize the throughput of the stream program. Although the problem is inherently NP-hard, we present an efficient and effective 2-approximation algorithm that provides a lower bound on the quality of the solution. We introduce a transformation that uses the closed form to identify and eliminate bottlenecks. We show experimentally that state-of-the art integer linear programming approaches for orchestrating stream graphs are (1) intractable or at least impractical for larger stream graphs and larger number of processors and (2)our 2-approximation algorithm is highly efficient and its results are close to the optimal solution for a standard set of StreamIt benchmark programs.

Orchestration by approximation

We present a novel 2-approximation algorithm for deploying stream graphs on multicore computers and a stream graph transformation that eliminates bottlenecks. The key technical insight is a data rate transfer model that enables the computation of a "closed form", i.e., the data rate transfer function of an actor depending on the arrival rate of the stream program. A combinatorial optimization problem uses the closed form to maximize the throughput of the stream program. Although the problem is inherently NP-hard, we present an efficient and effective 2-approximation algorithm that provides a lower bound on the quality of the solution. We introduce a transformation that uses the closed form to identify and eliminate bottlenecks. We show experimentally that state-of-the art integer linear programming approaches for orchestrating stream graphs are (1) intractable or at least impractical for larger stream graphs and larger number of processors and (2)our 2-approximation algorithm is highly efficient and its results are close to the optimal solution for a standard set of StreamIt benchmark programs.

A high efficiency parallel unstructured solver dedicated to internal combustion engine simulation

IFP-C3D, a hexahedral unstructured parallel solver dedicated to multiphysics calculation is being developed at IFP to compute the compressible combustion in internal engines. IFP-C3D uses an unstructured formalism, the finite volume method on staggered grid, time splitting, SIMPLE loop, subcycled advection, turbulent and Lagrangian spray and a liquid film model. Original algorithms and models such as the conditional temporal interpolation methodology for moving grids, the remapping algorithm for transferring quantities on different meshes during the computation enable IFP-C3D to deal with complex moving geometries with large volume deformation induced by all moving geometrical parts (intake/exhaust valve, piston). Large super-scalar machines up to 1000 processors are being widely used and IFP-C3D has been optimized for running on these Cluster machines. IFP-C3D is parallelized using the Message Passing Interface (MPI) library to distribute a calculation over a large number of processors. Moreover, IFP-C3D uses an optimized linear algebraic library to solve linear matrix systems and the METIS partitionner library to distribute the computational load equally for all meshes used during the calculation and in particular during the remap stage when new meshes are loaded. Numerical results and performance timing are presented to demonstrate the computational efficiency of the code.

Inexact Newton Methods with Restricted Additive Schwarz Based Nonlinear Elimination for Problems with High Local Nonlinearity

The classical inexact Newton algorithm is an efficient and popular technique for solving large sparse nonlinear systems of equations. When the nonlinearities in the system are well balanced, a near quadratic convergence is often observed; however, if some of the equations are much more nonlinear than the others in the system, the convergence is much slower. The slow convergence (or sometimes divergence) is often determined by the small number of equations in the system with the highest nonlinearities. The idea of nonlinear preconditioning has been proven to be very useful. Through subspace nonlinear solves, the local high nonlinearities are removed, and the fast convergence can then be restored when the inexact Newton algorithm is called after the preconditioning. Recently a left preconditioned inexact Newton method was proposed in which the nonlinear function is replaced by a preconditioned function with more balanced nonlinearities. In this paper, we combine an inexact Newton method with a restricted additive Schwarz based nonlinear elimination. The new approach is easier to implement than the left preconditioned method since the nonlinear function does not have to be replaced, and, furthermore, the nonlinear elimination step does not have to be called at every outer Newton iteration. We show numerically that it performs well for, as an example, solving the incompressible Navier–Stokes equations with high Reynolds numbers and on machines with large numbers of processors.

Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on the cubed-sphere

High resolution and scalable parallel algorithms for the shallow water equations on the sphere are very important for modeling the global climate. In this paper, we introduce and study some highly scalable multilevel domain decomposition methods for the fully implicit solution of the nonlinear shallow water equations discretized with a second-order well-balanced finite volume method on the cubed-sphere. With the fully implicit approach, the time step size is no longer limited by the stability condition, and with the multilevel preconditioners, good scalabilities are obtained on computers with a large number of processors. The investigation focuses on the use of semismooth inexact Newton method for the case with nonsmooth topography and the use of two- and three-level overlapping Schwarz methods with different order of discretizations for the preconditioning of the Jacobian systems. We test the proposed algorithm for several benchmark cases and show numerically that this approach converges well with smooth and nonsmooth bottom topography, and scales perfectly in terms of the strong scalability and reasonably well in terms of the weak scalability on machines with thousands and tens of thousands of processors.

A parallel two-level method for simulating blood flows in branching arteries with the resistive boundary condition

Computer modeling of blood flows in the arteries is an important and very challenging problem. In order to understand, computationally, the sophisticated hemodynamics in the arteries, it is essential to couple the fluid flow and the elastic wall structure effectively and specify physiologically realistic boundary conditions. The computation is expensive and the parallel scalability of the solution algorithm is a key issue of the simulation. In this paper, we introduce and study a parallel two-level Newton–Krylov–Schwarz method for simulating blood flows in compliant branching arteries by using a fully coupled system of linear elasticity equation and incompressible Navier–Stokes equations with the resistive boundary condition. We first focus on the accuracy of the resistive boundary condition by comparing it with the standard pressure type boundary condition. We then show the parallel scalability results of the two-level approach obtained on a supercomputer with a large number of processors and on problems with millions of unknowns.

Unstructured mesh partition improvement for implicit finite element at extreme scale

Parallel simulations at extreme scale require that the mesh is distributed across a large number of processors with equal work load and minimum inter-part communications. A number of algorithms hav...

Parametrically splitting algorithms

The notion of parametrically splitting algorithms is introduced, which are characterized by their capability of solving a given problem on a large number of processors with few data transfers. These algorithms include many numerical integration algorithms, Monte Carlo and quasi-Monte Carlo methods, and so on.

Fast and Message-Efficient Global Snapshot Algorithms for Large-Scale Distributed Systems

Large-scale distributed systems such as supercomputers and peer-to-peer systems typically have a fully connected logical topology over a large number of processors. Existing snapshot algorithms in such systems have high response time and/or require a large number of messages, typically O(n2), where n is the number of processes. In this paper, we present a suite of two algorithms: simple_tree, and hypercube, that are both fast and require a small number of messages. This makes the algorithms highly scalable. Simple_tree requires O(n) messages and has O(log n) response time. Hypercube requires O(n log n) messages and has O(log n) response time, in addition to having the property that the roles of all the processes are symmetrical. Process symmetry implies greater potential for balanced workload and congestion-freedom. All the algorithms assume non-FIFO channels.

A novel algorithm for incompressible flow using only a coarse grid projection

Large scale fluid simulation can be difficult using existing techniques due to the high computational cost of using large grids. We present a novel technique for simulating detailed fluids quickly. Our technique coarsens the Eulerian fluid grid during the pressure solve, allowing for a fast implicit update but still maintaining the resolution obtained with a large grid. This allows our simulations to run at a fraction of the cost of existing techniques while still providing the fine scale structure and details obtained with a full projection. Our algorithm scales well to very large grids and large numbers of processors, allowing for high fidelity simulations that would otherwise be intractable.

Opal web services for biomedical applications

Biomedical applications have become increasingly complex, and they often require large-scale high-performance computing resources with a large number of processors and memory. The complexity of application deployment and the advances in cluster, grid and cloud computing require new modes of support for biomedical research. Scientific Software as a Service (sSaaS) enables scalable and transparent access to biomedical applications through simple standards-based Web interfaces. Towards this end, we built a production web server (http://ws.nbcr.net) in August 2007 to support the bioinformatics application called MEME. The server has grown since to include docking analysis with AutoDock and AutoDock Vina, electrostatic calculations using PDB2PQR and APBS, and off-target analysis using SMAP. All the applications on the servers are powered by Opal, a toolkit that allows users to wrap scientific applications easily as web services without any modification to the scientific codes, by writing simple XML configuration files. Opal allows both web forms-based access and programmatic access of all our applications. The Opal toolkit currently supports SOAP-based Web service access to a number of popular applications from the National Biomedical Computation Resource (NBCR) and affiliated collaborative and service projects. In addition, Opal’s programmatic access capability allows our applications to be accessed through many workflow tools, including Vision, Kepler, Nimrod/K and VisTrails. From mid-August 2007 to the end of 2009, we have successfully executed 239 814 jobs. The number of successfully executed jobs more than doubled from 205 to 411 per day between 2008 and 2009. The Opal-enabled service model is useful for a wide range of applications. It provides for interoperation with other applications with Web Service interfaces, and allows application developers to focus on the scientific tool and workflow development. Web server availability: http://ws.nbcr.net.

A general parallelization strategy for random path based geostatistical simulation methods

The size of simulation grids used for numerical models has increased by many orders of magnitude in the past years, and this trend is likely to continue. Efficient pixel-based geostatistical simulation algorithms have been developed, but for very large grids and complex spatial models, the computational burden remains heavy. As cluster computers become widely available, using parallel strategies is a natural step for increasing the usable grid size and the complexity of the models. These strategies must profit from of the possibilities offered by machines with a large number of processors. On such machines, the bottleneck is often the communication time between processors. We present a strategy distributing grid nodes among all available processors while minimizing communication and latency times. It consists in centralizing the simulation on a master processor that calls other slave processors as if they were functions simulating one node every time. The key is to decouple the sending and the receiving operations to avoid synchronization. Centralization allows having a conflict management system ensuring that nodes being simulated simultaneously do not interfere in terms of neighborhood. The strategy is computationally efficient and is versatile enough to be applicable to all random path based simulation methods.

Dockres: a computer program that analyzes the output of virtual screening of small molecules

BackgroundThis paper describes a computer program named Dockres that is designed to analyze and summarize results of virtual screening of small molecules. The program is supplemented with utilities that support the screening process. Foremost among these utilities are scripts that run the virtual screening of a chemical library on a large number of processors in parallel.MethodsDockres and some of its supporting utilities are written Fortran-77; other utilities are written as C-shell scripts. They support the parallel execution of the screening. The current implementation of the program handles virtual screening with Autodock-3 and Autodock-4, but can be extended to work with the output of other programs.ResultsAnalysis of virtual screening by Dockres led to both active and selective lead compounds.ConclusionsAnalysis of virtual screening was facilitated and enhanced by Dockres in both the authors' laboratories as well as laboratories elsewhere.

Controlling Unstructured Mesh Partitions for Massively Parallel Simulations

Parallel simulations at extreme scale require that the mesh is distributed across a large number of processors with equal work load and minimum interpart communications. A number of algorithms have been developed to meet these goals, e.g., graph/hypergraph and coordinate-based methods. However, the global implementation of current approaches can fail on very large core counts, which is resolved by combining global and local partitioning using multiple parts per processor. The other limitation of graph/hypergraph-based partitioning is that it uses one type of mesh entity as graph nodes; thus, the balance of other mesh entities may not be optimal. In the case of three-dimensional (3-D) linear finite element analysis, it is common to select mesh regions (elements) as partition objects. In current examples, the regions are well balanced up to 163,840 parts for a 1.07 billion element mesh, while the vertices have an imbalance which is as high as 19.52%. Two methods are developed that work in conjunction with graph/hypergraph-based procedures to provide improved partitions. Example computations executed on an IBM Blue Gene/P system using up to 163,840 cores demonstrate the usefulness of the procedures, particularly for time-critical calculations where individual cores may be lightly loaded in terms of the number of mesh entities per core. The algorithms presented in this paper reduced the vertex imbalance from 17.8% to 4.97% for a partition with 131,072 parts and accelerated the equation solution phase of the finite element analysis by 10.4%.

Grid and Cloud Computing and their Application

Grid and Cloud Computing and their Application

Special Issue: Scalable Tools for High‐end Computing

Current high-end parallel systems consist of hundreds of thousands of compute cores arranged in a complex hierarchical structure; future systems will have millions of cores. Systems, such as the Altix 4700, Blue Gene, Roadrunner, and Cray XT5, deploy multiple compute cores (homogeneous or heterogeneous) with multiple levels of shared and private caches within a processor, clustered into SMP nodes and coupled via a communication network to large-scale distributed systems. The development of efficient programs is extremely complex since the architectural details are exposed to the programmer. Productive use of such machines requires highly scalable programming tools for debugging, performance analysis, and fault tolerance. In addition, new programming models might significantly facilitate the task of the programmer. This special issue of Concurrency and Computation: Practice and Experience is devoted to programming tools that facilitate the development of efficient programs for such large-scale architectures. It is a collection of the best papers submitted to the international workshop on Scalable Tools for High-end Computing (STHEC 2008) that was held in conjunction with the International Conference on Supercomputing on June 7th on the Greek Island Kos. The papers present state-of-the-art tools for performance analysis and checkpointing on those machines. Performance analysis tools use measurements gathered during the execution of the application to detect portions of the code that can be further improved. Thus, they have to be able to cope with the large number of processors. Tools for checkpointing provide the possibility to restart an application in the case of a system failure; they have to be able to handle large number of cores as well. The selected papers present different techniques for building tools that will scale to thousands of cores. HPCToolkit 1 is a profiling-based performance analysis environment presenting the data in close relation to the source code without requiring an instrumentation of the source code. Scalasca 2 performs a parallel replay of the execution on the application's processors to find performance bottlenecks automatically. The combination of TAU and MRNet 3 provides a scalable infrastructure to offload performance data. Establishing the overlay network requires no added support from the job manager or application. Periscope 4 is based on a network of analysis agents that performs an online analysis of the application's performance behavior. When the application is started, additional processors can be allocated for the analysis agents to scale the analysis. CPPC 5 is a tool for portable checkpointing of message-passing applications. It consists of a runtime library and a compiler that assists the user by performing time-consuming tasks, such as data flow and communications analyses as well as code instrumentation. We would like to thank the authors for their excellent contributions to this special issue. We hope that it inspires future research in tools that support programmers of high-end systems in the development of efficient programs.

IFP-C3D: an Unstructured Parallel Solver for Reactive Compressible Gas Flow with Spray

IFP-C3D, a hexahedral unstructured parallel solver dedicated to multiphysics calculation, is being developed at IFP to compute the compressible combustion in internal engines. IFP-C3D uses an unstructured formalism, the finite volume method on staggered grids, time splitting, SIMPLE loop, sub-cycled advection, turbulent and Lagrangian spray and a liquid film model. Original algorithms and models such as the conditional temporal interpolation methodology for moving grids, the remapping algorithm for transferring quantities on different meshes during the computation enable IFP-C3D to deal with complex moving geometries with large volume deformation induced by all moving geometrical parts (intake/exhaust valve, piston). The Van Leer and Superbee slop limiters are used for advective fluxes and the wall law for the heat transfer model. Physical models developed at IFP for combustion (ECFM gasoline combustion model and ECFM3Z for Diesel combustion model), for ignition (TKI for auto-ignition and AKTIM for spark plug ignition) and for spray modelling enable the simulation of a large variety of innovative engine configurations from non-conventional Diesel engines using for instance HCCI combustion mode, to direct injection hydrogen internal combustion engines. Large super-scalar machines up to 1 000 processors are being widely used and IFP-C3D has been optimized for running on these Cluster machines. IFP-C3D is parallelized using the Message Passing Interface (MPI) library to distribute calculation over a large number of processors. Moreover, IFP-C3D uses an optimized linear algebraic library to solve linear matrix systems and the METIS partitionner library to distribute the computational load equally for all meshes used during the calculation and in particular during the remap stage when new meshes are loaded. Numerical results and timing are presented to demonstrate the computational efficiency of the code.

Enhancing parallel quasi-static particle-in-cell simulations with a pipelining algorithm

A pipelining algorithm to overcome the limitation on scaling quasi-static particle-in-cell models of relativistic beams in plasmas to a very large number of processors is described. The pipelining algorithm uses multiple groups of processors and optimizes the job allocation on the processors in parallel computing. The algorithm is implemented on the quasi-static code QuickPIC and is shown to scale to over 10 3 processors and increased the scale and speed by two orders of magnitude over the non-pipelined model. The new approach opens the door to performing full scale 3D simulations of future plasma wakefield accelerators or full lifetime models of beam interaction with electron clouds in circular accelerators such as the Large Hadron Collider (LHC) at CERN.

Parallel kinetic Monte Carlo simulations of Ag(111) island coarsening using a largedatabase

The results of parallel kinetic Monte Carlo (KMC) simulations of the room-temperaturecoarsening of Ag(111) islands carried out using a very large database obtained via self-learningKMC simulations are presented. Our results indicate that, while cluster diffusion andcoalescence play an important role for small clusters and at very early times, at late timethe coarsening proceeds via Ostwald ripening, i.e. large clusters grow while small clustersevaporate. In addition, an asymptotic analysis of our results for the average island sizeS(t) as a function oftime t leads to acoarsening exponent n = 1/3 (where S(t)∼t2n), in good agreement with theoretical predictions. However, by comparing with simulationswithout concerted (multi-atom) moves, we also find that the inclusion of such movessignificantly increases the average island size. Somewhat surprisingly we also find that,while the average island size increases during coarsening, the scaled island-sizedistribution does not change significantly. Our simulations were carried out both as atest of, and as an application of, a variety of different algorithms for parallelkinetic Monte Carlo including the recently developed optimistic synchronousrelaxation (OSR) algorithm as well as the semi-rigorous synchronous sublattice (SL)algorithm. A variation of the OSR algorithm corresponding to optimistic synchronousrelaxation with pseudo-rollback (OSRPR) is also proposed along with a method forimproving the parallel efficiency and reducing the number of boundary events viadynamic boundary allocation (DBA). A variety of other methods for enhancingthe efficiency of our simulations are also discussed. We note that, because ofthe relatively high temperature of our simulations, as well as the large range ofenergy barriers (ranging from 0.05 to 0.8 eV), developing an efficient algorithm forparallel KMC and/or SLKMC simulations is particularly challenging. However,by using DBA to minimize the number of boundary events, we have achievedsignificantly improved parallel efficiencies for the OSRPR and SL algorithms. Finally, wenote that, among the three parallel algorithms which we have tested here, thesemi-rigorous SL algorithm with DBA led to the highest parallel efficiencies. Asa result, we have obtained reasonable parallel efficiencies in our simulations ofroom-temperature Ag(111) island coarsening for a small number of processors(e.g. Np = 2 and 4). Since the SL algorithm scales with system size for fixed processor size, we expectthat comparable and/or even larger parallel efficiencies should be possible for parallel KMCand/or SLKMC simulations of larger systems with larger numbers of processors.

Numerical Simulation of Wave Propagation, Part II: Parallel Computing

In the first part of this series,' we described the most commonly used methods for numerically solving wave propagation in heterogeneous media with a single processor. However, if we want to perform such simulations for large-scale media, computations on single processors can quickly become prohibitively expensive. This is particularly true for elastic waves because we must compute at least three field variables at each node of a three-dimensional grid. Consequently, many research efforts have focused on developing massively parallel strategies for large numbers of processors; the purpose of this article is to provide an overview of what this research has accomplished so far. In the second part of a two-part series, the authors discuss massively parallel strategies for numerically solving wave propagation in heterogeneous media.