Large Number Of Processors Research Articles

Visualizing Hierarchical Performance Profiles of Parallel Codes Using CallFlow.

Calling context trees (CCTs) couple performance metrics with call paths, helping understand the execution and performance of parallel programs. To identify performance bottlenecks, programmers and performance analysts visually explore CCTs to form and validate hypotheses regarding degraded performance. However, due to the complexity of parallel programs, existing visual representations do not scale to applications running on a large number of processors. We present CallFlow, an interactive visual analysis tool that provides a high-level overview of CCTs together with semantic refinement operations to progressively explore CCTs. Using a flow-based metaphor, we visualize a CCT by treating execution time as a resource spent during the call chain, and demonstrate the effectiveness of our design with case studies on large-scale, production simulation codes.

A parallel domain decomposition algorithm for large scale image denoising

Total variation denoising (TVD) is an effective technique for image denoising, in particular, for recovering blocky, discontinuous images from noisy background. The problem is formulated as an optimization problem in the space of bounded variation functions, and the solution is obtained by solving the associated Euler–Lagrange equation defined on the domain occupied by the entire image. The method offers high quality results, but is computationally expensive for large images, especially for three-dimensional problems. In this paper, we introduce a highly parallel version of the algorithm which formulates the problem as multiple overlapping, but independent, optimization problems, and each is defined on a portion of the image domain. This approach is similar to the overlapping Schwarz type domain decomposition method, but is non-iterative, for solving partial differential equations, and is highly scalable, without using any coarse grids, for parallel computers with a large number of processors. We show by a theory and also by some two- and three-dimensional numerical experiments that the new approach has similar numerical accuracy as the classical TVD approach, but is much more efficient on parallel computers.

A Simflowny-based finite-difference code for high-performance computing in numerical relativity

The tremendous challenge of comparing our theoretical models with the gravitational-wave observations in the new era of multimessenger astronomy requires accurate and fast numerical simulations of complicated physical systems described by the Einstein and the matter equations. These requirements can only be satisfied if the simulations can be parallelized efficiently on a large number of processors and advanced computational strategies are adopted. To achieve this goal we have developed Simflowny, an open platform for scientific dynamical models which automatically generates parallel code for different simulation frameworks, allowing the use of HPC infrastructures to non-specialist scientists. One of these frameworks is SAMRAI, a mature patch-based structured adaptive mesh refinement infrastructure, capable of reaching exascale in some specific problems. Here we present the numerical techniques that we have implemented on this framework by using Simflowny in order to perform fast, efficient, accurate and highly-scalable simulations. These techniques involve high-order schemes for smooth and non-smooth solutions, Adaptive Mesh Refinement with arbitrary resolution ratios and an optimal strategy for the sub-cycling in time. We validate the automatically generated codes for the SAMRAI infrastructure with some simple test examples (i.e. wave equation and Newtonian MHD) and finally with the Einstein equations.

Massively parallel symplectic algorithm for coupled magnetic spin dynamics and molecular dynamics

A parallel implementation of coupled spin–lattice dynamics in the LAMMPS molecular dynamics package is presented. The approach is very general, and can be applied to simple ferromagnets, magnetic alloys, or amorphous magnetic materials. Equations of motion for both spin only and coupled spin–lattice dynamics are first reviewed, including a detailed account of how magneto-mechanical potentials can be used to perform a proper coupling between spin and lattice degrees of freedom. A symplectic numerical integration algorithm is then presented which combines the Suzuki–Trotter decomposition for non-commuting variables and conserves the geometric properties of the equations of motion. The numerical accuracy of the serial implementation was assessed by verifying that it conserves the total energy and the norm of the total magnetization up to second order in the timestep size. Finally, a very general parallel algorithm is proposed that allows large spin–lattice systems to be efficiently simulated on large numbers of processors without degrading its mathematical accuracy. Its correctness as well as scaling efficiency were tested for realistic coupled spin–lattice systems, confirming that the new parallel algorithm is both accurate and efficient.

THE PROBLEM OF PROVIDING CACHE COHERENCE IN MULTIPROCESSOR SYSTEMS WITH MANY PROCESSORS

In this paper the tasks of managing the directory in coherence maintenance systems in multiprocessor systems with a large number of processors are solved. In microprocessor systems with a large number of processors (MSLP) the problem of maintaining the coherence of processor caches is significantly complicated. This is due to increased traffic on the memory buses and increased complexity of interprocessor communications. This problem is solved in various ways. In this paper, we propose the use of Bloom filters used to accelerate the determination of an element’s belonging to a certain array. In this article, such filters are used to establish the fact that the processor belongs to some subset of the processors and determine if the processor has a cache line in the set. In the paper, the processes of writing and reading information in the data shared between processors are discussed in detail, as well as the process of data replacement from private caches. The article also shows how the addresses of cache lines and processor numbers are removed from the Bloom filters. The system proposed in this paper allows significantly speeding up the implementation of operations to maintain cache coherence in the MSLP as compared to conventional systems. In terms of performance and additional hardware and software costs, the proposed system is not inferior to the most efficient of similar systems, but on some applications and significantly exceeds them.

Coarse-graining simulation approaches for polymer melts: the effect of potential range on computational efficiency.

The integral equation coarse-graining (IECG) approach is a promising high-level coarse-graining (CG) method for polymer melts, with variable resolution from soft spheres to multi CG sites, which preserves the structural and thermodynamical consistencies with the related atomistic simulations. When compared to the atomistic description, the procedure of coarse-graining results in smoother free energy surfaces, longer-ranged potentials, a decrease in the number of interaction sites for a given polymer, and more. Because these changes have competing effects on the computational efficiency of the CG model, care needs to be taken when studying the effect of coarse-graining on the computational speed-up in CG molecular dynamics simulations. For instance, treatment of long-range CG interactions requires the selection of cutoff distances that include the attractive part of the effective CG potential and force. In particular, we show how the complex nature of the range and curvature of the effective CG potential, the selection of a suitable CG timestep, the choice of the cutoff distance, the molecular dynamics algorithms, and the smoothness of the CG free energy surface affect the efficiency of IECG simulations. By direct comparison with the atomistic simulations of relatively short chain polymer melts, we find that the overall computational efficiency is highest for the highest level of CG (soft spheres), with an overall improvement of the computational efficiency being about 106-108 for various CG levels/resolutions. Therefore, the IECG method can have important applications in molecular dynamics simulations of polymeric systems. Finally, making use of the standard spatial decomposition algorithm, the parallel scalability of the IECG simulations for various levels of CG is presented. Optimal parallel scaling is observed for a reasonably large number of processors. Although this study is performed using the IECG approach, its results on the relation between the level of CG and the computational efficiency are general and apply to any properly-constructed CG model.

A performance comparison of data and memory allocation strategies for sequence aligners on NUMA architectures

Over the last several years, many sequence alignment tools have appeared and become popular for the fast evolution of next generation sequencing technologies. Obviously, researchers that use such tools are interested in getting maximum performance when they execute them in modern infrastructures. Today’s NUMA (Non-uniform memory access) architectures present major challenges in getting such applications to achieve good scalability as more processors/cores are used. The memory system in NUMA systems shows a high complexity and may be the main cause for the loss of an application’s performance. The existence of several memory banks in NUMA systems implies a logical increase in latency associated with the accesses of a given processor to a remote bank. This phenomenon is usually attenuated by the application of strategies that tend to increase the locality of memory accesses. However, NUMA systems may also suffer from contention problems that can occur when concurrent accesses are concentrated on a reduced number of banks. Sequence alignment tools use large data structures to contain reference genomes to which all reads are aligned. Therefore, these tools are very sensitive to performance problems related to the memory system. The main goal of this study is to explore the trade-offs between data locality and data dispersion in NUMA systems. We have performed experiments with several popular sequence alignment tools on two widely available NUMA systems to assess the performance of different memory allocation policies and data partitioning strategies. We find that there is not one method that is best in all cases. However, we conclude that memory interleaving is the memory allocation strategy that provides the best performance when a large number of processors and memory banks are used. In the case of data partitioning, the best results are usually obtained when the number of partitions used is greater, sometimes combined with an interleave policy.

Directory Organizations in Cache Coherency Systems for High Performance Computation.

Caching the required Data or Instruction is one of the common practice used for improving the system performance. But in Multiprocessor Systems, Caching of shared data requires a cache coherency mechanism to maintain the shared memory model. In small-scale multiprocessors where all of the processing nodes sit on a common bus, the coherency problem could be solved with a Snoopy Protocols, but they do not scale up for large number of Processors. To support large Multiprocessors Directory Based Protocols are the best option. In these schemes a separate directory memory is maintained that identifies the caches containing each cache-line-sized block of memory. When data is written, the directory information is used to direct invalidation or update messages, depending on the type of protocol, to only those caches that must receive them. A primary step in designing a directory-based coherence mechanism is selecting a directory organization. This paper discus the directory organization, the storage structures that make up the directory and the nature of the information stored within. We discus on Limited Pointers and Dynamic Pointer allocation directories. These organizations potentially provide good performance while incurring modest memory overhead, even in large-scale machines. We discus the characteristics of both strategies, and compare them with different directory-based approaches.

Mock BSPlib for Testing and Debugging Bulk Synchronous Parallel Software

Testing parallel applications on a large number of processors is often impractical. Not only does it require access to scarce compute resources, but tracking down defects with the available debugging tools can often be very time consuming. Highly parallel codes should be testable on one processor at a time, so that a developer’s workstation is sufficient for executing and debugging test cases on millions of processes. Thanks to its supersteps, Bulk Synchronous Parallel programs are well suited for this kind of testing. This paper presents a mocking library for BSPlib which enables testing of fast and complex parallel algorithms at scale.

A Parallel Finite Element Method for 3D Two-Phase Moving Contact Line Problems in Complex Domains

Moving contact line problem plays an important role in fluid-fluid interface motion on solid surfaces. The problem can be described by a phase-field model consisting of the coupled Cahn–Hilliard and Navier–Stokes equations with the generalized Navier boundary condition (GNBC). Accurate simulation of the interface and contact line motion requires very fine meshes, and the computation in 3D is even more challenging. Thus, the use of high performance computers and scalable parallel algorithms are indispensable. In this paper, we generalize the GNBC to surfaces with complex geometry and introduce a finite element method on unstructured 3D meshes with a semi-implicit time integration scheme. A highly parallel solution strategy using different solvers for different components of the discretization is presented. More precisely, we apply a restricted additive Schwarz preconditioned GMRES method to solve the systems arising from implicit discretization of the Cahn–Hilliard equation and the velocity equation, and an algebraic multigrid preconditioned CG method to solve the pressure Poisson system. Numerical experiments show that the strategy is efficient and scalable for 3D problems with complex geometry and on a supercomputer with a large number of processors.

An Ensemble Kalman Filter for Numerical Weather Prediction Based on Variational Data Assimilation: VarEnKF

AbstractSeveral NWP centers currently employ a variational data assimilation approach for initializing deterministic forecasts and a separate ensemble Kalman filter (EnKF) system both for initializing ensemble forecasts and for providing ensemble background error covariances for the deterministic system. This study describes a new approach for performing the data assimilation step within a perturbed-observation EnKF. In this approach, called VarEnKF, the analysis increment is computed with a variational data assimilation approach both for the ensemble mean and for all of the ensemble perturbations. To obtain a computationally efficient algorithm, a much simpler configuration is used for the ensemble perturbations, whereas the configuration used for the ensemble mean is similar to that used for the deterministic system. Numerous practical benefits may be realized by using a variational approach for both deterministic and ensemble prediction, including improved efficiency for the development and maintenance of the computer code. Also, the use of essentially the same data assimilation algorithm would likely reduce the amount of numerical experimentation required when making system changes, since their impacts in the two systems would be very similar. The variational approach enables the use of hybrid background error covariances and may also allow the assimilation of a larger volume of observations. Preliminary tests with the Canadian global 256-member system produced significantly improved ensemble forecasts with VarEnKF as compared with the current EnKF and at a comparable computational cost. These improvements resulted entirely from changes to the ensemble mean analysis increment calculation. Moreover, because each ensemble perturbation is updated independently, VarEnKF scales perfectly up to a very large number of processors.

An Optimal Routing Algorithm for Horizontal Moving Signals in OCN for Massively Parallel Systems with Faulty Node/Link

An Octagon-Cell Interconnected Network (OCN) has many attractive features. To represent OCN an undirected graph G = (V, E) is used, in which V is the set of nodes in the graph and E is the set of edges in the graph. Already the optimal routing algorithm had been presented with its features in our past research work. This research paper presents the optimal routing algorithm for horizontal moving signals in OCN with a faulty node/link along the optimal path. OCN is expandable. Also the algorithm tells that, even the OCN is expanded; there is no effect to find the optimal path in presence of faulty nodes. OCN can be utilized in massively parallel computing. In a massively parallel system a large number of processors are used to perform a set of coordinated computation simultaneously. So OCN is assumed a type of integrated circuit with an array of hundreds or thousands of central processing units (CPUs) and random-access memory banks.

Performance Evaluation of Parallel Sorting Algorithms on IMAN1 Supercomputer

Many sorting algorithms have been proposed and implemented in previous years. These algorithms are usually judged by their performance in term of algorithm growth rate according to the input size. Efficient sorting algorithm implementation is important for optimizing the use of other algorithms such as searching algorithms, load balancing algorithms, etc. In this paper, parallel Quicksort, parallel Merge sort, and parallel MergeQuicksort algorithms are evaluated and compared in terms of the running time, speedup, and parallel efficiency. These sorting algorithms are implemented using Message Passing Interface (MPI) library, and results have been conducted using IMAN1 supercomputer. Results show that the run time of parallel Quicksort algorithm outperforms both Merge sort and Merge-Quicksort algorithms. Moreover, on large number of processors, parallel Quicksort achieves the best parallel efficiency of up to 88%, while Merge sort and MergeQuicksort algorithms achieve up to 49% and 52% parallel efficiency, respectively.

GPU-accelerated raster map reprojection

<p>Reprojecting raster maps from one projection to another is an essential part of many cartographic processes (map comparison, overlays, data presentation, ...) and reducing the required computational time is desirable and often significantly decreases overall processing costs.</p><p>The raster reprojection process operates per-pixel and is, therefore, a good candidate for GPU-based parallelization where the large number of processors can lead to a very high degree of parallelism.</p><p>We have created an experimental implementation of the raster reprojection with GPU-based parallelization (using OpenCL API).<br />During the evaluation, we compared the performance of our implementation to the optimized GDAL and showed that there is a class of problems where GPU-based parallelization can lead to more than sevenfold speedup.</p>

The Additive Peaceman–Rachford Method

A new version of the parallel Alternating Direction Implicit (ADI) method by Peaceman and Rachford for solving systems of linear algebraic equations with positive-definite coefficient matrices represented as sums of two commuting terms is suggested. The algorithms considered are suited for solving two-dimensional grid boundary-value problems with separable variables, as well as the Sylvester and Lyapunov matrix equations. The approach to rising parallel efficiency proposed in the paper is based on representing rational functions as sums of partial fractions. An additive version of the factorized ADI method for solving Sylvester’s equation is described. Estimates of the speedup resulting from increasing the number of computer units are presented. These estimates demonstrate a potential advantage of using the additive algorithms when implemented on a supercomputer with large number of processors or cores.

Parallel data-driven decomposition algorithm for large-scale datasets: with application to transitional boundary layers

Many fluid flows of engineering interest, though very complex in appearance, can be approximated by low-order models governed by a few modes, able to capture the dominant behavior (dynamics) of the system. This feature has fueled the development of various methodologies aimed at extracting dominant coherent structures from the flow. Some of the more general techniques are based on data-driven decompositions, most of which rely on performing a singular value decomposition (SVD) on a formulated snapshot (data) matrix. The amount of experimentally or numerically generated data expands as more detailed experimental measurements and increased computational resources become readily available. Consequently, the data matrix to be processed will consist of far more rows than columns, resulting in a so-called tall-and-skinny (TS) matrix. Ultimately, the SVD of such a TS data matrix can no longer be performed on a single processor, and parallel algorithms are necessary. The present study employs the parallel TSQR algorithm of (Demmel et al. in SIAM J Sci Comput 34(1):206–239, 2012), which is further used as a basis of the underlying parallel SVD. This algorithm is shown to scale well on machines with a large number of processors and, therefore, allows the decomposition of very large datasets. In addition, the simplicity of its implementation and the minimum required communication makes it suitable for integration in existing numerical solvers and data decomposition techniques. Examples that demonstrate the capabilities of highly parallel data decomposition algorithms include transitional processes in compressible boundary layers without and with induced flow separation.

A Scalable Parallel PWTD-Accelerated SIE Solver for Analyzing Transient Scattering From Electrically Large Objects

A scalable parallel plane-wave time-domain (PWTD) algorithm for efficient and accurate analysis of transient scattering from electrically large objects is presented. The algorithm produces scalable communication patterns on very large numbers of processors by leveraging two mechanisms: 1) a hierarchical parallelization strategy to evenly distribute the computation and memory loads at all levels of the PWTD tree among processors and 2) a novel asynchronous communication scheme to reduce the cost and memory requirement of the communications between the processors. The efficiency and accuracy of the algorithm are demonstrated through its applications to the analysis of transient scattering from a perfect electrically conducting (PEC) sphere with a diameter of 70 wavelengths and a PEC square plate with a dimension of 160 wavelengths. Furthermore, the proposed algorithm is used to analyze transient fields scattered from realistic airplane and helicopter models under high frequency excitation.

Parallel coupling numerics for partitioned fluid–structure interaction simulations

Within the last decade, very sophisticated numerical methods for the iterative and partitioned solution of fluid–structure interaction problems have been developed that allow for high accuracy and very complex scenarios. The combination of these two aspects–accuracy and complexity–demands very high computational grid resolutions and, thus, high performance computing methods designed for massively parallel hardware architectures. For those architectures, currently used coupling methods, which mainly work with a staggered execution of the fluid and the structure solver, i.e., the execution of one solver after the other in every outer iteration, lead to severe load imbalances: if the flow solver, e.g., scales on a very large number of processors but the structural solver does not due to its limited amount of data and required operations, almost all processors assigned to the coupled simulations are idle during the execution of the structure solver. We propose two new iterative coupling methods that allow for the simultaneous execution of flow and structure solvers. In both cases, we show that pure fixed-point iterations based on the parallel execution of the solvers do not lead to good results, but the combination of parallel solver execution and so-called quasi-Newton methods yields very efficient and robust methods. Those methods are known to be very efficient also for the stabilization of critical scenarios solved with the standard staggered solver execution. We demonstrate the competitive convergence of our methods for various established benchmark scenarios. Both methods are perfectly suited for use with black-box solvers because the quasi-Newton approach uses solely input and output information of the solvers to approximate the effect of the unknown Jacobians that would be required in a standard Newton solver.

Orders of Magnitude Faster cryo-EM Refinement with GPUs

Direct-electron detectors, increased computational power and advances in sample preparation have rapidly made single particle cryogenic electron microscopy (cryo-em) a leading structural determination method. However, the growing number of images collected with constantly increasing resolution raises the demand on the computational processing capabilities, rendering the refinement step an increasingly notable bottleneck. For large macromolecular complexes, like ribosomes, this can require up to weeks of time on computer clusters using RELION, the currently fastest growing refinement tool. In recognition of this issue we have, in close collaboration with the original developers, optimized a significant portion of RELION for execution on consumer streaming multiprocessors (GPUs). The massive parallelism inherent in the algorithm enables an efficient utilization of such hardware layout, where the large number of processors can be assigned isolated small tasks. Here we present a new implementation yielding a 100-fold speed-up of the most time consuming step (3D classification) using GPUs, with comparable prices to conventional hardware. The resulting increase in overall cost efficiency, and removal of the need for large clusters, drastically increases the accessibility of the method to the field. Whereas, the reduced computational time makes it feasible to further strengthen the capabilities of the method by expanding the biophysical analysis, with other computationally intensive methods. Cryo-em is undoubtedly becoming a dominant tool within structural biology, why the acquired leap in throughput and applicability will have a major overall impact on the field in the coming years.

Parallel implementation of 3D FFT with volumetric decomposition schemes for efficient molecular dynamics simulations

Three-dimensional Fast Fourier Transform (3D FFT) plays an important role in a wide variety of computer simulations and data analyses, including molecular dynamics (MD) simulations. In this study, we develop hybrid (MPI+OpenMP) parallelization schemes of 3D FFT based on two new volumetric decompositions, mainly for the particle mesh Ewald (PME) calculation in MD simulations. In one scheme, (1d_Alltoall), five all-to-all communications in one dimension are carried out, and in the other, (2d_Alltoall), one two-dimensional all-to-all communication is combined with two all-to-all communications in one dimension. 2d_Alltoall is similar to the conventional volumetric decomposition scheme. We performed benchmark tests of 3D FFT for the systems with different grid sizes using a large number of processors on the K computer in RIKEN AICS. The two schemes show comparable performances, and are better than existing 3D FFTs. The performances of 1d_Alltoall and 2d_Alltoall depend on the supercomputer network system and number of processors in each dimension. There is enough leeway for users to optimize performance for their conditions. In the PME method, short-range real-space interactions as well as long-range reciprocal-space interactions are calculated. Our volumetric decomposition schemes are particularly useful when used in conjunction with the recently developed midpoint cell method for short-range interactions, due to the same decompositions of real and reciprocal spaces. The 1d_Alltoall scheme of 3D FFT takes 4.7 ms to simulate one MD cycle for a virus system containing more than 1 million atoms using 32,768 cores on the K computer.