Performance Of Scientific Applications Research Articles

An approach for realistically simulating the performance of scientific applications on high performance computing systems

Scientific applications often contain large, computationally-intensive, and irregular parallel loops or tasks that exhibit stochastic behavior leading to load imbalance. Load imbalance often manifests during the execution of parallel scientific applications on large and complex high performance computing (HPC) systems. The extreme scale of HPC systems on the road to Exascale computing only exacerbates the loss in performance due to load imbalance. Dynamic loop self-scheduling (DLS) techniques are instrumental in improving the performance of scientific applications on HPC systems via load balancing. Selecting a DLS technique that results in the best performance for different problem and system sizes requires a large number of exploratory experiments. Currently, a theoretical model that can be used to predict the scheduling technique that yields the best performance for a given problem and system has not yet been identified. Therefore, simulation is the most appropriate approach for conducting such exploratory experiments in a reasonable amount of time. However, conducting realistic and trustworthy simulations of application performance under different configurations is challenging. This work devises an approach to realistically simulate computationally-intensive scientific applications that employ DLS and execute on HPC systems. The proposed approach minimizes the sources of uncertainty in the simulative experiments results by bridging the native and simulative experimental approaches. A new method is proposed to capture the variation of application performance between different native executions. Several approaches to represent the application tasks (or loop iterations) are compared to establish their influence on the simulative application performance. A novel simulation strategy is introduced that applies the proposed approach, which transforms a native application code into simulative code. The native and simulative performance of two computationally-intensive scientific applications that employ eight task scheduling techniques (static, nonadaptive dynamic, and adaptive dynamic) are compared to evaluate the realism of the proposed simulation approach. The comparison of the performance characteristics extracted from the native and simulative performance shows that the proposed simulation approach fully captured most of the performance characteristics of interest. This work shows and establishes the importance of simulations that realistically predict the performance of DLS techniques for different applications and system configurations.

First Steps in Porting the LFRic Weather and Climate Model to the FPGAs of the EuroExa Architecture

In recent years, there has been renewed interest in the use of field-programmable gate arrays (FPGAs) for high-performance computing (HPC). In this paper, we explore the techniques required by traditional HPC programmers in porting HPC applications to FPGAs, using as an example the LFRic weather and climate model. We report on the first steps in porting LFRic to the FPGAs of the EuroExa architecture. We have used Vivado High-Level Syntheusywwi to implement a matrix-vector kernel from the LFRic code on a Xilinx UltraScale+ development board containing an XCZU9EG multiprocessor system-on-chip. We describe the porting of the code, discuss the optimization decisions, and report performance of 5.34 Gflop/s with double precision and 5.58 Gflop/s with single precision. We discuss sources of inefficiencies, comparisons with peak performance, comparisons with CPU and GPU performance (taking into account power and price), comparisons with published techniques, and comparisons with published performance, and we conclude with some comments on the prospects for future progress with FPGA acceleration of the weather forecast model. The realization of practical exascale-class high-performance computinems requires significant improvements in the energy efficiency of such systems and their components. This has generated interest in computer architectures which utilize accelerators alongside traditional CPUs. FPGAs offer huge potential as an accelerator which can deliver performance for scientific applications at high levels of energy efficiency. The EuroExa project is developing and building a high-performance architecture based upon ARM CPUs with FPGA acceleration targeting exascale-class performance within a realistic power budget.

Analyzing time-dimension communication characterizations for representative scientific applications on supercomputer systems

Exascale computing is one of the major challenges of this decade, and several studies have shown that communications are becoming one of the bottlenecks for scaling parallel applications. The analysis on the characteristics of communications can effectively aid to improve the performance of scientific applications. In this paper, we focus on the statistical regularity in time-dimension communication characteristics for representative scientific applications on supercomputer systems, and then prove that the distribution of communication-event intervals has a power-law decay, which is common in scientific interests and human activities. We verify the distribution of communication-event intervals has really a power-law decay on the Tianhe-2 supercomputer, and also on the other six parallel systems with three different network topologies and two routing policies. In order to do a quantitative study on the power-law distribution, we exploit two groups of statistics: bursty vs. memory and periodicity vs. dispersion. Our results indicate that the communication events show a “strong-bursty and weak-memory” characteristic and the communication event intervals show the periodicity and the dispersion. Finally, our research provides an insight into the relationship between communication optimizations and time-dimension communication characteristics.

Interference-aware co-scheduling method based on classification of application characteristics from hardware performance counter using data mining

Computational scientists and engineers who are eager to obtain the best performance of scientific applications need efficient application characterization methods to successfully exploit high-performance hardware resources. However, modern processors are accompanied by high-bandwidth on-chip memory or a large number of cores. Therefore, application characterization research that takes into account the newly introduced hardware features in next-generation high performance computing environments is insufficient and complex. In this paper, we propose a simple and fast method to classify the application characteristics in systems state-of-the-art processors using hardware performance counters. The proposed method utilizes hardware performance counters to monitor hardware events related to system performance. A clustering approach is adopted that requires limited understanding of the correlation between hardware events and application characteristics. The application characterization technique is applied to NAS parallel benchmarks in two systems, including Intel Knights Landing and SkyLake Xeon processors. We demonstrate that the proposed techniques can capture system and application characteristics and provide users with useful insights into application execution.

A Genetic Algorithm Based Data Replica Placement Strategy for Scientific Applications in Clouds

Cloud computing is a promising distributed computing platform for big data applications, e.g., scientific applications, since excessive resources can be obtained from cloud services for processing and storing both existing and generated application datasets. However, when tasks process big data stored in distributed data centers, the inevitable data movements will cause huge bandwidth cost and execution delay. In this paper, we construct a tripartite graph based model to formulate the data replica placement problem and propose a genetic algorithm based data replica placement strategy for scientific applications to reduce data transmissions in cloud. Our approach can reduce 1) the size of moved data, 2) the time of data movement and 3) the number of movements. We conduct experiments to compare the proposed strategy with the random placement strategy used in Hadoop Distributed Files System (HDFS), which demonstrates that our strategy has better performance for scientific applications in clouds.

Solving Large Problem Sizes of Index-Digit Algorithms on GPU: FFT and Tridiagonal System Solvers

Current Graphics Processing Units (GPUs) are capable of obtaining high computational performance in scientific applications. Nevertheless, programmers have to use suitable parallel algorithms for these architectures and usually have to consider optimization techniques in the implementation in order to achieve said performance. There are many efficient proposals for limited-size problems which fit directly in the shared memory of CUDA GPUs, however, there are few GPU proposals that tackle the design of efficient algorithms for large problem sizes that exceed shared memory storage capacity. In this work, we present a tuning strategy that addresses this problem for some parallel prefix algorithms that can be represented according to a set of common permutations of the digits of each of its element indices [1], denoted as Index-Digit (ID) algorithms. Specifically, our strategy has been applied to develop flexible Multi-Stage (MS) algorithms for the Fast Fourier Transform (FFT) algorithm (MS-ID-FFT) and a tridiagonal system solver (MS-ID-TS) on the GPU. The resulting implementation is compact and outperforms other well-known and commonly used state-of-the-art libraries, with an improvement of up to 1.47x with respect to NVIDIA's complex CUFFT, and up to 33.2x in comparison with NVIDIA's CUSPARSE for real data tridiagonal systems.

Performance Improvement Options of Scientific Applications on XeonPhi KNL Architectures

Intel's recent manycore processor KNights Landing (KNL) promises high performance for scientific applications. Careful tuning for the complex chip architecture is required to efficiently exploit the chip's hardware resources. This paper describes performance improvement techniques and demonstrates their effectiveness for scientific applications. Experiments were conducted with some of the National Aeronautics and Space Administration (NASA's) advanced supercomputing (NAS) parallel benchmarks, and the effectiveness of: 1) advanced vector extensions (AVX-512) vectorisation support; 2) manycore threading support; 3) the utilisation of thread affinities for different KNL modes, was analysed.

Data‐aware tuning of scientific applications with model‐based autotuning

SummaryAutotuning is a widely accepted method for adapting the execution of scientific applications to the underlying hardware. One method for speeding up the search for the optimal candidate implementation in autotuning, which often is very time‐consuming, is the performance prediction using an analytical model. Using such a model‐based autotuning method, autotuning may be performed at runtime, which enables the application to consider data characteristics for the tuning decision. The article gives an overview on existing works, which apply model‐based autotuning for data‐aware tuning decisions in scientific computing. It shows that model‐based autotuning is a feasible method for increasing the performance of scientific applications by considering influences from the input data, where ‘performance’ may be execution time or energy, among others. It furthermore presents a systematisation of model‐based autotuning: it proposes a distinction of algorithm parameters, machine‐specific parameters and data‐specific parameters. The article gives suggestions on how to set up an application‐specific analytical model, presents an example for that and discusses the limits of model‐based autotuning. Copyright © 2016 John Wiley & Sons, Ltd.

Performance characterization of irregular I/O at the extreme scale

We statistically model the I/O behavior of two irregular applications.We feed the model into two ADIOS I/O kernels and two HDF5 I/O kernels.We measure the performance of the two applications' I/O under different I/O parameter settings.We critically compare and contrast the results for a large set of I/O scenarios.We show how I/O size and irregular I/O patterns are relevant factors when tuning performance. This paper reports our experience with irregular I/O and describes lessons learned when running applications with such I/O on supercomputers at the extreme scale. Specifically, we study how irregularities in I/O patterns (i.e., irregular amount of data written per process at each I/O step) in scientific simulations can cause increasing I/O times and substantial loss in scalability. To this end, we quantify the impact of irregular I/O patterns on the I/O performance of scientific applications at the extreme scale by statistically modeling the irregular I/O behavior of two scientific applications: the Monte Carlo application QMCPack and the adaptive mesh refinement application ENZO. For our testing, we feed our model into I/O kernels of two well-known I/O data models (i.e., ADIOS and HDF) to measure the performance of the two applications' I/O under different I/O settings. Empirically, we show how the growing data sizes and the irregular I/O patterns in these applications are both relevant factors impacting performance.

Compiler transformation of nested loops for general purpose GPUs

SummaryManycore accelerators have the potential to significantly improve performance of scientific applications when offloading computationally intensive program portions to accelerators. Directive‐based high‐level programming models, such as OpenACC and OpenMP, are used to create applications for accelerators through annotating regions of code meant for offloading. OpenACC is an emerging directive‐based programming model for programming accelerators that typically enable inexperienced programmers to achieve portable and productive performance within applications. In this paper, we present our research in developing challenges and solutions when creating an open‐source OpenACC compiler in an industrial framework (OpenUH as a branch of Open64). We then discuss in detail techniques we developed for loop scheduling reduction operations on general purpose GPUs. The compiler is evaluated with benchmarks from the NAS Parallel Benchmarks suite and self‐written micro‐benchmarks for reduction operations. This implementation has been designed to serve as a compiler infrastructure for researchers to explore advanced compiler techniques, extend OpenACC to other programming models, and build performance tools used in conjunction with OpenACC programs. Copyright © 2015 John Wiley & Sons, Ltd.

The Case for Partitioning Virtual Machines on Multicore Architectures

In this paper we argue that partitioning is required for attaining the best performance of scientific applications when running on virtual machines. Current memory management and I/O handling techniques introduce high overhead when running scientific applications. Using KVM, we quantify this impact on applications written in multiple paradigms: message passing, shared memory and partitioned global address spaces. Our analysis shows that on NUMA systems, current memory translation schemes cannot preserve the locality of access and introduce up to 82% slowdown. We discuss the interaction between contemporary OS and VM architectures and argue that partitioning is the only feasible solution to enforce memory locality. Current I/O solutions using one assistant task cannot provide the level of I/O parallelism required by scientific applications and we observe an average 7.2x application slowdown on a cluster with 16 cores per node. More specialized solutions that implement shared memory by-pass within the communication stack also do not scale well with cores and we observe an average 2.4x application slowdown. Overall, our results indicate that using partitioning and direct inter-VM shared memory support is enough to provide close to native performance in manycore clusters.

Application-specific I/O Optimizations on Petascale Supercomputers

Data-intensive science frontiers and challenges are emerg ing as computer technology has evolved substantially. Large-scale simu lations demand significant I/O workload, and as a result the I/O performance often becomes a bottleneck preventing high performance in scientific applications. In this paper we introduce a variety of I/O optimization techniques developed and imple mented when scaling a seismic application to petascale. These techniques include file system striping, data aggreg ation, reader/writer limit ing and less interleaving of data, collective MPI-IO, and data staging. The optimizations result in nearly perfect scalability of the target applicat ion on some of the most advanced petascale systems. The techniques introduced in this paper are applicab le to other scientific applications facing similar petascale I/O challenges.

HiCOO: Hierarchical cooperation for scalable communication in Global Address Space programming models on Cray XT systems

Global Address Space (GAS) programming models enable a convenient, shared-memory style addressing model. Typically this is realized through one-sided operations that can enable asynchronous communication and data movement. With the size of petascale systems reaching 10,000s of nodes and 100,000s of cores, the underlying runtime systems face critical challenges in (1) scalably managing resources (such as memory for communication buffers), and (2) gracefully handling unpredictable communication patterns and any associated contention. For any solution that addresses these resource scalability challenges, equally important is the need to maintain the performance of GAS programming models. In this paper, we describe a Hierarchical COOperation (HiCOO) architecture for scalable communication in GAS programming models. HiCOO formulates a cooperative communication architecture: with inter-node cooperation amongst multiple nodes (a.k.a multinode) and hierarchical cooperation among multinodes that are arranged in various virtual topologies. We have implemented HiCOO for a popular GAS runtime library, Aggregate Remote Memory Copy Interface (ARMCI). By extensively evaluating different virtual topologies in HiCOO in terms of their impact to memory scalability, network contention, and application performance, we identify MFCG as the most suitable virtual topology. The resulting HiCOO architecture is able to realize scalable resource management and achieve resilience to network contention, while at the same time maintaining or enhancing the performance of scientific applications. In one case, it reduces the total execution time of an NWChem application by 52%.

A spectral clustering-based optimal deployment method for scientific application in cloud computing

Similar to Grid computing systems, scientific applications in cloud are large scale distributed systems that are deployed on distributed cloud nodes. Scientific applications usually have a lot of communications between the nodes for deployment. Therefore conventional ranking methods are not appropriate for deploying scientific applications. The reason is ranking methods do not consider the relations between nodes. We propose a novel spectral clustering based deployment method that takes not only the computing qualities of cloud nodes into account, but also the communication performance between different nodes. Experimental results show the effectiveness of our method for improving the performance of scientific applications.

Process Arrival Pattern Aware Alltoall and Allgather on InfiniBand Clusters

Recent studies show that MPI processes in real applications could arrive at an MPI collective operation at different times. This imbalanced process arrival pattern can significantly affect the performance of the collective operation. MPI_Alltoall() and MPI_Allgather() are communication-intensive collective operations that are used in many scientific applications. Therefore, their efficient implementations under different process arrival patterns are critical to the performance of scientific applications running on modern clusters. In this paper, we propose novel RDMA-based process arrival pattern aware MPI_Alltoall() and MPI_Allgather() for different message sizes over InfiniBand clusters. We also extend the algorithms to be shared memory aware for small to medium size messages under process arrival patterns. The performance results indicate that the proposed algorithms outperform the native MVAPICH implementations as well as other non-process arrival pattern aware algorithms when processes arrive at different times. Specifically, the RDMA-based process arrival pattern aware MPI_Alltoall() and MPI_Allgather() are 3.1 times faster than MVAPICH for 8 KB messages. On average, the applications studied in this paper (FT, RADIX, and N-BODY) achieve a speedup of 1.44 using the proposed algorithms.

Performance analysis of a hybrid MPI/OpenMP application on multi-core clusters

The mixing of shared memory and message passing programming models within a single application has often been suggested as a method for improving scientific application performance on clusters of shared memory or multi-core systems. DL_POLY, a large scale molecular dynamics application programmed using message passing programming has been modified to add a layer of shared memory threading and the performance analysed on two multi-core clusters. At lower processor numbers, the extra overheads from shared memory threading in the hybrid code outweigh performance benefits gained over the pure MPI code. On larger core counts the hybrid model performs better than pure MPI, with reduced communication time decreasing the overall runtime.

Interfaces for parallel numerical linear algebra libraries in high level languages

In many high performance engineering and scientific applications there is a need to use parallel software libraries. Researchers behind these applications find it difficult to understand the interfaces to these libraries because they carry arguments that are related to the parallel environment and performance in addition to arguments related to the problem at hand. In this paper we introduce the use of high level user interfaces for ScaLAPACK. Concretely, a Python-based interface to ScaLAPACK is proposed. Numerical experiments comparing traditional programming practices with our proposed approach are presented. These experiments evaluate not only the performance of the Python interfaces but also how user friendlier they are, compared to the original calls, and show that PyScaLAPACK does not hinder the performance deliverance of ScaLAPACK. Finally, an example of a real scientific application code, whose functionality can be prototyped or extended with the use of PyScaLAPACK, is presented.

Multilevel summation of electrostatic potentials using graphics processing units

Physical and engineering practicalities involved in microprocessor design have resulted in flat performance growth for traditional single-core microprocessors. The urgent need for continuing increases in the performance of scientific applications requires the use of many-core processors and accelerators such as graphics processing units (GPUs). This paper discusses GPU acceleration of the multilevel summation method for computing electrostatic potentials and forces for a system of charged atoms, which is a problem of paramount importance in biomolecular modeling applications. We present and test a new GPU algorithm for the long-range part of the potentials that computes a cutoff pair potential between lattice points, essentially convolving a fixed 3D lattice of “weights” over all sub-cubes of a much larger lattice. The implementation exploits the different memory subsystems provided on the GPU to stream optimally sized data sets through the multiprocessors. We demonstrate for the full multilevel summation calculation speedups of up to 26 using a single GPU and 46 using multiple GPUs, enabling the computation of a high-resolution map of the electrostatic potential for a system of 1.5 million atoms in under 12 s.

Scientific Application Performance On Leading Scalar and Vector Supercomputering Platforms

The last decade has witnessed a rapid proliferation of superscalar cache-based microprocessors to build high-end computing (HEC) platforms, primarily because of their generality, scalability, and cost effectiveness. However, the growing gap between sustained and peak performance for full-scale scientific applications on conventional supercomputers has become a major concern in high performance computing, requiring significantly larger systems and application scalability than implied by peak performance in order to achieve desired performance. The latest generation of custom-built parallel vector systems have the potential to address this issue for numerical algorithms with sufficient regularity in their computational structure. In this work we explore applications drawn from four areas: magnetic fusion (GTC), plasma physics (LB-MHD-3D), astrophysics (Cactus), and material science (PARATEC). We compare performance of the vector-based Cray X1, X1E, Earth Simulator, NEC SX-8, with performance of three leading commodity-based super-scalar platforms utilizing the IBM Power3, Intel Itanium2, and AMD Opteron processors. Our work makes several significant contributions: a new data-decomposition scheme for GTC that (for the first time) enables a breakthrough of the teraflop barrier; the introduction of a new three-dimensional lattice Boltzmann magneto-hydrodynamic implementation used to study the onset evolution of plasma turbulence that achieves over 26 Tflop/s on 4800 ES processors; the highest per processor performance (by far) achieved by the full-production version of the Cactus ADM-BSSN; and the largest PARATEC cell size atomistic simulation to date. Overall, results show that the vector architectures attain unprecedented aggregate performance across our application suite, demonstrating the tremendous potential of modern parallel vector systems.

Dynamic load balancing with adaptive factoring methods in scientific applications

To improve the performance of scientific applications with parallel loops, dynamic loop scheduling methods have been proposed. Such methods address performance degradations due to load imbalance caused by predictable phenomena like nonuniform data distribution or algorithmic variance, and unpredictable phenomena such as data access latency or operating system interference. In particular, methods such as factoring, weighted factoring, adaptive weighted factoring, and adaptive factoring have been developed based on a probabilistic analysis of parallel loop iterates with variable running times. These methods have been successfully implemented in a number of applications such as: N-Body and Monte Carlo simulations, computational fluid dynamics, and radar signal processing. The focus of this paper is on adaptive weighted factoring (AWF), a method that was designed for scheduling parallel loops in time-stepping scientific applications. The main contribution of the paper is to relax the time-stepping requirement, a modification that allows the AWF to be used in any application with a parallel loop. The modification further allows the AWF to adapt to load imbalance that may occur during loop execution. Results of experiments to compare the performance of the modified AWF with the performance of the other loop scheduling methods in the context of three nontrivial applications reveal that the performance of the modified method is comparable to, and in some cases, superior to the performance of the most recently introduced adaptive factoring method.