Large-scale Parallel Applications Research Articles

Multi-objective and multi constrained task scheduling framework for computational grids

Grid computing emerged as a powerful computing domain for running large-scale parallel applications. Scheduling computationally intensive parallel applications such as scientific, commercial etc., computational grids is a NP-complete problem. Many researchers have proposed several task scheduling algorithms on grids based on formulating and solving it as an optimization problem with different objective functions such as makespan, cost, energy etc. Further to address the requirements/demands/needs of the users (lesser cost, lower latency etc.) and grid service providers (high utilization and high profitability), a task scheduler needs to be designed based on solving a multi-objective optimization problem due to several trade-offs among the objective functions. In this direction, we propose an efficient multi-objective task scheduling framework to schedule computationally intensive tasks on heterogeneous grid networks. This framework minimizes turnaround time, communication, and execution costs while maximizing grid utilization. We evaluated the performance of our proposed algorithm through experiments conducted on standard, random, and scientific task graphs using the GridSim simulator.

QTMS: A quadratic time complexity topology-aware process mapping method for large-scale parallel applications on shared HPC system

Communication exacerbates the performance for parallel applications with thousands of CPU cores and quantities of data to exchange. The high communication cost is usually attributed to the mismatch between the communication patterns of parallel applications and the physical topology graphs of the computing resources (or the underlying network topologies). The topology-aware process mapping method can usually obtain a better embedding scheme with the aim to improve communication performance. Many existing heuristic-search based mapping methods have high execution time for large-scale applications. Some low-cost graph-partitioning based mapping methods depend on that the allocated resources form a regular structure, which is usually impractical in most high performance computing systems shared by multiple users and applications. This weakens their performance. Other graph-partitioning based mapping methods come at a high cost or require users to provide the network structure information. To address these issues, a quadratic time complexity topology-aware process mapping method is presented in this paper. The experimental results show that the proposed method often achieves a better application communication performance than several state-of-the-art mapping methods on a shared HPC system, while maintaining a significantly lower execution cost. Moreover, the real-world scientific application proxies gain an execution time reduction as large as 14.60% in the 512 process-scale compared to the system default process placement on the TianHe-2 HPC systems.

AllScale toolchain pilot applications: PDE based solvers using a parallel development environment

AllScale is a programming environment targeting simplified development of highly scalable parallel applications by dividing development responsibilities into silos. The front-end AllScale API provides a simple C++ development environment through a suite of parallel constructs expressions denoting tasks operating concurrently. This interfaces with the other components of the toolchain (core-level API, compiler and runtime) which manages tasks related to the machine and system level, hidden to the user. The paper describes the development of two large-scale parallel applications within the AllScale API, namely, an advection– diffusion model with data assimilation and a Lagrangian space-weather simulation model based on a particle-in-cell method. We present mathematical formulations and implementations and evaluate parallel constructs developed using the AllScale API. The performance of the applications from the perspective of both parallel scalability, and more importantly productivity are assessed. We demonstrate how the AllScale API can greatly improve developer productivity while maintaining parallel performance in two applications with distinct numerical characteristics. Code complexity metrics demonstrate reduction in application specific implementations of up to 30% while performance tests on three different compute systems demonstrate comparable parallel scalability to an MPI version of the code.

Framework for parallelisation on big data.

The parallelisation of big data is emerging as an important framework for large-scale parallel data applications such as seismic data processing. The field of seismic data is so large or complex that traditional data processing software is incapable of dealing with it. For example, the implementation of parallel processing in seismic applications to improve the processing speed is complex in nature. To overcome this issue, a simple technique which that helps provide parallel processing for big data applications such as seismic algorithms is needed. In our framework, we used the Apache Hadoop with its MapReduce function. All experiments were conducted on the RedHat CentOS platform. Finally, we studied the bottlenecks and improved the overall performance of the system for seismic algorithms (stochastic inversion).

JAUMIN: a programming framework for large-scale numerical simulation on unstructured meshes

Domain-specific programming frameworks are usually effective to simplify the development of large-scale applications on supercomputers. This paper introduces a programming framework named JAUMIN for large-scale numerical applications on unstructured meshes. Based on this framework, serial codes are only required to be written for large-scale parallel application suitable for modern supercomputers with tens of thousands of CPU cores.

Hardware Cost Design Optimization for Functional Safety-Critical Parallel Applications on Heterogeneous Distributed Embedded Systems

Industrial embedded systems are cost sensitive, and hardware cost of industrial production should be reduced for high profit. The functional safety requirement must be satisfied according to industrial functional safety standards. This study proposes three hardware cost optimization algorithms for functional safety-critical parallel applications on heterogeneous distributed embedded systems during the design phase. The explorative hardware cost optimization (EHCO), enhanced EHCO (EEHCO), and simplified EEHCO (SEEHCO) algorithms are proposed step by step. Experimental results reveal that EEHCO can obtain minimum hardware cost, whereas SEEHCO is efficient for large-scale parallel applications compared with the existing algorithms.

Identifying the Root Causes of Wait States in Large-Scale Parallel Applications

Driven by growing application requirements and accelerated by current trends in microprocessor design, the number of processor cores on modern supercomputers is increasing from generation to generation. However, load or communication imbalance prevents many codes from taking advantage of the available parallelism, as delays of single processes may spread wait states across the entire machine. Moreover, when employing complex point-to-point communication patterns, wait states may propagate along far-reaching cause-effect chains that are hard to track manually and that complicate an assessment of the actual costs of an imbalance. Building on earlier work by Meira, Jr., et al., we present a scalable approach that identifies program wait states and attributes their costs in terms of resource waste to their original cause. By replaying event traces in parallel both forward and backward, we can identify the processes and call paths responsible for the most severe imbalances, even for runs with hundreds of thousands of processes.

Performance Prediction for Large-Scale Parallel Applications Using Representative Replay

Automatically predicting performance of parallel applications has been a long-standing goal in the area of high performance computing. However, accurate performance prediction is challenging, since the execution time of parallel applications is determined by several factors, such as sequential computation time, communication time and their complex interactions. Despite previous efforts, accurately estimating the sequential computation time in each process for large-scale parallel applications remains an open problem. In this paper, we propose a novel approach to acquiring accurate sequential computation time using a parallel debugging technique called deterministic replay. The main advantage of our approach is that we only need a single node of a target platform but the whole target platform does not need to be available. Therefore, with this approach we can simply measure the real sequential computation time on a target node for each process on by one. Moreover, we observe that there is great computation similarity in parallel applications, not only within each process but also among different processes. Based on this observation, we further propose representative replay that can significantly reduce replay overhead, because we only need to replay partial iterations for representative processes instead of all of them. Finally, we implement a complete performance prediction system, called Phantom , which combines the above computation-time acquisition approach and a trace-driven simulator. We validate our approach on both traditional HPC platforms and the latest Amazon EC2 cloud platform. On both types of platforms, prediction error of our approach is less than 7 percent on average up to 2,500 processes.

Plasmon enhanced optical tweezers with gold-coated black silicon

Plasmonic optical tweezers are a ubiquitous tool for the precise manipulation of nanoparticles and biomolecules at low photon flux, while femtosecond-laser optical tweezers can probe the nonlinear optical properties of the trapped species with applications in biological diagnostics. In order to adopt plasmonic optical tweezers in real-world applications, it is essential to develop large-scale fabrication processes without compromising the trapping efficiency. Here, we develop a novel platform for continuous wave (CW) and femtosecond plasmonic optical tweezers, based on gold-coated black silicon. In contrast with traditional lithographic methods, the fabrication method relies on simple, single-step, maskless tabletop laser processing of silicon in water that facilitates scalability. Gold-coated black silicon supports repeatable trapping efficiencies comparable to the highest ones reported to date. From a more fundamental aspect, a plasmon-mediated efficiency enhancement is a resonant effect, and therefore, dependent on the wavelength of the trapping beam. Surprisingly, a wavelength characterization of plasmon-enhanced trapping efficiencies has evaded the literature. Here, we exploit the repeatability of the recorded trapping efficiency, offered by the gold-coated black silicon platform, and perform a wavelength-dependent characterization of the trapping process, revealing the resonant character of the trapping efficiency maxima. Gold-coated black silicon is a promising platform for large-scale parallel trapping applications that will broaden the range of optical manipulation in nanoengineering, biology, and the study of collective biophotonic effects.

Validating the Simulation of Large-Scale Parallel Applications Using Statistical Characteristics

Simulation is a widely adopted method to analyze and predict the performance of large-scale parallel applications. Validating the hardware model is highly important for complex simulations with a large number of parameters. Common practice involves calculating the percent error between the projected and the real execution time of a benchmark program. However, in a high-dimensional parameter space, this coarse-grained approach often suffers from parameter insensitivity, which may not be known a priori . Moreover, the traditional approach cannot be applied to the validation of software models, such as application skeletons used in online simulations. In this work, we present a methodology and a toolset for validating both hardware and software models by quantitatively comparing fine-grained statistical characteristics obtained from execution traces. Although statistical information has been used in tasks like performance optimization, this is the first attempt to apply it to simulation validation. Our experimental results show that the proposed evaluation approach offers significant improvement in fidelity when compared to evaluation using total execution time, and the proposed metrics serve as reliable criteria that progress toward automating the simulation tuning process.

Debugging high-performance computing applications at massive scales

Dynamic analysis techniques help programmers find the root cause of bugs in large-scale parallel applications.

Static Analysis Techniques for Semiautomatic Synthesis of Message Passing Software Skeletons

The design of high-performance computing architectures requires performance analysis of large-scale parallel applications to derive various parameters concerning hardware design and software development. The process of performance analysis and benchmarking an application can be done in several ways with varying degrees of fidelity. One of the most cost-effective ways is to do a coarse-grained study of large-scale parallel applications through the use of program skeletons. The concept of a “program skeleton” that we discuss in this article is an abstracted program that is derived from a larger program where source code that is determined to be irrelevant is removed for the purposes of the skeleton. In this work, we develop a semiautomatic approach for extracting program skeletons based on compiler program analysis. We demonstrate correctness of our skeleton extraction process by comparing details from communication traces, as well as show the performance speedup of using skeletons by running simulations in the SST/macro simulator.

Diagnosis of Performance Faults in LargeScale MPI Applications via Probabilistic Progress-Dependence Inference

Debugging large-scale parallel applications is challenging. Most existing techniques provide little information about failure root causes. Further, most debuggers significantly slow down program execution, and run sluggishly with massively parallel applications. This paper presents a novel technique that scalably infers the tasks in a parallel program on which a failure occurred, as well as the code in which it originated. Our technique combines scalable runtime analysis with static analysis to determine the least-progressed task(s) and to identify the code lines at which the failure arose. We present a novel algorithm that infers probabilistically progress dependence among MPI tasks using a globally constructed Markov model that represents tasks’ control-flow behavior. In comparison to previous work, our algorithm infers more precisely the least-progressed task. We combine this technique with static backward slicing analysis, further isolating the code responsible for the current state. A blind study demonstrates that our technique isolates the root cause of a concurrency bug in a molecular dynamics simulation, which only manifests itself at 7,996 tasks or more. We extensively evaluate fault coverage of our technique via fault injections in 10 HPC benchmarks and show that our analysis takes less than a few seconds on thousands of parallel tasks.

On the impact of process replication on executions of large-scale parallel applications with coordinated checkpointing

Processor failures in post-petascale parallel computing platforms are common occurrences. The traditional fault-tolerance solution, checkpoint–rollback–recovery, severely limits parallel efficiency. One solution is to replicate application processes so that a processor failure does not necessarily imply an application failure. Process replication, combined with checkpoint–rollback–recovery, has been recently advocated. We first derive novel theoretical results for Exponential failure distributions, namely exact values for the Mean Number of Failures To Interruption and the Mean Time To Interruption. We then extend these results to arbitrary failure distributions, obtaining closed-form solutions for Weibull distributions. Finally, we evaluate process replication in simulation using both synthetic and real-world failure traces so as to quantify average application makespan. One interesting result from these experiments is that, when process replication is used, application performance is not sensitive to the checkpointing period, provided that period is within a large neighborhood of the optimal period. More generally, our empirical results make it possible to identify regimes in which process replication is beneficial.

Accurate application progress analysis for large-scale parallel debugging

Debugging large-scale parallel applications is challenging. In most HPC applications, parallel tasks progress in a coordinated fashion, and thus a fault in one task can quickly propagate to other tasks, making it difficult to debug. Finding the least-progressed tasks can significantly reduce the effort to identify the task where the fault originated. However, existing approaches for detecting them suffer low accuracy and large overheads; either they use imprecise static analysis or are unable to infer progress dependence inside loops. We present a loop-aware progress-dependence analysis tool, Prodometer, which determines relative progress among parallel tasks via dynamic analysis. Our fault-injection experiments suggest that its accuracy and precision are over 90% for most cases and that it scales well up to 16,384 MPI tasks. Further, our case study shows that it significantly helped diagnosing a perplexing error in MPI, which only manifested at large scale.

ELASTIC: A Large Scale Dynamic Tuning Environment

The spectacular growth in the number of cores in current supercomputers poses design challenges for the development of performance analysis and tuning tools. To be effective, such analysis and tuning tools must be scalable and be able to manage the dynamic behaviour of parallel applications. In this work, we present ELASTIC, an environment for dynamic tuning of large-scale parallel applications. To be scalable, the architecture of ELASTIC takes the form of a hierarchical tuning network of nodes that perform a distributed analysis and tuning process. Moreover, the tuning network topology can be configured to adapt itself to the size of the parallel application. To guide the dynamic tuning process, ELASTIC supports a plugin architecture. These plugins, called ELASTIC packages, allow the integration of different tuning strategies into ELASTIC. We also present experimental tests conducted using ELASTIC, showing its effectiveness to improve the performance of large-scale parallel applications.

ELASTIC: A Large Scale Dynamic Tuning Environment

The spectacular growth in the number of cores in current supercomputers poses design challenges for the development of performance analysis and tuning tools. To be effective, such analysis and tuning tools must be scalable and be able to manage the dynamic behaviour of parallel applications. In this work, we present ELASTIC, an environment for dynamic tuning of large- scale parallel applications. To be scalable, the architecture of ELASTIC takes the form of a hierarchical tuning network of nodes that perform a distributed analysis and tuning process. Moreover, the tuning network topology can be configured to adapt itself to the size of the parallel application. To guide the dynamic tuning process, ELASTIC supports a plugin architecture. These plugins, called ELASTIC packages, allow the integration of different tuning strategies into ELASTIC. We also present experimental tests conducted using ELASTIC, showing its effectiveness to improve the performance of large-scale parallel applications.

Improving Scalability of Application-Level Checkpoint-Recovery by Reducing Checkpoint Sizes

The execution times of large-scale parallel applications on nowadays multi/many-core systems are usually longer than the mean time between failures. Therefore, parallel applications must tolerate hardware failures to ensure that not all computation done is lost on machine failures. Checkpointing and rollback recovery is one of the most popular techniques to implement fault-tolerant applications. However, checkpointing parallel applications is expensive in terms of computing time, network utilization and storage resources. Thus, current checkpoint-recovery techniques should minimize these costs in order to be useful for large scale systems. In this paper three different and complementary techniques to reduce the size of the checkpoints generated by application-level checkpointing are proposed and implemented. Detailed experimental results obtained on a multicore cluster show the effectiveness of the proposed methods to reduce checkpointing cost.

Achieving Checkpointing Global Consistency Through a Hybrid Compile Time and Runtime Protocol

The execution times of large-scale parallel applications on modern multi/many-core systems are usually longer than their mean time between failures. Therefore, parallel applications must tolerate hardware failures to ensure that not all computation is lost on machine failures. Checkpointing and rollback recovery are very useful techniques to implement fault-tolerant applications. In parallel applications a checkpointing protocol is required to guarantee that individual checkpoints form a consistent global state. Coordinated approaches are the most popular solution to achieve global checkpointing consistency. However, their main drawback is their poor scalability due to the required runtime coordination. This work presents a new hybrid protocol that combines the detection of valid recovery lines at compile time with a light and asynchronous protocol at runtime to negotiate the closest valid recovery line. Experimental results prove the efficiency and scalability of the proposal.

Porting and scaling OpenACC applications on massively-parallel, GPU-accelerated supercomputers

An increasing number of massively-parallel supercomputers are based on heterogeneous node architectures combining traditional, powerful multicore CPUs with energy-efficient GPU accelerators. Such systems offer high computational performance with modest power consumption. As the industry trend of closer integration of CPU and GPU silicon continues, these architectures are a possible template for future exascale systems. Given the longevity of large-scale parallel HPC applications, it is important that there is a mechanism for easy migration to such hybrid systems. The OpenACC programming model offers a directive-based method for porting existing codes to run on hybrid architectures. In this paper, we describe our experiences in porting the Himeno benchmark to run on the Cray XK6 hybrid supercomputer. We describe the OpenACC programming model and the changes needed in the code, both to port the functionality and to tune the performance. Despite the additional PCIe-related overheads when transferring data from one GPU to another over the Cray Gemini interconnect, we find the application gives very good performance and scales well. Of particular interest is the facility to launch OpenACC kernels and data transfers asynchronously, which speeds the Himeno benchmark by 5%–10%. Comparing performance with an optimised code on a similar CPU-based system (using 32 threads per node), we find the OpenACC GPU version to be just under twice the speed in a node-for-node comparison. This speed-up is limited by the computational simplicity of the Himeno benchmark and is likely to be greater for more complicated applications.