High Performance Computing Applications Research Articles

A multi-aspect online tuning framework for HPC applications

Developing software applications for high-performance computing (HPC) requires careful optimizations targeting a myriad of increasingly complex, highly interrelated software, hardware and system components. The demands placed on minimizing energy consumption on extreme-scale HPC systems and the associated shift towards hete rogeneous architectures add yet another level of complexity to program development and optimization. As a result, the software optimization process is often seen as daunting, cumbersome and time-consuming by software developers wishing to fully exploit HPC resources. To address these challenges, we have developed the Periscope Tuning Framework (PTF), an online automatic integrated tuning framework that combines both performance analysis and performance tuning with respect to the myriad of tuning parameters available to today’s software developer on modern HPC systems. This work introduces the architecture, tuning model and main infrastructure components of PTF as well as the main tuning plugins of PTF and their evaluation.

Guest Editorial High Performance Computing (HPC) Applications for a More Resilient and Efficient Power Grid

The power grid has been evolving over the last 120 years, but it is seeing more changes in this decade and next than it has seen over the past century. In particular, the widespread deployment of intermittent renewable generation, smart loads and devices, hierarchical and distributed control technologies, phasor measurement units, energy storage, and widespread usage of electric vehicles will require fundamental changes in methods and tools for the operation and planning of the power grid. The resulting new dynamic and stochastic behaviors will demand the inclusion of more complexity in modeling the power grid. Solving such complex models in the traditional computing environment will be a major challenge. Along with the increasing complexity of power system models, the increasing complexity of smart grid data further adds to the prevailing challenges. In this environment, the myriad of smart sensors and meters in the power grid increase by multiple orders of magnitude, so do the volume and speed of the data. The information infrastructure will need to drastically change to support the exchange of enormous amounts of data as smart grid applications will need the capability to collect, assimilate, analyze and process the data, to meet real-time grid functions. High performance computing (HPC) holds the promise to enhance these functions, but it is a great resource that has not been fully explored and adopted for the power grid domain.

Task-based programming in COMPSs to converge from HPC to big data

Task-based programming has proven to be a suitable model for high-performance computing (HPC) applications. Different implementations have been good demonstrators of this fact and have promoted the acceptance of task-based programming in the OpenMP standard. Furthermore, in recent years, Apache Spark has gained wide popularity in business and research environments as a programming model for addressing emerging big data problems. COMP Superscalar (COMPSs) is a task-based environment that tackles distributed computing (including Clouds) and is a good alternative for a task-based programming model for big data applications. This article describes why we consider that task-based programming models are a good approach for big data applications. The article includes a comparison of Spark and COMPSs in terms of architecture, programming model, and performance. It focuses on the differences that both frameworks have in structural terms, on their programmability interface, and in terms of their efficiency by means of three widely known benchmarking kernels: Wordcount, Kmeans, and Terasort. These kernels enable the evaluation of the more important functionalities of both programming models and analyze different work flows and conditions. The main results achieved from this comparison are (1) COMPSs is able to extract the inherent parallelism from the user code with minimal coding effort as opposed to Spark, which requires the existing algorithms to be adapted and rewritten by explicitly using their predefined functions, (2) it is an improvement in terms of performance when compared with Spark, and (3) COMPSs has shown to scale better than Spark in most cases. Finally, we discuss the advantages and disadvantages of both frameworks, highlighting the differences that make them unique, thereby helping to choose the right framework for each particular objective.

THE FEASIBILITY STUDY OF RUNNING HPC WORKLOADS ON COMPUTATIONAL CLOUDS

High-performance computing (HPC) applications require high-end computing systems, but not all scientists have access to such powerful systems. Cloud computing provides an opportunity to run these applications on the cloud without the requirement of investing in high-end parallel computing systems. We can analyze the performance of the HPC applications on private as well as public clouds. The performance of the workload on the cloud can be calculated using different benchmarking tools such as NAS parallel benchmarking and Rally. The workloads of HPC applications require use of many parallel computing systems to be run on a physical setup, but this facility is available on cloud computing environment without the need of investing in physical machines. We aim to analyze the ability of the cloud to perform well when running HPC workloads. We shall get the detailed performance of the cloud when running these applications on a private cloud and find the pros and cons of running HPC workloads on cloud environment.

Evaluation of DVFS techniques on modern HPC processors and accelerators for energy‐aware applications

SummaryEnergy efficiency is becoming increasingly important for computing systems, in particular for large scale High Performance Computing (HPC) facilities. In this work, we evaluate, from a user perspective, the use of Dynamic Voltage and Frequency Scaling techniques, assisted by the power and energy monitoring capabilities of modern processors to tune applications for energy efficiency. We run selected kernels and a full HPC application on 2 high‐end processors widely used in the HPC context, namely, an NVIDIA K80 GPU and an Intel Haswell CPU. We evaluate the available trade‐offs between energy‐to‐solution and time‐to‐solution, attempting a function‐by‐function frequency tuning. We finally estimate the benefits obtainable running the full code on an HPC multi‐GPU node, with respect to default clock frequency governors. We instrument our code to accurately monitor power consumption and execution time without the need of any additional hardware, and we enable it to change CPUs and GPUs clock frequencies while running. We analyze our results on the different architectures using a simple energy‐performance model and derive a number of energy saving strategies, which can be easily adopted on recent high‐end HPC systems for generic applications.

Main Memory in HPC

An important aspect of High-Performance Computing (HPC) system design is the choice of main memory capacity. This choice becomes increasingly important now that 3D-stacked memories are entering the market. Compared with conventional Dual In-line Memory Modules (DIMMs), 3D memory chiplets provide better performance and energy efficiency but lower memory capacities. Therefore, the adoption of 3D-stacked memories in the HPC domain depends on whether we can find use cases that require much less memory than is available now. This study analyzes the memory capacity requirements of important HPC benchmarks and applications. We find that the High-Performance Conjugate Gradients (HPCG) benchmark could be an important success story for 3D-stacked memories in HPC, but High-Performance Linpack (HPL) is likely to be constrained by 3D memory capacity. The study also emphasizes that the analysis of memory footprints of production HPC applications is complex and that it requires an understanding of application scalability and target category, i.e., whether the users target capability or capacity computing. The results show that most of the HPC applications under study have per-core memory footprints in the range of hundreds of megabytes, but we also detect applications and use cases that require gigabytes per core. Overall, the study identifies the HPC applications and use cases with memory footprints that could be provided by 3D-stacked memory chiplets, making a first step toward adoption of this novel technology in the HPC domain.

A runtime estimation framework for ALICE

The European Organization for Nuclear Research (CERN) is the largest research organization for particle physics. ALICE, short for ALarge Ion Collider Experiment, serves as one of the main detectors at CERN and produces approximately 15 petabytes of data each year. The computing associated with an ALICE experiment consists of both online and offline processing. An online cluster retrieves data while an offline cluster farm performs a broad range of data analysis. Online processing occurs as collision events are streamed from the detector to the online cluster. This process compresses and calibrates the data before storing it in a data storage system for subsequent offline processing, e.g., event reconstruction. Due to the large volume of stored data to process, offline processing seeks to minimize execution time and data-staging time of the applications via a two-tier offline cluster — the Event Processing Node (EPN) as the first tier and the World LHC Grid Computing (WLGC) as the second tier. This two-tier cluster requires a smart job scheduler to efficiently manage the running of the application. Thus, we propose a runtime estimation method for this offline processing in the ALICE environment.Our approach exploits application profiles to predict the runtime of a high-performance computing (HPC) application without the need for any additional metadata. To evaluate our proposed framework, we performed our experiment on the actual ALICE applications. In addition, we also test the efficacy of our runtime estimation method to predict the run times of the HPC applications on the Amazon EC2 cloud. The results show that our approach generally delivers accurate predictions, i.e., low error percentages.

Optical packet switching in HPC. An analysis of applications performance

Optical Packet Switches (OPS) could provide the needed low latency transmissions in today large data centers. OPS can deliver lower latency and higher bandwidth than traditional electrical switches. These features are needed for parallel High Performance Computing (HPC) applications. For this purpose, it has been recently designed full optical network architectures for HPC system such as the Architecture-On-Demand (AoD) network infrastructure. Although light-based transmission has its advantage over electrical-based transmissions, optical devices such as OPS cannot store light. Therefore, in case of an optical packet collision occurs for accessing the same output port in OPS only one packet can proceed and the others must be dropped, triggering afterwards a retransmission procedure. Obviously, packet retransmissions are delaying the actual transmission and also increase the buffer utilization at the network interfaces cards (NICs) that deals with retransmissions. In this paper, it is proposed a technique based on mapping application processes to servers reducing the number of simultaneous packets in the network (Concurrency) and therefore, it is able to significantly reduce optical collisions at the OPS while substantially reduces the resource needed at the NICs for retransmissions. Our proposed concurrency-aware mapping technique can reduce the extra buffer size utilization up to 4.2 times and the execution time degradation up to 2.6 times.

RedThreads: An Interface for Application-Level Fault Detection/Correction Through Adaptive Redundant Multithreading

In the presence of accelerated fault rates, which are projected to be the norm on future exascale systems, it will become increasingly difficult for high-performance computing (HPC) applications to accomplish useful computation. Due to the fault-oblivious nature of current HPC programming paradigms and execution environments, HPC applications are insufficiently equipped to deal with errors. We believe that HPC applications should be enabled with capabilities to actively search for and correct errors in their computations. The redundant multithreading (RMT) approach offers lightweight replicated execution streams of program instructions within the context of a single application process. However, the use of complete redundancy incurs significant overhead to the application performance. In this paper we present RedThreads, an interface that provides application-level fault detection and correction based on RMT, but applies the thread-level redundancy adaptively. We describe the RedThreads syntax and semantics, and the supporting compiler infrastructure and runtime system. Our approach enables application programmers to scope the extent of redundant computation. Additionally, the runtime system permits the use of RMT to be dynamically enabled, or disabled, based on the resiliency needs of the application and the state of the system. Our experimental results demonstrate how adaptive RMT exploits programmer insight and runtime inference to dynamically navigate the trade-off space between an application’s resilience coverage and the associated performance overhead of redundant computation.

The READEX formalism for automatic tuning for energy efficiency

Energy efficiency is an important aspect of future exascale systems, mainly due to rising energy cost. Although High performance computing (HPC) applications are compute centric, they still exhibit varying computational characteristics in different regions of the program, such as compute-, memory-, and I/O-bound code regions. Some of today’s clusters already offer mechanisms to adjust the system to the resource requirements of an application, e.g., by controlling the CPU frequency. However, manually tuning for improved energy efficiency is a tedious and painstaking task that is often neglected by application developers. The European Union’s Horizon 2020 project READEX (Runtime Exploitation of Application Dynamism for Energy-efficient eXascale computing) aims at developing a tools-aided approach for improved energy efficiency of current and future HPC applications. To reach this goal, the READEX project combines technologies from two ends of the compute spectrum, embedded systems and HPC, constituting a split design-time/runtime methodology. From the HPC domain, the Periscope Tuning Framework (PTF) is extended to perform dynamic auto-tuning of fine-grained application regions using the systems scenario methodology, which was originally developed for improving the energy efficiency in embedded systems. This paper introduces the concepts of the READEX project, its envisioned implementation, and preliminary results that demonstrate the feasibility of this approach.

Marcher: A Heterogeneous System Supporting Energy-Aware High Performance Computing and Big Data Analytics

Abstract Excessive energy consumption is a major constraint in designing and deploying the next generation of supercomputers. Minimizing energy consumption of high performance computing and big data applications requires novel energy-conscious technologies (both hardware and software) at multiple layers from architecture, system support, and applications. In the past decade, we have witnessed the significant progress toward developing more energy-efficient hardware and facility infrastructure. However, the energy efficiency of software has not been improved much. One obstacle that hinders the exploration of green software technologies is the lack of tools and systems that can provide accurate, fine-grained, and real-time power and energy measurement for technology evaluation and verification. Marcher, a heterogeneous high performance computing infrastructure, is built to fill the gap by providing support to research in energy-aware high performance computing and big data analytics. The Marcher system is equipped with Intel Xeon CPUs, Intel Many Integrated Cores (Xeon Phi), Nvidia GPUs, power-aware memory systems and hybrid storage with Hard Disk Drives (HDDs) and Solid State Disks (SSDs). It provides easy-to-use tools and interfaces for researchers to obtain decomposed and fine-grained power consumption data of these primary computing components. This paper presents the design of the Marcher system and demonstrates the usage of Marcher power measurement tools to obtain detailed power consumption data in various research projects.

On providing on-the-fly resizing of the elasticity grain when executing HPC applications in the cloud

Today, we observe that cloud infrastructures are gaining more and more space to execute high-performance computing (HPC) applications. Unlike clusters and grids, the cloud offers elasticity, which refers to the ability to increase or reduce the number of resources (and consequently, processes) to support, as closely as possible, the needs at a particular moment of the execution. To the best of our knowledge, current initiatives explore the elasticity for HPC applications by always handling the same number of resources at each scaling-in or scaling-out operation. This fixed elasticity grain commonly results in a stair-shaped behaviour, where successive elasticity operations take place to address the load curve. In this context, this article presents GrainElastic, an elasticity model for executing HPC applications with the capacity to adapt the elasticity grain to the requirements of each elasticity operation. Its contribution involves a mathematical formalism that uses historical execution traces and the ARIMA time-series model to predict the required number of resources (in our case, VMs) to address a reconfiguration point. Based on the proposed model, we developed a prototype that was compared with two other scenarios: 1) non-elastic application; 2) elastic middleware with a fixed grain. The results presented gains of up to 30% using GrainElastic, which demonstrates the relevance of adapting the elasticity grain to enhance system reactivity and performance.

Impact of using multi-levels of parallelism on HPC applications performance hosted on Azure cloud computing

The use of high-performance computing (HPC) applications has increased progressively in scientific research and industry. Cloud computing attracts HPC users because of its extreme cost efficiency. The reduced cost is the result of the successful employment of multilayer-virtualisation enabling dynamic elastic resource-sharing between different tenants. In this paper, we evaluate the impact of using multi-levels of parallelism on computationally intensive parallel tasks hosted on a cloud virtualised HPC cluster. We use multi-levels of parallelism through a set of experiments employing both message passing and multi-threading techniques. Our evaluation addresses two main perspectives, the performance of applications and cost of running HPC applications on clouds. We use millions of operations per seconds (MOPS) and speed-up to evaluate the computational performance. To evaluate the cost we use United States Dollar/MOPS (USD/MOPS). The experiments on two different clouds are compared against each other and with published results for Amazon EC2 cloud. Results show that balancing the workload between processes and threads per process is the key factor to maintain high performance with reasonable cost.

MCM: A new MPI Communication Management for Cloud Environments

On-demand Cloud Computing offers an attractive new dimension to High Performance Computing (HPC). Many HPC applications are moving to clouds due to some characteristics of this environment, like elasticity, pay per use, and the maintenance cost. HPC applications normally use Message Passing Interface (MPI). However, for this kind of applications on cloud environments, network performance remains a challenge due to the effects of network virtualization. To use a cloud environment with scientific applications of this kind, low latency communication mechanisms are required, and the hidden network topology information makes difficult to use the existing optimizations based on network topology information. In this paper a method named MPI Communication Management (MCM) is presented. This method characterizes the underlying network topology and analyzes parallel applications behavior in the cloud, improving the application’s messages latency time. MCM achieves lower application execution time in case of congestion, obtaining better performance in clouds. Finally, experiments verify the functionality and improvements of MCM with MPI Applications in Amazon EC2 public cloud.

An Integrated Tutorial on InfiniBand, Verbs, and MPI

This tutorial presents the details of the interconnection network utilized in many high performance computing (HPC) systems today. “InfiniBand” is the hardware interconnect utilized by over 35% of the top 500 supercomputers in the world as of June, 2017. “Verbs” is the term used for both the semantic description of the interface in the InfiniBand architecture specifications, and the name used for the functions defined in the widely used OpenFabrics alliance implementation of the software interface to InfiniBand. “Message passing interface” is the primary software library by which HPC applications portably pass messages between processes across a wide range of interconnects including InfiniBand. Our goal is to explain how these three components are designed and how they interact to provide a powerful, efficient interconnect for HPC applications. We provide a succinct look into the inner technical workings of each component that should be instructive to both novices to HPC applications as well as to those who may be familiar with one component, but not necessarily the others, in the design and functioning of the total interconnect. A supercomputer interconnect is not a monolithic structure, and this tutorial aims to give non-experts a “big-picture” overview of its substructure with an appreciation of how and why features in one component influence those in others. We believe this is one of the first tutorials to discuss these three major components as one integrated whole. In addition, we give detailed examples of practical experience and typical algorithms used within each component in order to give insights into what issues and trade-offs are important.

Porting the MPI-parallelized LES model PALM to multi-GPU systems and many integrated core processors: an experience report

The computational power and availability of graphics processing units (GPUs) and many integrated core (MIC) processors on high performance computing (HPC) systems is rapidly evolving. However, HPC applications need to be ported to take advantage of such hardware. This paper is a report on our experience of porting the MPI+OpenMP parallelised large-eddy simulation model (PALM) to multi-GPU as well as to MIC processor environments using OpenACC and OpenMP. PALM is written in Fortran, entails 140 kLOC and runs on HPC farms of up to 43,200 cores. The main porting challenges are the size and complexity of PALM, its inconsistent modularisation and no unit-tests. We report the methods used to identify performance issues as well as our experiences with state-of-the-art profiling tools. Moreover, we outline the required porting steps, describe the problems and bottlenecks we encountered and present separate performance tests for both architectures. We however, do not provide benchmark information.

Performance analysis of Linux containers for high performance computing applications

Although cloud infrastructures can be used as High Performance Computing (HPC) platforms, many issues from virtualisation overhead have kept them almost unrelated. However, with advent of container-based virtualisation, this scenario acquires new perspectives because this technique promises to decrease the virtualisation overhead, achieving a near-native performance. In this work, we analyse the performance of a container-based virtualisation solution - Linux Container (LXC) - against a hypervisor-based virtualisation solution - KVM - under HPC activities. For our experiments, we consider CPU and (network and inter-process) communication performance. Results show that hypervisor type can impact distinctly in performance according to resource used by HPC application.

Application of Neural Network Technology and High-performance Computing for Identification and Real-time Hardware-in-the-loop Simulation of Gas Turbine Engines

The engineering method for the recurrent neural network construction and identification of a mathematical model of gas turbine engines on a real data is proposed, describing the learning algorithm and the network structure. The complete process of modeling and experimental investigation – from designing of a gas turbines model in form of neural networks to its testing and debugging on the test-bed – are presented. The method was approved on a hardware-in-the-loop test-bed with a FADEC closed loop control for the start-up, ground and flight modes.

Acceleration by Inline Cache for Memory-Intensive Algorithms on FPGA via High-Level Synthesis

Using FPGA-based acceleration of high-performance computing (HPC) applications to reduce energy and power consumption is becoming an interesting option, thanks to the availability of high-level synthesis (HLS) tools that enable fast design cycles. However, obtaining good performance for memory-intensive algorithms, which often exchange large data arrays with external DRAM, still requires time-consuming optimization and good knowledge of hardware design. This article proposes a new design methodology, based on dedicated application- and data array-specific caches. These caches provide most of the benefits that can be achieved by coding optimized DMA-like transfer strategies by hand into the HPC application code, but require only limited manual tuning (basically the selection of architecture and size), are neutral to target HLS tool and technology (FPGA or ASIC), and do not require changes to application code. We show experimental results obtained on five common memory-intensive algorithms from very diverse domains, namely machine learning, data sorting, and computer vision. We test the cost and performance of our caches against both out-of-the-box code originally optimized for a GPU, and manually optimized implementations specifically targeted for FPGAs via HLS. The implementation using our caches achieved an 8X speedup and 2X energy reduction on average with respect to out-of-the-box models using only simple directive-based optimizations (e.g., pipelining). They also achieved comparable performance with much less design effort when compared with the versions that were manually optimized to achieve efficient memory transfers specifically for an FPGA.

Experimental and Analytical Analysis of Sorting Algorithms Error Criticality for HPC and Large Servers Applications

In this paper, we investigate neutron-induced errors in three implementations of sort algorithms (QuickSort, MergeSort, and RadixSort) executed on modern graphics processing units designed for high-performance computing and large server applications. We measure the radiation-induced error rate of sort algorithms taking advantage of the neutron beam available at the Los Alamos Neutron Science Center facility. We also analyze output error criticality by identifying specific output error patterns. We found that radiation can cause wrong elements to appear in the sorted array, misalign values as well as application crashes or system hangs. This paper presents results showing that the criticality of the radiation-induced output error pattern depends on the application. Additionally, an extensive fault-injection campaign has been performed. This campaign allows for better understanding of the observed phenomena. We take advantage of SASS-assembly Intrumentator Fault Injector developed by NVIDIA, which can inject faults into all the user-accessible architectural state. Comparing fault-injection results with radiation experiments data provides an understanding that not all the output errors observed under radiation can be replicated in fault injection. However, fault injection is useful in identifying possible root causes of the output errors observed in radiation testing. Finally, we take advantage of our experimental and analytical study to design efficient experimentally tuned hardening strategies. We detect the error patterns that are critical to the final application and find the more efficient way to detect them. With an overhead as low as 16% of the execution time, we are able to reduce the output error rate of sort of about one order of magnitude.