Heterogeneous Many-core Systems Research Articles

Exploring Multi-Reader Buffers and Channel Placement During Dataflow Network Mapping to Heterogeneous Many-Core Systems

Exploring Multi-Reader Buffers and Channel Placement During Dataflow Network Mapping to Heterogeneous Many-Core Systems

Task Migration Policy for Thermal-Aware Dynamic Performance Optimization in Many-Core Systems

The steady downsizing of semiconductor technology nodes in recent years has led to a rapid increase in the density of power consumption on chips which, in turn, renders <i>temperature</i> a major issue for many-core systems, adversely affecting their performance, reliability, leakage, cost, etc. In this context, <i>task migration</i> is a powerful technique that is widely used for controlling the temperature profile of many-core systems under dynamic workloads with the goal to improve their performance, utilization, reliability, etc. In this paper, we present a task migration policy for thermal-aware performance optimization in heterogeneous many-core systems. The proposed policy is developed based on an analytical and thermally safe power-budgeting scheme and uses Dynamic Voltage and Frequency Scaling (DVFS) for power and thermal management of the system. Our migration policy aims at maximizing the system&#x2019;s performance and, at the same time, proactively enforcing thermal safety using DVFS. To that end, it iteratively adapts the distribution of active cores in the system (through proper migration decisions) to maximize the thermally safe power budget of active cores and, thereby, enable them to operate on higher frequencies without violating their safe thermal threshold. Experimental results demonstrate that the proposed policy offers <inline-formula> <tex-math notation="LaTeX">$2\times $ </tex-math></inline-formula> higher performance gain in comparison to existing approaches which aim at greedily reducing the average, variance, or gradient of temperature as an indirect means to enhance performance.

Gem5-X

The increasing adoption of smart systems in our daily life has led to the development of new applications with varying performance and energy constraints, and suitable computing architectures need to be developed for these new applications. In this article, we present gem5-X, a system-level simulation framework, based on gem-5, for architectural exploration of heterogeneous many-core systems. To demonstrate the capabilities of gem5-X, real-time video analytics is used as a case-study. It is composed of two kernels, namely, video encoding and image classification using convolutional neural networks (CNNs). First, we explore through gem5-X the benefits of latest 3D high bandwidth memory (HBM2) in different architectural configurations. Then, using a two-step exploration methodology, we develop a new optimized clustered-heterogeneous architecture with HBM2 in gem5-X for video analytics application. In this proposed clustered-heterogeneous architecture, ARMv8 in-order cluster with in-cache computing engine executes the video encoding kernel, giving 20% performance and 54% energy benefits compared to baseline ARM in-order and Out-of-Order systems, respectively. Furthermore, thanks to gem5-X, we conclude that ARM Out-of-Order clusters with HBM2 are the best choice to run visual recognition using CNNs, as they outperform DDR4-based system by up to 30% both in terms of performance and energy savings.

Optimizing high-resolution Community Earth System Model on a heterogeneous many-core supercomputing platform

Abstract. With semiconductor technology gradually approaching its physical and thermal limits, recent supercomputers have adopted major architectural changes to continue increasing the performance through more power-efficient heterogeneous many-core systems. Examples include Sunway TaihuLight that has four management processing elements (MPEs) and 256 computing processing elements (CPEs) inside one processor and Summit that has two central processing units (CPUs) and six graphics processing units (GPUs) inside one node. Meanwhile, current high-resolution Earth system models that desperately require more computing power generally consist of millions of lines of legacy code developed for traditional homogeneous multicore processors and cannot automatically benefit from the advancement of supercomputer hardware. As a result, refactoring and optimizing the legacy models for new architectures become key challenges along the road of taking advantage of greener and faster supercomputers, providing better support for the global climate research community and contributing to the long-lasting societal task of addressing long-term climate change. This article reports the efforts of a large group in the International Laboratory for High-Resolution Earth System Prediction (iHESP) that was established by the cooperation of Qingdao Pilot National Laboratory for Marine Science and Technology (QNLM), Texas A&amp;amp;M University (TAMU), and the National Center for Atmospheric Research (NCAR), with the goal of enabling highly efficient simulations of the high-resolution (25 km atmosphere and 10 km ocean) Community Earth System Model (CESM-HR) on Sunway TaihuLight. The refactoring and optimizing efforts have improved the simulation speed of CESM-HR from 1 SYPD (simulation years per day) to 3.4 SYPD (with output disabled) and supported several hundred years of pre-industrial control simulations. With further strategies on deeper refactoring and optimizing for remaining computing hotspots, as well as redesigning architecture-oriented algorithms, we expect an equivalent or even better efficiency to be gained on the new platform than traditional homogeneous CPU platforms. The refactoring and optimizing processes detailed in this paper on the Sunway system should have implications for similar efforts on other heterogeneous many-core systems such as GPU-based high-performance computing (HPC) systems.

Runtime Task Scheduling Using Imitation Learning for Heterogeneous Many-Core Systems

Domain-specific systems-on-chip, a class of heterogeneous many-core systems, are recognized as a key approach to narrow down the performance and energy-efficiency gap between custom hardware accelerators and programmable processors. Reaching the full potential of these architectures depends critically on optimally scheduling the applications to available resources at runtime. Existing optimization-based techniques cannot achieve this objective at runtime due to the combinatorial nature of the task scheduling problem. As the main theoretical contribution, this paper poses scheduling as a classification problem and proposes a hierarchical imitation learning (IL)-based scheduler that learns from an Oracle to maximize the performance of multiple domain-specific applications. Extensive evaluations with six streaming applications from wireless communications and radar domains show that the proposed IL-based scheduler approximates an offline Oracle policy with more than 99% accuracy for performance- and energy-based optimization objectives. Furthermore, it achieves almost identical performance to the Oracle with a low runtime overhead and successfully adapts to new applications, many-core system configurations, and runtime variations in application characteristics.

Aging-Resilient Topology Synthesis of Heterogeneous Manycore Network-On-Chip Using Genetic Algorithm with Flexible Number of Routers

As semiconductor processes enter the nanoscale, system-on-chip (SoC) interconnects suffer from link aging owing to negative bias temperature instability (NBTI), hot carrier injection (HCI), and electromigration. In network-on-chip (NoC) for heterogeneous manycore systems, there is a difference in the aging speed of links depending on the location and utilization of resources. In this paper, we propose a heterogeneous manycore NoC topology synthesis that predicts the aging effect of each link and deploys routers and error correction code (ECC) logic. Aging-aware ECC logic is added to each link to achieve the same link lifetime with less area and latency than the Bose-Chaudhuri-Hocquenghem (BCH) logic. Moreover, based on the modified genetic algorithm, we search for a solution that minimizes the average latency while ensuring the link lifetime by changing the number of routers, location, and network connectivity. Simulation results demonstrate that the aging-aware topology synthesis reduces the average latency of the network by up to 26.68% compared with the aging analysis and the addition of ECC logic on the link after the topology synthesis. Furthermore, topology synthesis with aging-aware ECC logic reduces the maximum average latency by up to 39.49% compared with added BCH logic.

Energy-Efficient Thread Mapping for Heterogeneous Many-Core Systems via Dynamically Adjusting the Thread Count

Improving computing performance and reducing energy consumption are a major concern in heterogeneous many-core systems. The thread count directly influences the computing performance and energy consumption for a multithread application running on a heterogeneous many-core system. For this work, we studied the interrelation between the thread count and the performance of applications to improve total energy efficiency. A prediction model of the optimum thread count, hereafter the thread count prediction model (TCPM), was designed by using regression analysis based on the program running behaviors and heterogeneous many-core architecture feature. Subsequently, a dynamic predictive thread mapping (DPTM) framework was proposed. DPTM uses the prediction model to estimate the optimum thread count and dynamically adjusts the number of active hardware threads according to the phase changes of the running program in order to achieve the optimal energy efficiency. Experimental results show that DPTM obtains a nearly 49% improvement in performance and a 59% reduction in energy consumption on average. Moreover, DPTM introduces about 2% additional overhead compared with traditional thread mapping for PARSEC (The Princeton Application Repository for Shared-Memory Computers) benchmark programs running on an Intel MIC (Many integrated core) heterogeneous many-core system.

Towards high performance data analytic on heterogeneous many-core systems: A study on Bayesian Sequential Partitioning

Bayesian Sequential Partitioning (BSP) is a statistically effective density estimation method to comprehend the characteristics of a high dimensional data space. The intensive computation of the statistical model and the counting of enormous data have caused serious design challenges for BSP to handle the growing volume of the data. This paper proposes a high performance design of BSP by leveraging a heterogeneous CPU/GPGPU system that consists of a host CPU and a K80 GPGPU. A series of techniques, on both data structures and execution management policies, is implemented to extensively exploit the computation capability of the heterogeneous many-core system and alleviate system bottlenecks. When compared with a parallel design on a high-end CPU, the proposed techniques achieve 48x average runtime enhancement while the maximum speedup can reach 78.76x.

Speedup and Power Scaling Models for Heterogeneous Many-Core Systems

Traditional speedup models, such as Amdahl’s law, Gustafson’s, and Sun and Ni’s, have helped the research community and industry better understand system performance capabilities and application parallelizability. As they mostly target homogeneous hardware platforms or limited forms of processor heterogeneity, these models do not cover newly emerging multi-core heterogeneous architectures. This paper reports on novel speedup and energy consumption models based on a more general representation of heterogeneity, referred to as the normal form heterogeneity, that supports a wide range of heterogeneous many-core architectures. The modelling method aims to predict system power efficiency and performance ranges, and facilitates research and development at the hardware and system software levels. The models were validated through extensive experimentation on the off-the-shelf big.LITTLE heterogeneous platform and a dual-GPU laptop, with an average error of 1 percent for speedup and of less than 6.5 percent for power dissipation. A quantitative efficiency analysis targeting the system load balancer on the Odroid XU3 platform was used to demonstrate the practical use of the method.

On-Chip Communication Network for Efficient Training of Deep Convolutional Networks on Heterogeneous Manycore Systems

Convolutional Neural Networks (CNNs) have shown a great deal of success in diverse application domains including computer vision, speech recognition, and natural language processing. However, as the size of datasets and the depth of neural network architectures continue to grow, it is imperative to design high-performance and energy-efficient computing hardware for training CNNs. In this paper, we consider the problem of designing specialized CPU-GPU based heterogeneous manycore systems for energy-efficient training of CNNs. It has already been shown that the typical on-chip communication infrastructures employed in conventional CPU-GPU based heterogeneous manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. To address this issue, we first analyze the on-chip traffic patterns that arise from the computational processes associated with training two deep CNN architectures, namely, LeNet and CDBNet, to perform image classification. By leveraging this knowledge, we design a hybrid Network-on-Chip (NoC) architecture, which consists of both wireline and wireless links, to improve the performance of CPU-GPU based heterogeneous manycore platforms running the above-mentioned CNN training workloads. The proposed NoC achieves 1.8x reduction in network latency and improves the network throughput by a factor of 2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC. For the considered CNN workloads, these network-level improvements translate into 25% savings in full-system energy-delay-product (EDP). This demonstrates that the proposed hybrid NoC for heterogeneous manycore architectures is capable of significantly accelerating training of CNNs while remaining energy-efficient.

FoToNoC: A Folded Torus-Like Network-on-Chip Based Many-Core Systems-on-Chip in the Dark Silicon Era

Dark silicon refers to the phenomenon that a fraction of a many-core chip has to become “dark” or “dim” in order to guarantee the system to be kept in a safe temperature range and allowable power budget. Techniques have been developed to selectively activate non-adjacent cores on many-core chip to avoid temperature hotspot, while resulting unexpected increase of communication overhead due to the longer average distance between active cores, and in turn affecting application performance and energy efficiency, when Network-on-Chip (NoC) is used as a scalable communication subsystem. To address the brand-new challenges brought by dark silicon, in this paper, we present FoToNoC, a Folded Torus-like NoC, coupled with a hierarchical management strategy for heterogeneous many-core systems. On top of it, objectives of maximizing application performance, energy efficiency and chip reliability are isolated and well achieved by hardware-software co-design in several different phases, including application mapping and scheduling, cluster management and DVFS control. Evaluations on PARSEC benchmark applications demonstrate the significance of the entire strategy. Compared with state-of-the-art approaches, the proposed FoToNoC organization can achieve on average 35.4 and 35.2 percent on communication efficiency and application performance improvement, respectively, when maintaining the safe chip temperature. The hierarchical cluster-based management strategy can further reduce an average 34.6 percent of the total energy consumption with a notable reduction on the chip peak temperature. The significant achievements on system energy efficiency and the reduction on chip temperature of H.264 decoder and DSP-stone benchmarks additionally verify the effectiveness of the proposed methods.

Thermal Safe Power (TSP): Efficient Power Budgeting for Heterogeneous Manycore Systems in Dark Silicon

Chip manufacturers provide the Thermal Design Power (TDP) for a specific chip. The cooling solution is designed to dissipate this power level. But because TDP is not necessarily the maximum power that can be applied, chips are operated with Dynamic Thermal Management (DTM) techniques. To avoid excessive triggers of DTM, usually, system designers also use TDP as power constraint. However, using a single and constant value as power constraint, e.g., TDP, can result in significant performance losses in homogeneous and heterogeneous manycore systems. Having better power budgeting techniques is a major step towards dealing with the dark silicon problem. This paper presents a new power budget concept, called Thermal Safe Power (TSP), which is an abstraction that provides safe power and power density constraints as a function of the number of simultaneously active cores. Executing cores at any power consumption below TSP ensures that DTM is not triggered. TSP can be computed offline for the worst cases, or online for a particular mapping of cores. TSP can also serve as a fundamental tool for guiding task partitioning and core mapping decisions, specially when core heterogeneity or timing guarantees are involved. Moreover, TSP results in dark silicon estimations which are less pessimistic than estimations using constant power budgets.

Architecture, Languages, Compilation and Hardware support for Emerging ManYcore systems (ALCHEMY): Preface

Manycore systems are one of the key enabler technologies for most of current computational paradigms, including Internet of things, data-centers on chip and big data processing. These paradigms are characterized by tight and demanding requirements such as code portability, dynamicity, high performance, usability, predictability, reliability, low power and security.This combination of requirements has led to heterogeneous manycore systems which are extremely challenging to design and to program. As a result, a large body of research has focused on development of languages, simulation environments and analysis tools that allow to model and predict the behavior of this type of systems from early design stages.ALCHEMY 2017 presents five research works addressing the challenges of code portability, high performance, usability, security and reliability in manycore systems. These are namely:1. ”An OpenMP backend for the Sigma-C streaming language, addresses the software portability challenge of manycore architectures. It proposes an implementation of an OpenMP backend for the SigmaC language, a cycle-static data flow abstraction to program many-core embedded platforms. Its compilation scheme allows for utilization of future manycore embedded systems such as Kalray’s MPPA.2. A multi-level optimization strategy to improve the performance of the stencil computation combines manual vectorization, space tiling and stencil composition for achieving high performance of stencil kernels on manycore systems. The evaluation with three compilers (Intel, Clang and GCC) and two target multi-core platforms (Intel Broadwell and Ivybridge) reports better results compared to the state of the art.3. A Distributed Shared Memory Model and C++ Templated Meta-Programming Interface for the Epiphany RISC Array Processor addresses the usability challenge. It proposes techniques for data layout and parallel loop order abstraction as a parallel programming API targeting the Epiphany architecture. This results into a transparent distributed shared memory (DSM) model for Epiphany that eliminates the need to manage local data movement between cores.4. Towards Protected MPSoC Communication for Information Protection against a Malicious NoC deals with vulnerabilities on Network-on-Chip (NoC). The authors propose a security protocol which allows the secure communication among the cores of the system, even in the presence of Trojan insertions at the NoC whose aim is to modify and steal data.5. GPU-Accelerated Real-Time Path Planning and the Predictable Execution Model addresses the reliability challenge and tackles the important problem of ensuring reliable Worst Case Execution Time for Real-Time and Cyber Physical Systems. While considering heterogeneous (CPU/GPU), the idea is to separate memory and processor operations through Time-Division Multiplexing (TDM).

Hardware-software collaboration for dark silicon heterogeneous many-core systems

In dark silicon many-core systems, system-level techniques have been developed to selectively activate nonadjacent cores in physical locations to maintain the allowable power budget and eliminate thermal hotspot. However, this will unexpectedly increase communication overhead due to the longer average distance between active cores in a typical mesh-based Network-on-Chip (NoC), and in turn reduce application performance. Inspired by ARM’s big.LITTLE architecture coupled with Heterogeneous Multi-Processing (HMP) which enables energy-efficient solutions, in this paper, we propose two alternative hardware–software collaborated techniques to address the temperature/communication conflict in the dark silicon era. A Folded Torus-like NoC, FoToNoC, is presented to address the Cluster-switching based heterogeneous system, and a Quad-core-group based NoC, QcNoC, is presented to address the In-kernel switcher based heterogeneous system. Matched management strategies are accordingly proposed to collaboratively solve the trade-offs on inter-core communication, application performance, chip temperature, and system energy-efficiency in different design phases and aspects. Experimental results show that, FoToNoC can improve as large as 36.85% on application performance, save up to 30.35% on system energy consumption and reduce chip peak temperature by 4.11∘C on average compared with traditional techniques. Furthermore, due to the heterogeneous core organization and the fine-grained optimization on energy-efficiency in QcNoC, averagely 18.40% reduction on system energy consumption can be achieved with an average 2.24∘C reduction on chip peak temperature compared with that in FoToNoC. We further demonstrate the practicality and the effectiveness of the proposed methods by case studies on real-world applications including the industry standard H.264 decoder and DSP-stone benchmarks.

FIPIP: A novel fine-grained parallel partition based intra-frame prediction on heterogeneous many-core systems

Intra-frame prediction is an important time-consuming component of the widely used H.264/AVC encoder. To speed up prediction, one promising direction is to introduce parallelism and there have been many heterogeneous many-core based approaches proposed. But most of these approaches are limited by their use of highly irregular prediction formulas, which require significant amount of branch instructions. They only use coarse-grained parallel partition, which considers blocks or sub-region of images as parallel processing units. In this paper, by contrast, we propose a fine-grained intra-frame prediction approach based on parallel partition (FIPIP) and implement it on Graphics Processing Unit (GPU) based heterogeneous many-core systems. The approach is characterized by the following aspects. First, our approach takes individual pixels as parallel processing units, instead of blocks. Imposing pixel-level parallelism is capable of fully exploiting the computational power of heterogeneous GPU-based systems and hence tremendously reduces the encoding time. Second, we unify irregular prediction formulas in intra-frame prediction into a well-designed uniform one, and propose a table-lookup method to efficiently perform intra-frame prediction. Our formula can eliminate unnecessary branch instructions by using a unified predictor array, which improves the efficiency of the fine-grained parallel partition significantly. Third, two optimized encoding orders assisted by an improved combined frame strategy are adopted to implement multi-level parallelism. Finally, an efficient self-synchronizing method is realized for fine-grained task scheduling on heterogeneous CPU–GPU architecture. We apply FIPIP to encode a set of benchmark videos under varying conditions and compare it with other popular intra-frame prediction methods. Results show that FIPIP outperforms existing state-of-the-art work with speedups factor of 2–6.

Optimistic semaphores with non‐deterministic choice operation for heterogeneous manycore systems

SummaryThe Nix operating system permits different roles to be assigned to the cores. One of the roles is to be able to run user‐space code with no interrupts from the operating system, which is particularly useful for high‐performance computing. System calls are especially expensive to a core playing this role. This paper presents a new implementation of optimistic semaphores that avoid performing expensive system calls in an uncontended scenario. The implementation is straightforward and somewhat unorthodox: a semaphore is based on a data structure and a lock that are shared between user space and kernel space. This study aims at evaluating if such an approach is viable. In addition, the implementation includes a non‐deterministic choice operation over a collection of semaphores, altsems. This novel operation facilitates the creation of higher‐level communication mechanisms, such as sockets and channels. To support this claim, we implemented a new kind of buffered communication channels named tubes, tailored for communicating processes running on different (heterogeneous) cores. The paper describes the implementation of the semaphores and the tubes, a comparative analysis of optimistic and non‐optimistic semaphores on Nix, and a comparative analysis of tubes and other kinds of communication channels that are available on the Nix operating system. Copyright © 2014 John Wiley &amp; Sons, Ltd.

Improving application behavior on heterogeneous manycore systems through kernel mapping

Many-core accelerators are being more frequently deployed to improve the system processing capabilities. In such systems, application mapping must be enhanced to maximize utilization of the underlying architecture. Especially, in graphics processing units (GPUs), mapping kernels that are part of multi-kernel applications has a great impact on overall performance, since kernels may exhibit different characteristics on different CPUs and GPUs. While some kernels run faster on GPUs, others may perform better in CPUs. Thus, heterogeneous execution may yield better performance than executing the application only on a CPU or only on a GPU. In this paper, we investigate on two approaches: a novel profiling-based adaptive kernel mapping algorithm to assign each kernel of an application to the proper device, and a Mixed-Integer Programming (MIP) implementation to determine optimal mapping. We utilize profiling information for kernels on different devices and generate a map that identifies which kernel should run where in order to improve the overall performance of an application. Initial experiments show that our approach can efficiently map kernels on CPUs and GPUs, and outperforms CPU-only and GPU-only approaches.

A 182 mW 94.3 f/s in Full HD Pattern-Matching Based Image Recognition Accelerator for an Embedded Vision System in 0.13-$\mu{\rm m}$ CMOS Technology

A pattern-matching based image recognition accelerator (PRA) is presented for embedded vision applications. It is a hardware accelerator that performs interest point detection and matching for image-based recognition applications in real time in both mobile devices and vehicles. The proposed system is implemented as a small IP, and it has eight times higher throughput than state-of-the-art object recognition processors, which are implemented based on a heterogeneous many-core system. PRA has three key features: joint algorithm-architecture optimizations for exploiting bit-level parallelism; a low-power unified hardware platform for interest point detection and matching; and scalable hardware architecture. PRA achieves 9.5× performance improvement with only 30% of logic gates including static random-access memory (SRAM) compared to the state-of-the-art object recognition processors. It consists of 78.3 k logic gates and 128 kB SRAM, which are integrated in a test chip implemented for PRA verification. It achieves 94.3 frames per second (fps) in 1080 p full HD resolution at 200-MHz operating frequency while consuming 182 mW. Each complete operation for interest point detection and matching requires 2.09 cycles and 8 cycles on average, respectively, based on a unified bit-level matching accelerator, which is implemented only with 680 logic gates.

Using explicit platform descriptions to support programming of heterogeneous many-core systems

Heterogeneous many-core systems constitute a viable approach for coping with power constraints in modern computer architectures and can now be found across the whole computing landscape ranging from mobile devices, to desktop systems and servers, all the way to high-end supercomputers and large-scale data centers. While these systems promise to offer superior performance-power ratios, programming heterogeneous many-core architectures efficiently has been shown to be notoriously difficult. Programmers typically are forced to take into account a plethora of low-level architectural details and usually have to resort to a combination of different programming models within a single application. In this paper we propose a platform description language (PDL) that enables to capture key architectural patterns of commonly used heterogeneous computing systems. PDL architecture descriptions support both programmers and toolchains by providing platform-specific information in a well-defined and explicit manner. We have developed a prototype source-to-source compilation framework that utilizes PDL descriptors to transform sequential task-based programs with source code annotations into a form that is convenient for execution on heterogeneous many-core systems. Our framework relies on a component-based approach that accommodates for different implementation variants of tasks, customized for different parts of a heterogeneous platform, and utilizes an advanced runtime system for exploiting parallelism through dynamic task scheduling. We show various usage scenarios of our PDL and demonstrate the effectiveness of our framework for a commonly used scientific kernel and a financial application on different configurations of a state-of-the-art CPU/GPU system.

Sorting networks on FPGAs

Computer architectures are quickly changing toward heterogeneous many-core systems. Such a trend opens up interesting opportunities but also raises immense challenges since the efficient use of heterogeneous many-core systems is not a trivial problem. Software-configurable microprocessors and FPGAs add further diversity but also increase complexity. In this paper, we explore the use of sorting networks on field-programmable gate arrays (FPGAs). FPGAs are very versatile in terms of how they can be used and can also be added as additional processing units in standard CPU sockets. Our results indicate that efficient usage of FPGAs involves non-trivial aspects such as having the right computation model (a sorting network in this case); a careful implementation that balances all the design constraints in an FPGA; and the proper integration strategy to link the FPGA to the rest of the system. Once these issues are properly addressed, our experiments show that FPGAs exhibit performance figures competitive with those of modern general-purpose CPUs while offering significant advantages in terms of power consumption and parallel stream evaluation.