Many-core Architectures Research Articles

Heterogeneity of abstractions in EDA tools: Reviewing models of computation for many-core systems targeting intensive signal processing applications

Designing many-core systems for intensive signal processing applications necessitates refreshing common practices in the design of Electronic Design Automation (EDA) tools. This should change the focus of the EDA community from old-fashioned foundations including heavy implementations and time-consuming simulations, to recent engineering practices promoting rapid design and acceptable accuracy. Hence, the challenge of the EDA community comes to define a design process for many-core systems which enables four goals: modeling conciseness, estimation accuracy, design rapidity, and exploration flexibility. In this scenario, the modeling task can be defined as a delicate design phase that requires choosing the adequate Model of Computation (MoC) at each step of the design process. Design models at each level of abstraction provide the basis for applying analysis, synthesis or verification techniques. Since modeling concepts and techniques can have a large influence on the quality, accuracy and rapidity of results, models and corresponding abstraction levels should be well-defined with clear and unambiguous semantics. In this paper, we aim to provide an analysis and comparative overview of common MoCs used in the specification and performance analysis of intensive signal processing applications and many-core architectures. After identifying conventional classifications, we propose a new taxonomy of MoCs based on their purpose in the design flow. The heterogeneity of MoCs in EDA tools is also discussed and various tools are reviewed and compared.

Cache-Oblivious MPI All-to-All Communications Based on Morton Order

Many-core systems with a rapidly increasing number of cores pose a significant challenge to parallel applications to use their complex memory hierarchies efficiently. Many such applications rely on collective communications in performance-critical phases, which become a bottleneck if they are not optimized. We address this issue by proposing cache-oblivious algorithms for MPI_Alltoall, MPI_Allgather, and the MPI neighborhood collectives to exploit the data locality. To implement the cache-oblivious algorithms, we allocate the send and receive buffers on a shared heap and use Morton order to guide the memory copies. Our analysis shows that our algorithm for MPI_Alltoall is asymptotically optimal. We show an extension to our algorithms to minimize the communication distance on NUMA systems while maintaining optimality within each socket. We further demonstrate how the cache-oblivious algorithms can be applied to multi-node machines. Experiments are conducted on different many-core architectures. For MPI_Alltoall, our implementation achieves on average 1.40X speedup over the naive implementation based on shared heap for small and medium block sizes (less than 16 KB) on a Xeon Phi KNC, achieves on average 3.03X speedup over MVAPICH2 on a Xeon E7-8890, and achieves on average 2.23X speedup over MVAPICH2 on a 256-node Xeon E5-2680 cluster for block sizes less than 1 KB.

Accelerating the SVD bi-diagonalization of a batch of small matrices using GPUs

The acceleration of many small-sized linear algebra problems has become extremely challenging for current many-core architectures, and in particular GPUs. Standard interfaces have been proposed for some of these problems, called batched problems, so that they get targeted for optimization and used in a standard way in applications, calling them directly from highly optimized, standard numerical libraries, like (batched) BLAS and LAPACK. While most of the developments have been for one-sided factorizations and solvers, many important applications – from big data analytics to information retrieval, low-rank approximations for solvers and preconditioners – require two-sided factorizations, and most notably the SVD factorization. To address these needs and the parallelization challenges related to them, we developed a number of new batched computing techniques and designed batched Basic Linear Algebra Subroutines (BLAS) routines, and in particular the Level-2 BLAS GEMV and the Level-3 BLAS GEMM routines, to solve them. We propose a device functions-based methodology and big-tile setting techniques in our batched BLAS design. The different optimization techniques result in many software versions that must be tuned, for which we adopt an auto-tuning strategy to automatically derive the optimized instances of the routines. We illustrate our batched BLAS approach to optimize batched SVD bi-diagonalization progressively on GPUs. The progression is illustrated on an NVIDIA K40c GPU, and also, ported and presented on AMD Fiji Nano GPU, using AMD's Heterogeneous–Compute Interface for Portability (HIP) C++ runtime API. We demonstrate achieving 80% of the theoretically achievable peak performance for the overall algorithm, and significant acceleration of the Level-2 BLAS GEMV and Level-3 BLAS GEMM needed compared to vendor-optimized libraries on GPUs and multicore CPUs. The optimization techniques in this paper are applicable to the other two-sided factorizations as well.

Application Deployment Strategies for Spatial Isolation on Many-Core Accelerators

Current cache Side-Channel Attacks (SCAs) countermeasures have not been designed for many-core architectures and need to be revisited in order to be practical for these new technologies. Spatial isolation of resources for sensitive applications has been proposed taking advantage of the large number of resources offered by these architectures. This solution avoids cache sharing with sensitive processes. Consequently, their cache activity cannot be monitored and cache SCAs cannot be performed. This work focuses on the implementation of this technique in order to minimize the induced performance overhead. Different strategies for the management of isolated secure zones are implemented and compared.

Batched one-sided factorizations of tiny matrices using GPUs: Challenges and countermeasures

The use of batched matrix computations recently gained a lot of interest for applications, where the same operation is applied to many small independent matrices. The batched computational pattern is frequently encountered in applications of data analytics, direct/iterative solvers and preconditioners, computer vision, astrophysics, and more, and often requires specific designs for vectorization and extreme parallelism to map well on today's high-end many-core architectures. This has led to the development of optimized software for batch computations, and to an ongoing community effort to develop standard interfaces for batched linear algebra software. Furthering these developments, we present GPU design and optimization techniques for high-performance batched one-sided factorizations of millions of tiny matrices (of size 32 and less). We quantify the effects and relevance of different techniques in order to select the best-performing LU, QR, and Cholesky factorization designs. While we adapt common optimization techniques, such as optimal memory traffic, register blocking, and concurrency control, we also show that a different mindset and techniques are needed when matrices are tiny, and in particular, sub-vector/warp in size. The proposed routines are part of the MAGMA library and deliver significant speedups compared to their counterparts in currently available vendor-optimized libraries. Notably, we tune the developments for the newest V100 GPU from NVIDIA to show speedups of up to 11.8×.

A thread-level parallelization of pairwise additive potential and force calculations suitable for current many-core architectures

In molecular dynamics (MD) simulations, calculations of potentials and their derivatives by coordinate, i.e., forces, in a pairwise additive manner such as the Lennard–Jones interactions and a short-range part of the Coulombic interactions form the main part of arithmetic operations. It is essential to achieve high thread-level parallelization efficiency of these pairwise additive calculations of potentials and forces to use current supercomputers with many-core architectures effectively. In this paper, we propose four new thread-level parallelization algorithms for the pairwise additive potential and force calculations. We implement the four codes in a MD calculation code based on the fast multipole method. Performance benchmarks were taken on the FX100 supercomputer and Intel Xeon Phi coprocessor. The code succeeds in achieving high thread-level parallelization efficiency with 32 threads on the FX100 and up to 60 threads on the Xeon Phi.

Symmetry-Eliminating Design Space Exploration for Hybrid Application Mapping on Many-Core Architectures

Large scale many-core systems are able to execute concurrently changing mixes of different parallel applications. Hybrid application mapping combines the strengths of design-time exploration/analysis of resource constellations for task-to-core mappings with the flexibility of choosing concrete mappings at run time. However, state-of-the-art design space exploration (DSE) techniques so far ignore the problem of symmetries in modern heterogeneous architectures: not only recurring patterns in the architecture but the mapping of tasks to instances of the same processor type may unnecessarily increase the search space by redundant, symmetrical implementations which typically affects the quality of the DSE. As a remedy, we propose a novel meta-heuristic DSE approach that eliminates architectural symmetries by abstracting the problem to a clustering of tasks and their mapping to processor types. However, we demonstrate that simple task clustering and type mappings may again introduce encoding symmetries in our search space. Thus, we present a formulation of the task clustering and type mapping as a 0–1 integer linear program (ILP) which eliminates all architectural as well as encoding symmetries from the search space. We also contribute a formal feasibility check to ensure that only implementations with at least one feasible concrete mapping are considered. To further improve the search process for feasible solutions, we apply satisfiability modulo theories-like learning techniques: from each infeasible implementation, we extract conditions why the implementation is infeasible and enrich our 0–1 ILP by additional constraints continuously during the DSE. Experimental results show that a DSE equipped with the novel symmetry-eliminating search space and the proposed learning techniques clearly outperforms a state-of-the-art approach known from literature in terms of the quality of the gained implementation classes.

Exploring System Availability During Software-Based Self-Testing of Multi-core CPUs

As technology scales, the increased vulnerability of modern systems due to unreliable components becomes a major problem in the era of multi-/many-core architectures. Recently, several on-line testing techniques have been proposed, aiming towards error detection of wear-out/aging-related defects that can appear during the lifetime of a system. In this work, firstly we investigate the relation between system test latency and test-time overhead in multi-/many-core systems with shared Last-Level Cache (LLC) for periodic Software-Based Self-Testing (SBST), under different test scheduling policies. Secondly, we propose a new methodology aiming to reduce the extra overhead related to testing that is incurred as the system scales up (i.e., the number of on-chip cores increases). The investigated scheduling policies primarily vary the number of cores concurrently under test in the overall system test session. Our extensive, workload-driven dynamic exploration reveals that there is an inverse relationship between the two test measures; as the number of cores concurrently under test increases, system test latency decreases, but at the cost of significantly increased test time, which sacrifices system availability for the actual workloads. Under given system test latency constraints, which dictate the recovery time in the event of error detection, our exploration framework identifies the scheduling policy under which the overall test-time overhead is minimized and, hence, system availability is maximized. For the evaluation of the proposed techniques, multi-/many-core systems consisting of 16 and 64 cores are explored in a full-system, execution-driven simulation framework running multi-threaded PARSEC workloads.

Optimizing the performance of reactive molecular dynamics simulations for many-core architectures

Reactive molecular dynamics simulations are computationally demanding. Reaching spatial and temporal scales where interesting scientific phenomena can be observed requires efficient and scalable implementations on modern hardware. In this article, we focus on optimizing the performance of the widely used LAMMPS/ReaxC package for many-core architectures. As hybrid parallelism allows better leverage of the increasing on-node parallelism, we adopt thread parallelism in the construction of bonded and nonbonded lists and in the computation of complex ReaxFF interactions. To mitigate the I/O overheads due to large volumes of trajectory data produced and to save users the burden of post-processing, we also develop a novel in situ tool for molecular species analysis. We analyze the performance of the resulting ReaxC-OMP package on two different architectures: (i) Mira, an IBM Blue Gene/Q system and (ii) Cori-II, a Cray XC-40 sytem with Knights Landing processors. For Pentaerythritol tetranitrate (PETN) systems of sizes ranging from 32 thousand to 16.6 million particles, we observe speedups in the range of 1.5–4.5×. We observe sustained performance improvements for up to 262,144 cores (1,048,576 processes) of Mira and a weak scaling efficiency of 91.5% in large simulations containing 16.6 million particles. The in situ molecular species analysis tool incurs only insignificant overheads across various system sizes and runs configurations.

Nonlinear BDDC Methods with approximate solvers

New nonlinear BDDC (Balancing Domain Decomposition by Constraints) domain decomposition methods using inexact solvers for the subdomains and the coarse problem are proposed. In nonlinear domain decomposition methods, the nonlinear problem is decomposed before linearization to improve concurrency and robustness. For linear problems, the new methods are equivalent to known inexact BDDC methods. The new approaches are therefore discussed in the context of other known inexact BDDC methods for linear problems. Relations are pointed out, and the advantages of the approaches chosen here are highlighted. For the new approaches, using an algebraic multigrid method as a building block, parallel scalability is shown for more than half a million (524288) MPI ranks on the JUQUEEN IBM BG/Q supercomputer (JSC Jülich, Germany) and on up to 193600 cores of the Theta Xeon Phi supercomputer (ALCF, Argonne National Laboratory, USA), which is based on the recent Intel Knights Landing (KNL) many-core architecture. One of our nonlinear inexact BDDC domain decomposition methods is also applied to three-dimensional plasticity problems. Comparisons to standard Newton-Krylov-BDDC methods are provided.

Applications of Ear Decomposition to Efficient Heterogeneous Algorithms for Shortest Path/Cycle Problems

Graph algorithms play an important role in several fields of sciences and engineering. Prominent among them are the All-Pairs-Shortest-Paths ( APSP ) and related problems. Indeed there are several efficient implementations for such problems on a variety of modern multi- and many-core architectures. It can be noticed that for several graph problems, parallelism offers only a limited success as current parallel architectures have severe short-comings when deployed for most graph algorithms. At the same time, some of these graphs exhibit clear structural properties due to their sparsity. This calls for particular solution strategies aimed at scalable processing of large, sparse graphs on modern parallel architectures. In this paper, we study the applicability of an ear decomposition of graphs to problems such as all-pairs-shortest-paths and minimum cost cycle basis. Through experimentation, we show that the resulting solutions are scalable in terms of both memory usage and also their speedup over best known current implementations. We believe that our techniques have the potential to be relevant for designing scalable solutions for other computations on large sparse graphs.

Enabling Massive Multi-Threading with Fast Hashing

The next generation of high-performance computers is expected to execute threads in orders of magnitude higher than today’s systems. Improper management of such huge amount of threads can create resource contention, leading to overall degraded system performance. By leveraging more practical approaches to distribute threads on the available resources, execution models and manycore chips are expected to overcome limitations of current systems. Here, we present DELTA—a Data-Enabled muLti-Threaded Architecture, where a producer-consumer scheme is used to execute threads via complete distributed thread management mechanism. We consider a manycore tiled-chip architecture where Network-on-Chip (NoC) routers are extended to support our execution model. The proposed extension is analysed, while simulation results confirm that DELTA can manage a large number of simultaneous threads, relying on a simple hardware structure.

On the Effect of Mate Selection Scheme of Multi-objective Evolutionary Algorithms with Large Population on Constrained Multi-objective Optimization Problems

Multi-objective evolutionary algorithms (MOEAs) are active research topics for multi-objective design problems. MOEAs are population-based global optimization algorithms and it is said that the performance of the MOEAs depends on the population size. Meanwhile, one of the recent trends of computer development is in the large-scale many-core architectures and massive parallel computation is getting feasible in more companies and laboratories. One of the simple yet effective ways of MOEAs for utilizing the large-scale computational resources would be to increase the population size. This study examines the effect of the population size on MOEAs' performance on a real-world-derived benchmarking optimization problem, with large population size. In this paper, three mate selection schemes with different degree of elitist strategy are adapted to M2M-decomposed NSGA-II. The experimental results show that the elitist strategy can efficiently make use of the effect of the large population size, therefore can reduce the turn-around time of an MOEA.

CF-TUNE: Collaborative Filtering Auto-Tuning for Energy Efficient Many-Core Processors

Energy efficiency is considered today as a first class design principle of modern many-core computing systems in the effort to overcome the limited power envelope. However, many-core processors are characterised by high micro-architectural complexity, which is propagated up to the application level affecting both performance and energy consumption. In this paper, we present CF-TUNE , an online and scalable auto-tuning framework for energy aware applications mapping on emerging many-core architectures. CF-TUNE enables the extraction of an energy-efficient tuning configuration point with minimal application characterisation on the whole tuning configuration space. Instead of analyzing every application against every tuning configuration, it adopts a collaborative filtering technique that quickly and with high accuracy configures the application’s tuning parameters by identifying similarities with previously optimized applications. We evaluate CF-TUNE ’s efficiency against a set of demanding and diverse applications mapped on Intel Many Integrated Core processor and we show that with minimal characterization, e.g., only either two or four evaluations, CF-TUNE recommends a tuning configuration that performs at least at the 94 percent level of the optimal one.

Building and Auto-Tuning Computing Kernels: Experimenting with Boast and Starpu in the Gysela Code

Modeling turbulent transport is a major goal in order to predict confinement performance in a tokamak plasma. The gyrokinetic framework considers a computational domain in five dimensions to look at kinetic issues in a plasma; this leads to huge computational needs. Therefore, optimization of the code is an especially important aspect, especially since coprocessors and complex manycore architectures are foreseen as building blocks for Exascale systems. This project aims to evaluate the applicability of two auto-tuning approaches with the BOAST and StarPU tools on the GYSELA code in order to circumvent performance portability issues. A specific computation intensive kernel is considered in order to evaluate the benefit of these methods. StarPU enables to match the performance and even sometimes outperform the hand-optimized version of the code while leaving scheduling choices to an automated process. BOAST on the other hand reveals to be well suited to get a gain in terms of execution time on four architectures. Speedups in-between 1.9 and 5.7 are obtained on a cornerstone computation intensive kernel.

Reliable low power NoC interconnect

Information communicated through Network on Chip (NoC) in System on Chip (SoC) is highly prone to different sources of noise, like coupling, radiation and electromagnetic interference. The outcome is multi-bit errors, which can either be random or burst. As the demand for reliable NoC increases, optimal error correcting coding techniques become imperative for SoC and various multi-core and many-core architectures. A novel Multi-bit Error Correcting Coding with Reduced Link Bandwidth (MECCRLB) is proposed to achieve reliable data transmission through NoC. The proposed technique corrects burst error of four bits or random error of eleven bits or combined burst and random errors of total four bits for an input flit size of 32 bits. Analytical model based performance estimation for coding technique is extensively used in NoC. Reliability, link swing voltage and link power consumption are estimated using analytical model for the proposed MECCRLB coding technique. All the results obtained for MECCRLB coding technique are compared with Hamming product code with Type II HARQ. Estimated results show that at a probability of residual error of 10−25, the link swing voltage and the link power are reduced by 30% and 75% respectively. Results obtained from simulation followed by synthesis indicate that there is a reduction of 65%, 44%, 27%, 28% and 49% in bit overhead, NoC router area, NoC router power, codec power and codec area respectively. Furthermore, MECCRLB coding technique achieves higher error correction capability and reduces the need for retransmission. This signifies that the proposed coding technique outperforms Hamming product code with Type II HARQ in reliability, area and power.

Efficient multilayer shallow-water simulation system based on GPUs

The computational simulation of shallow stratified fluids is a very active research topic because these types of systems are very common in a variety of natural environments. The simulation of such systems can be modeled using multilayer shallow-water equations but do impose important computational requirements, especially when applied to large domains.General Purpose Computing on Graphics Processing Units (GPGPU) has become a vivid research field due to the arrival of massively parallel hardware platforms (based on graphics cards) and adequate programming frameworks which have allowed important speed-up factors with respect to not only sequential but also parallel CPU based simulation systems.In this work we present simulation of shallow stratified fluids with an arbitrary number of layers using GPUs. The designed system does fully adapt to the many-core architecture of modern GPUs and several experiments have been carried out to illustrate its scalability and behavior on different GPU models. We propose a new multilayer computational scheme for an underlying 2D mathematical model. This scheme is capable of handling an arbitrary number of layers. The system adds no overhead when used for two-layer scenarios, compared to an existing 2D system specifically designed for just two layers.Our proposal is aimed at creating a GPU-based computational scheme suitable for the simulation of multilayer large-scale real-world scenarios.

Low Overhead CS-Based Heterogeneous Framework for Big Data Acceleration

Big data processing on hardware gained immense interest among the hardware research community to take advantage of fast processing and reconfigurability. Though the computation latency can be reduced using hardware, big data processing cost is dominated by data transfers. In this article, we propose a low overhead framework based on compressive sensing (CS) to reduce data transfers up to 67% without affecting signal quality. CS has two important kernels: “sensing” and “reconstruction.” In this article, we focus on CS reconstruction is using orthogonal matching pursuit (OMP) algorithm. We implement the OMP CS reconstruction algorithm on a domain-specific PENC many-core platform and a low-power Jetson TK1 platform consisting of an ARM CPU and a K1 GPU. Detailed performance analysis of OMP algorithm on each platform suggests that the PENC many-core platform has 15× and 18× less energy consumption and 16× and 8× faster reconstruction time as compared to the low-power ARM CPU and K1 GPU, respectively. Furthermore, we implement the proposed CS-based framework on heterogeneous architecture, in which the PENC many-core architecture is used as an “accelerator” and processing is performed on the ARM CPU platform. For demonstration, we integrate the proposed CS-based framework with a hadoop MapReduce platform for a face detection application. The results show that the proposed CS-based framework with the PENC many-core as an accelerator achieves a 26.15% data storage/transfer reduction, with an execution time and energy consumption overhead of 3.7% and 0.002%, respectively, for 5,000 image transfers. Compared to the CS-based framework implementation on the low-power Jetson TK1 ARM CPU+GPU platform, the PENC many-core implementation is 2.3× faster for the image reconstruction part, while achieving 29% higher performance and 34% better energy efficiency for the complete face detection application on the Hadoop MapReduce platform.

A Majority-Based Reliability-Aware Task Mapping in High-Performance Homogenous NoC Architectures

This article presents a new reliability-aware task mapping approach in a many-core platform at design time for applications with DAG-based task graphs. The main goal is to devise a task mapping which meets a predefined reliability threshold considering a minimized performance degradation. The proposed approach uses a majority-voting replication technique to fulfill error-masking capability. A quantitative reliability model is also proposed for the platform. Our platform is a homogenous many-core architecture with mesh-based interconnection using traditional deterministic XY routing algorithm. Our iterative approach is applicable to an unlimited number of system fault types. All parts of the platform, including cores, links, and routers, are assumed to be prone to failures. We used the MNLP optimization technique to find the optimal mapping of the presented task graph. Experimental results show that our suggested task mappings not only comply with predefined reliability thresholds but also achieve notable time complexity reduction with respect to exhaustive space exploration.

Many-core needs fine-grained scheduling: A case study of query processing on Intel Xeon Phi processors

Emerging many-core processors feature very high memory bandwidth and computational power. For example, Intel Xeon Phi many-core processors of the Knights Corner (KNC) and Knights Landing (KNL) architectures embrace 60 to 64 x86-based CPU cores with 512-bit SIMD capabilities and high-bandwidth memories like the GDDR5 on KNC and on-package DRAMs on KNL. In this paper, we study the performance main-memory database operators and online analytical processing (OLAP) on such many-core architectures. We find that even the state-of-the-art database operators suffer severely from memory stalls and resource underutilization on those many-core processors. We argue that a software approach decomposing a coarse-grained operator into fine-grained phases and executing two independent phases with complementary resource requirements concurrently can address this problem. This approach allows more fine-grained control of resource utilization. Our experiments demonstrate significant performance gain and high resource utilization achieved by our proposed approaches on both KNC and KNL.