Many-core Machines Research Articles

CMSSW Scaling Limits on Many-Core Machines

Today the LHC offline computing relies heavily on CPU resources, despite the interest in compute accelerators, such as GPUs, for the longer term future. The number of cores per CPU socket has continued to increase steadily, reaching the levels of 64 cores (128 threads) with recent AMD EPYC processors, and 128 cores on Ampere Altra Max ARM processors. Over the course of the past decade, the CMS data processing framework, CMSSW, has been transformed from a single-threaded framework into a highly concurrent one. The first multithreaded version was brought into production by the start of the LHC Run 2 in 2015. Since then, the framework’s threading efficiency has gradually been improved by adding more levels of concurrency and reducing the amount of serial code paths. The latest addition was support for concurrent Runs. In this work we review the concurrency model of the CMSSW, and measure its scalability with real CMS applications, such as simulation and reconstruction, on modern many-core machines. We show metrics such as event processing throughput and application memory usage with and without the contribution of I/O, as I/O has been the major scaling limitation for the CMS applications.

Performance Analysis of RCU-Style Non-Blocking Synchronization Mechanisms on a Manycore-Based Operating System

There have been recent advances in multi-core machines with tens and hundreds of cores, and there is an increasing emphasis on the software structure. Many different synchronization mechanism techniques have been developed to improve the performance and the scalability of manycore systems. As the non-blocking algorithms are promising in overcoming performance limits in traditional lock-based blocking synchronization mechanisms, we are observing an increased usage ratio and a number of non-blocking synchronization algorithms. For example, the usage ratio of RCU increased sharply in recent years. Since RCU exhibits low write performance and is difficult to use, the research community introduced RLU and MV-RLU synchronization algorithms to address the issues. RLU and MV-RLU, which are called RCU-style synchronization mechanisms, are promising in terms of providing easy-to-use APIs (Application Programming Interfaces) and better performance in manycore machines. To expand the applicability of RCU-style mechanisms, we need to measure the performance and analyze their measurements in various environments. To meet the goal, we evaluate them at the user and kernel level in sv6 variant, which is a research operating system on a manycore system. In order to enable RCU-style synchronization algorithms in sv6 variant, we implemented and modified some of the libraries and memory allocators in sv6 variant. We use micro-benchmarks that exploit a linked list and hash table to measure the performance while experimenting with parameters of the benchmarks and types of data structures. In most of the experiments, we observed that MV-RLU is scalable. MV-RLU exhibits about thirteen times better throughput than RCU in the case of running 70 threads. In addition, we compare the operation procedures and APIs of each RCU-style synchronization algorithm to analyze the pros and cons of the algorithms.

Future-Based Persistent Spatial Data Structure for NVM-Based Manycore Machines

R-trees have been popular for their support of multidimensional data and high-performing queries. FBR-tree is the state-of-the-art concurrent variant of the R-tree for Intel DC Persistent Memory (DCPM). However, its adoption on manycore servers is impeded by concurrency limitations due to lengthy, lock-based synchronization, including structure modification operations, split and merge. Additionally, emerging DCPM-based machines are equipped with multiple CPU sockets, forming a non-uniform memory access (NUMA) architecture. FBR-tree’s lack of NUMA-awareness induces further performance overhead from remote memory accesses. In this paper, we propose MPR-tree, a concurrent, NUMA-aware and persistent future-based R-tree for DCPM servers. MPR-tree focuses on insert operations due to their laborious nature. MPR-tree relies on per-thread local future objects and a global R-tree. To introduce NUMA-awareness and minimize remote memory accesses, MPR-tree adopts per-socket dedicated asynchronous evaluate threads to checkpoint future objects to the global R-tree. MPR-tree employs an in-memory hash table to mitigate the read overhead of key searches over the future objects. We implemented MPR-tree atop FBR-tree and evaluated its performance on a server with 40 physical cores for insert and lookup queries, and it showed that MPR-tree outperforms FBR-tree on average by 2× on log10 scale.

On the use of many-core machines for the acceleration of a mesh truncation technique for FEM

Finite element method (FEM) has been used for years for radiation problems in the field of electromagnetism. To tackle problems of this kind, mesh truncation techniques are required, which may lead to the use of high computational resources. In fact, electrically large radiation problems can only be tackled using massively parallel computational resources. Different types of multi-core machines are commonly employed in diverse fields of science for accelerating a number of applications. However, properly managing their computational resources becomes a very challenging task. On the one hand, we present a hybrid message passing interface + OpenMP-based acceleration of a mesh truncation technique included in a FEM code for electromagnetism in a high-performance computing cluster equipped with 140 compute nodes. Results show that we obtain about 85% of the theoretical maximum speedup of the machine. On the other hand, a graphics processing unit has been used to accelerate one of the parts that presents high fine-grain parallelism.

An optimized unsupervised manifold learning algorithm for manycore architectures

Multimedia data, such as images and videos, has become very popular in people’s daily life as a result of the widespread use of mobile devices. The ever-increasing amount of such data, along with the necessity for real-time retrieval, has lead to the development of new methods that can process them in a timely fashion with acceptable accuracy. In this paper, we study the performance of ReckNN, an unsupervised manifold learning algorithm based on the reciprocal neighbourhood and the authority of ranked lists. Most of the related work in this field do not fully investigate optimization strategies, an aspect that is becoming more important with the high availability of manycore machines. In order to address that issue, we fully investigate optimization opportunities in this article and make the following three main contributions. Firstly, we develop an efficient and scalable method for storing and accessing the distances between objects (e.g., video or image) based on dictionaries. Secondly, we employ memoization to speed up the computation of authority scores, leading to a significant performance gain even on single-core architectures. Lastly, we devise and implement several parallelization strategies and show that they are scalable on a 72-core Intel machine. The experimental results with MPEG-7, Corel5k and MediaEval benchmarks show that the optimized ReckNN delivers both efficiency and scalability, highlighting the importance of the proposed optimizations for manycore machines.

A Very Fast Trace-Driven Simulation Platform for Chip-Multiprocessors Architectural Explorations

Simulation is the main tool for computer architects and parallel application developers for developing new architectures and parallel algorithms on many-core machines. Simulating a many-core architecture represent a challenge to software simulators even with parallelization of these SW on multi-cores. Field Programmable Gate Arrays offer an excellent implementation platform due to inherent parallelism. Existing FPGA-based simulators however, are mostly execution-driven which consumes too many FPGA resources. Hence, they still trade-off accuracy with simulation speed as SW simulators do. In this work, an application-level trace-driven FPGA-based many-core simulator is presented. A parameterized Verilog template was developed that can generate any number of simulator tiles. The input trace has an architecturally agnostic format that is directly interpreted by the FPGA-based timing model to re-construct the execution events of the original application with accurate timing. This allows fitting a large number of simulation tiles on a single FPGA without sacrificing simulation speed or accuracy. Experimental results show that the simulator's average accuracy is ∼14 percent with simulation speeds ranging from 100’s of MIPs to over 2,200 MIPS for a 16-core target architecture. Hence, with accuracy similar to SW simulators, its speed is higher than all other FPGA-based simulators.

Analysis of classic algorithms on highly-threaded many-core architectures

The recently developed Threaded Many-core Memory (TMM) model provides a framework for analyzing algorithms for highly-threaded many-core machines such as GPUs and Cray supercomputers. In particular, it tries to capture the fact that these machines hide memory latencies via the use of a large number of threads and large memory bandwidth. The TMM model analysis contains two components: computational and memory complexity.A model is only useful if it can explain and predict empirical data. In this work, we investigate the effectiveness of the TMM model. Under this model, we analyze algorithms for 5 classic problems— suffix tree/array for string matching, fast Fourier transform, merge sort, list ranking, and all-pairs shortest paths—on a variety of GPUs. We also analyze memory access, matrix multiply and a sequence alignment algorithm on a set of Cray XMT supercomputers, the latest NVIDIA and AMD GPUs. We compare the results of the analysis with the experimental findings of ours and other researchers who have implemented and measured the performance of these algorithms on a spectrum of diverse GPUs and Cray appliances. We find that the TMM model is able to predict important, non-trivial, and sometimes previously unexplained trends and artifacts in the experimental data.

CSR

One of the adverse effects of shrinking transistor sizes is that processors have become increasingly prone to hardware faults. At the same time, the number of cores per die rises. Consequently, core failures can no longer be ruled out, and future operating systems for many-core machines will have to incorporate fault tolerance mechanisms. We present CSR, a strategy for recovery from unexpected permanent processor faults in commodity operating systems. Our approach overcomes surprise removal of faulty cores, and also tolerates cascading core failures. When a core fails in user mode, CSR terminates the process executing on that core and migrates the remaining processes in its run-queue to other cores. We further show how hardware transactional memory may be used to overcome failures in critical kernel code. Our solution is scalable, incurs low overhead, and is designed to integrate into modern operating systems. We have implemented it in the Linux kernel, using Haswell's Transactional Synchronization Extension, and tested it on a real system.

CSR

One of the adverse effects of shrinking transistor sizes is that processors have become increasingly prone to hardware faults. At the same time, the number of cores per die rises. Consequently, core failures can no longer be ruled out, and future operating systems for many-core machines will have to incorporate fault tolerance mechanisms. We present CSR, a strategy for recovery from unexpected permanent processor faults in commodity operating systems. Our approach overcomes surprise removal of faulty cores, and also tolerates cascading core failures. When a core fails in user mode, CSR terminates the process executing on that core and migrates the remaining processes in its run-queue to other cores. We further show how hardware transactional memory may be used to overcome failures in critical kernel code. Our solution is scalable, incurs low overhead, and is designed to integrate into modern operating systems. We have implemented it in the Linux kernel, using Haswell's Transactional Synchronization Extension, and tested it on a real system.

Staring into the abyss

Computer architectures are moving towards an era dominated by many-core machines with dozens or even hundreds of cores on a single chip. This unprecedented level of on-chip parallelism introduces a new dimension to scalability that current database management systems (DBMSs) were not designed for. In particular, as the number of cores increases, the problem of concurrency control becomes extremely challenging. With hundreds of threads running in parallel, the complexity of coordinating competing accesses to data will likely diminish the gains from increased core counts. To better understand just how unprepared current DBMSs are for future CPU architectures, we performed an evaluation of concurrency control for on-line transaction processing (OLTP) workloads on many-core chips. We implemented seven concurrency control algorithms on a main-memory DBMS and using computer simulations scaled our system to 1024 cores. Our analysis shows that all algorithms fail to scale to this magnitude but for different reasons. In each case, we identify fundamental bottlenecks that are independent of the particular database implementation and argue that even state-of-the-art DBMSs suffer from these limitations. We conclude that rather than pursuing incremental solutions, many-core chips may require a completely redesigned DBMS architecture that is built from ground up and is tightly coupled with the hardware.

WideTable

This paper presents a technique called WideTable that aims to improve the speed of analytical data processing systems. A WideTable is built by denormalizing the database, and then converting complex queries into simple scans on the underlying (wide) table. To avoid the pitfalls associated with denormalization, e.g. space overheads, WideTable uses a combination of techniques including dictionary encoding and columnar storage. When denormalizing the data, WideTable uses outer joins to ensure that queries on tables in the schema graph, which are now nested as embedded tables in the WideTable, are processed correctly. Then, using a packed code scan technique, even complex queries on the original database can be answered by using simple scans on the WideTable(s). We experimentally evaluate our methods in a main memory setting using the queries in TPC-H, and demonstrate the effectiveness of our methods, both in terms of raw query performance and scalability when running on many-core machines.

Fully concurrent garbage collection of actors on many-core machines

Disposal of dead actors in actor-model languages is as important as disposal of unreachable objects in object-oriented languages. In current practice, programmers are required to either manually terminate actors, or they have to rely on garbage collection systems that monitor actor mutation through write barriers, thread coordination through locks etc. These techniques, however, prevent the collector from being fully concurrent. We developed a protocol that allows garbage collection to run fully concurrently with all actors. The main challenges in concurrent garbage collection is the detection of cycles of sleeping actors in the actors graph, in the presence of concurrent mutation of this graph. Our protocol is solely built on message passing: it uses deferred direct reference counting, a dedicated actor for the detection of (cyclic) garbage, and a confirmation protocol (to deal with the mutation of the actor graph). We present our ideas informally through an example, and then present a formal model, prove soundness and argue completeness. We have implemented the protocol as part of a runtime library. As a preliminary performance evaluation, we discuss the performance of our approach as currently used at a financial institution, and use four benchmarks from the literature to compare our approach with other actor-model systems. These preliminary results indicate that the overhead of our approach is small.

A memory access model for highly-threaded many-core architectures

A number of highly-threaded, many-core architectures hide memory-access latency by low-overhead context switching among a large number of threads. The speedup of a program on these machines depends on how well the latency is hidden. If the number of threads were infinite, theoretically, these machines could provide the performance predicted by the PRAM analysis of these programs. However, the number of threads per processor is not infinite, and is constrained by both hardware and algorithmic limits. In this paper, we introduce the Threaded Many-core Memory (TMM) model which is meant to capture the important characteristics of these highly-threaded, many-core machines. Since we model some important machine parameters of these machines, we expect analysis under this model to provide a more fine-grained and accurate performance prediction than the PRAM analysis. We analyze 4 algorithms for the classic all pairs shortest paths problem under this model. We find that even when two algorithms have the same PRAM performance, our model predicts different performance for some settings of machine parameters. For example, for dense graphs, the dynamic programming algorithm and Johnson’s algorithm have the same performance in the PRAM model. However, our model predicts different performance for large enough memory-access latency and validates the intuition that the dynamic programming algorithm performs better on these machines. We validate several predictions made by our model using empirical measurements on an instantiation of a highly-threaded, many-core machine, namely the NVIDIA GTX 480.

Studying ROOT I/O performance with PROOF-Lite

Parallelism aims to improve computing performance by executing a set of computations concurrently. Since the advent of today's many-core machines the full exploitation of the available CPU power has been one of the main challenges. In High Energy Physics (HEP) final data analysis the bottleneck is not only the available CPU but also the available I/O bandwidth. Most of today's HEP analysis frameworks depend on ROOT I/O. In this paper we will discuss the results obtained studying the ROOT I/O performance using PROOF-Lite, a parallel multi-process approach whose results can be directly applied to the generic case of many jobs running concurrently on the same machine. We will also discuss the impact of running the applications in virtual machines.

JIT compilation policy for modern machines

Dynamic or Just-in-Time (JIT) compilation is crucial to achieve acceptable performance for applications (written in managed languages, such as Java and C#) distributed as intermediate language binary codes for a virtual machine (VM) architecture. Since it occurs at runtime, JIT compilation needs to carefully tune its compilation policy to make effective decisions regarding 'if' and 'when' to compile different program regions to achieve the best overall program performance. Past research has extensively tuned JIT compilation policies, but mainly for VMs with a single compiler thread and for execution on single-processor machines. This work is driven by the need to explore the most effective JIT compilation strategies in their modern operational environment, where (a) processors have evolved from single to multi/many cores, and (b) VMs provide support for multiple concurrent compiler threads. Our results confirm that changing 'if' and 'when' methods are compiled have significant performance impacts. We construct several novel configurations in the HotSpot JVM to facilitate this study. The new configurations are necessitated by modern Java benchmarks that impede traditional static whole-program discovery, analysis and annotation, and are required for simulating future many-core hardware that is not yet widely available. We study the effects on performance of increasing compiler aggressiveness for VMs with multiple compiler threads running on existing single/multi-core and future many-core machines. Our results indicate that although more aggressive JIT compilation policies show no benefits on single-core machines, these can often improve program performance for multi/many-core machines. However, accurately prioritizing JIT method compilations is crucial to realize such benefits.

Dynamic terrain for multiuser real-time environments.

Despite the pursuit for increased realism in VR training simulations and massively multiplayer online games, we find little deployed capability to modify terrain. Usually, terrain is stored in a static database and then preprocessed to support multiple levels of detail and to rectify features with the ground. Also, dynamic-terrain research has concentrated on graphics simulations for single users; we've found no research on systems that can support 500 or more user systems. Many obstacles to implementing large-scale dynamic-terrain simulations are hardware based such simulations need large amounts of memory and computing power. However, recent advances in computer hardware can help ameliorate these problems. In particular, high-performance computing clusters (HPCCs) offer an expandable set of computing elements, large amounts of memory and storage, and high-speed connections that remove many of the hardware limitations. Additionally, the industry's movement toward multi- or many-core machines indicates that research done today on high-performance computers (HPCs) could find its way into future commodity systems.

Many-Core vs. Many-Thread Machines: Stay Away From the Valley

We study the tradeoffs between many-core machines like Intelpsilas Larrabee and many-thread machines like Nvidia and AMD GPGPUs. We define a unified model describing a superposition of the two architectures, and use it to identify operation zones for which each machine is more suitable. Moreover, we identify an intermediate zone in which both machines deliver inferior performance. We study the shape of this ldquoperformance valleyrdquo and provide insights on how it can be avoided.