Multi-core Design Research Articles

Low Latency and Energy Efficient Scalable Architecture for Massive NoCs Using Generalized de Bruijn Graph

Employing thousands of cores in a single chip is the natural trend to handle the ever increasing performance requirements of complex applications such as those used in graphics and multimedia processing. System-on-chips (SoCs) platforms based on network-on-chips (NoCs) could be a viable option for the deployment of large multicore designs with thousands of cores. This paper proposes the generalized binary de Bruijn (GBDB) graph as a reliable and efficient network topology for a large NoC. We propose a reliable routing algorithm to detour a faulty channel between two adjacent switches. In addition, using integer linear programming, we propose an optimal tile-based implementation for a GBDB-based NoC in which the number of channels is less than that of Torus which has the same number of links. Our experimental results show that the latency and energy consumption of the generalized de Bruijn graph are much less than those of Mesh and Torus. The low energy consumption of a de Bruijn graph-based NoC makes it suitable for portable devices which have to operate on limited batteries. Also, the gate level implementation of the proposed reliable routing shows small area, power, and timing overheads due to the proposed reliable routing algorithm.

Workload Mode Identification for Chip Heterogeneous Multiprocessors

With the potential for tens to hundreds of processing elements on future single chip multicore designs also comes the potential to execute a wider variety of input streams, or workloads. At the same time, the trend is for single users to utilize an entire single chip multicore computer. A central challenge for these computers is how to model and identify persistent changes in the input stream, or workload modes. Computer architects often model single program phases as Markov chains. We define workload modes and analyze and evaluate two modeling techniques, a Workload Classification Model (WCM) and a Hidden Markov Model (HMM). We include experimentation on a cell phone example, illustrating how WCM is, on average, 34 times more time efficient and 83% more space efficient than HMM, while improving overall performance by an average of 191% and being, on average, 56% more energy efficient. We found that even sub-optimal use of WCM can outperform HMM, further supporting the need for design time workload models. Our main contribution is to show how the design of single user multicore architecture to models of workloads that arise from single user usage patterns will be necessary as the complexity of applications and architectures grows. Thus, we advocate Workload Specific Processors as a new means of orienting single-user chip heterogeneous multiprocessors.

Dark silicon and the end of multicore scaling

Since 2005, processor designers have increased core counts to exploit Moore's Law scaling, rather than focusing on single-core performance. The failure of Dennard scaling, to which the shift to multicore parts is partially a response, may soon limit multicore scaling just as single-core scaling has been curtailed. This paper models multicore scaling limits by combining device scaling, single-core scaling, and multicore scaling to measure the speedup potential for a set of parallel workloads for the next five technology generations. For device scaling, we use both the ITRS projections and a set of more conservative device scaling parameters. To model single-core scaling, we combine measurements from over 150 processors to derive Pareto-optimal frontiers for area/performance and power/performance. Finally, to model multicore scaling, we build a detailed performance model of upper-bound performance and lower-bound core power. The multicore designs we study include single-threaded CPU-like and massively threaded GPU-like multicore chip organizations with symmetric, asymmetric, dynamic, and composed topologies. The study shows that regardless of chip organization and topology, multicore scaling is power limited to a degree not widely appreciated by the computing community. Even at 22 nm (just one year from now), 21% of a fixed-size chip must be powered off, and at 8 nm, this number grows to more than 50%. Through 2024, only 7.9x average speedup is possible across commonly used parallel workloads, leaving a nearly 24-fold gap from a target of doubled performance per generation.

Area-Efficient Antenna-Scalable MIMO Detector for K-best Sphere Decoding

K-best sphere decoding is one of the most popular MIMO (Multi-Input Multi-Output) detection algorithms because of its low complexity and close to Maximum Likelihood (ML) Bit Error Rate (BER) performance. Unfortunately, conventional multi-stage sphere decoders suffer from the inability to adapt to varying antenna configurations, requiring implementation redesign for each specific array structure. In this paper, we propose a reconfigurable in-place architecture that is scalable to an arbitrary number of antennas at run-time, while reducing area significantly compared with other sphere decoders. To improve the throughput of the in-place architecture without any degradation in BER performance, we propose partial-sort-bypass and symbol interleaving techniques, and also exploit multi-core design. Implementation results for a 16-QAM MIMO decoder in a 130 nm CMOS technology show a 41% reduction in area compared to the smallest sphere decoder while maintaining antenna reconfigurability, and better throughput. When implemented for the 802.11n standard, our architecture results in 42% reduction in area compared to the multi-stage architecture.

Gyrokinetic particle-in-cell optimization on emerging multi- and manycore platforms

The next decade of high-performance computing (HPC) systems will see a rapid evolution and divergence of multi- and manycore architectures as power and cooling constraints limit increases in microprocessor clock speeds. Understanding efficient optimization methodologies on diverse multicore designs in the context of demanding numerical methods is one of the greatest challenges faced today by the HPC community. In this work, we examine the efficient multicore optimization of GTC, a petascale gyrokinetic toroidal fusion code for studying plasma microturbulence in tokamak devices. For GTC’s key computational components (charge deposition and particle push), we explore efficient parallelization strategies across a broad range of emerging multicore designs, including the recently-released Intel Nehalem-EX, the AMD Opteron Istanbul, and the highly multithreaded Sun UltraSparc T2+. We also present the first study on tuning gyrokinetic particle-in-cell (PIC) algorithms for graphics processors, using the NVIDIA C2050 (Fermi). Our work discusses several novel optimization approaches for gyrokinetic PIC, including mixed-precision computation, particle binning and decomposition strategies, grid replication, SIMDized atomic floating-point operations, and effective GPU texture memory utilization. Overall, we achieve significant performance improvements of 1.3–4.7× on these complex PIC kernels, despite the inherent challenges of data dependency and locality. Our work also points to several architectural and programming features that could significantly enhance PIC performance and productivity on next-generation architectures.

Divide-and-Conquer Way Access for Low Power Mobile Caches

As multi-core design concept is becoming dominant, power consumption of the shared level-2 caches is one of the critical issues along with its performance. This is more significant for mobile processors which are used in battery-powered devices. Designing a cache memory, increasing the cache size or adding more set-associativity is one of the simplest ways to improve the performance for both mobile processor and even general-purpose processors. In mobile processors, however, simple increase of the cache size can significantly affect the chip area and power consumption. Modern integration technologies allow integrating the bigger caches into tiny single-chip multi-core mobile processors, but relevant architectural issues cannot be ignored. These issues will be worse in mobile processors which are integrated in a small form-factor without any cooling fan. In this paper, we propose a novel cache mechanism to reduce power consumption in level-2 cache by using the divide-and-conquer way-accessing technique. This idea is derived from the observation that cache way-hit distribution shows the unbalanced pattern in which primary-way(s) have more way-hits. Based on this way utilizations and activities, we came up with a biased way-access mechanism to get benefits from lower associativity which requires relatively lower power consumption. Considering unbalanced way-hit distribution of the cache, all other ways but primary way(s) can be accessed one step later when the primary way-access has across the miss. By doing that, if way-hits are achieved from the primary-way, power can be saved by turning only primary-way on. However, the penalties from the primary-way misses will not be ignored in access time. Through the careful tradeoff analysis between power-saving and penalty in access time, appropriate cache configurations can be chosen in early design stage.

Operating System Design in Future Wireless Sensor Networks

Traditional operating systems do not take into consideration the limitations in space and energy of wireless sensor networks. Thus, contemporary architectural demands in terms of power, heat, size and cost will not be satisfactorily met by such uniprocessing design. Also, the transition to multithreaded, multi-core designs places a greater responsibility on programmers and software for improving performance which is becoming increasingly important as sensor nodes are migrating towards dual processor designs. By analyzing and summarizing the activity of a system, one could locate sections of code that have a potential to generate enhanced performance. First, this paper studies the differences between different operating system designs introducing a thread-driven scheduling algorithm which focuses on the value of preemption to overcome the energy tradeoff brought by event-driven systems. We then devise efficient techniques that will enable us to locate sections in OS code that could behave more efficiently when parallelized, especially in terms of energy consumption. Finally, we provide simulation results that will validate our proposed techniques.

A Novel Threshold Voltage Assignment for 3D Multicore Designs

Chips organized in a 3D stack exhibit widely varying thermal characteristics, driven by the fundamental principles of heat flow. This heterogeneous thermal profile, in conjunction with the strong dependence of leakage with temperature and process variation, imposes a severe challenge of power provisioning in 3D multicore systems. In this paper, we propose a novel physical design paradigm for 3D multicore systems, where circuit level optimizations are guided by the knowledge of the intended placement of a die in the 3D stack. We illustrate this design paradigm using a concrete example of a threshold voltage (V t ) assignment algorithm for a 3D multicore. Our algorithm integrates the impact of temperature dependent leakage in thermal provisioning, expected variation in thermal profile of a 3D multicore, and the dependency of V t levels on PV induced leakage variation. Rigorous evaluation based on detailed architectural simulation demonstrates 2-15% improvement in energy efficiency of a typical 3D multicore system using our proposed technique.

Modeling critical sections in Amdahl's law and its implications for multicore design

This paper presents a fundamental law for parallel performance: it shows that parallel performance is not only limited by sequential code (as suggested by Amdahl's law) but is also fundamentally limited by synchronization through critical sections. Extending Amdahl's software model to include critical sections, we derive the surprising result that the impact of critical sections on parallel performance can be modeled as a completely sequential part and a completely parallel part. The sequential part is determined by the probability for entering a critical section and the contention probability (i.e., multiple threads wanting to enter the same critical section). This fundamental result reveals at least three important insights for multicore design. (i) Asymmetric multicore processors deliver less performance benefits relative to symmetric processors than suggested by Amdahl's law, and in some cases even worse performance. (ii) Amdahl's law suggests many tiny cores for optimum performance in asymmetric processors, however, we find that fewer but larger small cores can yield substantially better performance. (iii) Executing critical sections on the big core can yield substantial speedups, however, performance is sensitive to the accuracy of the critical section contention predictor.

The Review of Cache Partitioning in Multi-Core Processor

With the development of microelectronics technology, Chip Multi-Processor (CMP) or multi-core design has become a mainstream choice for major microprocessor vendors. But in a chip-multiprocessor with a shared cache structure , the competing accesses from different applications degrade the system performance , resulting in non-optimal performance and non-predicting executing time. Cache partitioning techniques can exclusively partition the shared cache among multiple competing applications. In this paper, we first introduce the problems caused by Cache pollution in multicore processor structure; then present the different methods of Cache partitioning in multicore processor structure¬ －－categorizing them based on the different metrics. And finally, we discuss some possible directions for future research in the area.

Photonic BandGap Fiber With Multiple Hollow Cores

We report the fabrication and characterization of the first photonic bandgap fiber with multiple hollow waveguiding cores. Perspectives for scaling to highly multicore designs are discussed.

Multicore Image Processing with OpenMP [Applications Corner

One of the recent innovations in computer engineering has been the development of multicore processors, which are composed of two or more independent cores in a single physical package. Today, many processors, including digital signal processors (DSPs), mobile, graphics, and general-purpose central processing units (CPUs) have a multicore design, driven by the demand of higher performance. Major CPU vendors have changed strategy away from increasing the raw clock rate to adding on-chip support for multithreading by increases in the number of cores; dual and quad-core processors are now commonplace. Signal and image processing programmers can benefit dramatically from these advances in hardware, by modifying single-threaded code to exploit parallelism to run on multiple cores.

Power7: IBM's Next-Generation Server Processor

The Power7 is IBM's first eight-core processor, with each core capable of four-way simultaneous-multithreading operation. Its key architectural features include an advanced memory hierarchy with three levels of on-chip cache; embedded-DRAM devices used in the highest level of the cache; and a new memory interface. This balanced multicore design scales from 1 to 32 sockets in commercial and scientific environments.

A Power, Performance Scalable Eight-Cores Media Processor for Mobile Multimedia Applications

We have developed an eight-core media processor for mobile multimedia applications, which achieves low power consumption and high performance. The processor uses a specific parallel process execution scheme that achieves high performance and allows parallel processing with performance and power consumption scalability over available resources. In addition, the parallel process execution scheme enables software reusability and binary compatibility, which notably reduces development costs. The evaluated application for this multi-core processor design was an A/V playback with H.264 video decoder and an AAC audio decoder. The evaluation includes performance and power consumption for software without recompilation. The results demonstrate that by efficiently using the software scheme it is possible to achieve important gains in hardware potential.

A survey of multicore processors

General-purpose multicore processors are being accepted in all segments of the industry, including signal processing and embedded space, as the need for more performance and general-purpose programmability has grown. Parallel processing increases performance by adding more parallel resources while maintaining manageable power characteristics. The implementations of multicore processors are numerous and diverse. Designs range from conventional multiprocessor machines to designs that consist of a "sea" of programmable arithmetic logic units (ALUs). In this article, we cover some of the attributes common to all multicore processor implementations and illustrate these attributes with current and future commercial multicore designs. The characteristics we focus on are application domain, power/performance, processing elements, memory system, and accelerators/integrated peripherals.

Design and Fabrication of an Er-Doped Silica Optical Fiber with Six Photosensitive Subcores

A type of multi-core Er-doped photosensitive silica optical fiber (MC-EDPF) is proposed and fabricated, in which a high consistency Er-doped core is surrounded by six high consistency Ge-doped cores. The multi-core design can overcome the difficulties encountered in the design and fabrication of single-core EDPFs through a modified chemical vapor deposition method combined with solution doping technology, and there is a conflict between high consistency Er doping and high consistency Ge doping. The absorption of MC-EDPFs achieved 15.876 dB/m at 1550nm and 10 dB/m at 980nm. The reflectivity of the fiber Bragg gratings (FBGs) written directly on the MC-EDPFs is as much as 96.84%.

Boosting single-thread performance in multi-core systems through fine-grain multi-threading

Industry has shifted towards multi-core designs as we have hit the memory and power walls. However, single thread performance remains of paramount importance since some applications have limited thread-level parallelism (TLP), and even a small part with limited TLP impose important constraints to the global performance, as explained by Amdahl's law. In this paper we propose a novel approach for leveraging multiple cores to improve single-thread performance in a multi-core design. The proposed technique features a set of novel hardware mechanisms that support the execution of threads generated at compile time. These threads result from a fine-grain speculative decomposition of the original application and they are executed under a modified multi-core system that includes: (1) mechanisms to support multiple versions; (2) mechanisms to detect violations among threads; (3) mechanisms to reconstruct the original sequential order; and (4) mechanisms to checkpoint the architectural state and recovery to handle misspeculations. The proposed scheme outperforms previous hardware-only schemes to implement the idea of combining cores for executing single-thread applications in a multi-core design by more than 10% on average on Spec2006 for all configurations. Moreover, single-thread performance is improved by 41% on average when the proposed scheme is used on a Tiny Core, and up to 2.6x for some selected applications.

Architectural core salvaging in a multi-core processor for hard-error tolerance

The incidence of hard errors in CPUs is a challenge for future multicore designs due to increasing total core area. Even if the location and nature of hard errors are known a priori, either at manufacture-time or in the field, cores with such errors must be disabled in the absence of hard-error tolerance. While caches, with their regular and repetitive structures, are easily covered against hard errors by providing spare arrays or spare lines, structures within a core are neither as regular nor as repetitive. Previous work has proposed microarchitectural core salvaging to exploit structural redundancy within a core and maintain functionality in the presence of hard errors. Unfortunately microarchitectural salvaging introduces complexity and may provide only limited coverage of core area against hard errors due to a lack of natural redundancy in the core. This paper makes a case for architectural core salvaging. We observe that even if some individual cores cannot execute certain operations, a CPU die can be instruction-set-architecture (ISA) compliant, that is execute all of the instructions required by its ISA, by exploiting natural cross-core redundancy. We propose using hardware to migrate offending threads to another core that can execute the operation. Architectural core salvaging can cover a large core area against faults, and be implemented by leveraging known techniques that minimize changes to the microarchitecture. We show it is possible to optimize architectural core salvaging such that the performance on a faulty die approaches that of a fault-free die--assuring significantly better performance than core disabling for many workloads and no worse performance than core disabling for the remainder.

Parallelizing sequential applications on commodity hardware using a low-cost software transactional memory

Multicore designs have emerged as the mainstream design paradigm for the microprocessor industry. Unfortunately, providing multiple cores does not directly translate into performance for most applications. The industry has already fallen short of the decades-old performance trend of doubling performance every 18 months. An attractive approach for exploiting multiple cores is to rely on tools, both compilers and runtime optimizers, to automatically extract threads from sequential applications. However, despite decades of research on automatic parallelization, most techniques are only effective in the scientific and data parallel domains where array dominated codes can be precisely analyzed by the compiler. Thread-level speculation offers the opportunity to expand parallelization to general-purpose programs, but at the cost of expensive hardware support. In this paper, we focus on providing low-overhead software support for exploiting speculative parallelism. We propose STMlite, a light-weight software transactional memory model that is customized to facilitate profile-guided automatic loop parallelization. STMlite eliminates a considerable amount of checking and locking overhead in conventional software transactional memory models by decoupling the commit phase from main transaction execution. Further, strong atomicity requirements for generic transactional memories are unnecessary within a stylized automatic parallelization framework. STMlite enables sequential applications to extract meaningful performance gains on commodity multicore hardware.

MPTLsim

The introduction of multicore microprocessors in the recent years has made it imperative to use cycleaccurate and full-system simulators in the architecture research community. We introduce MPTLsim - a multicore simulator for the X86 ISA that meets this need. MPTLsim is a uop-accurate, cycle-accurate, full-system simulator for multicore designs based on the X86-64 ISA. MPTLsim extends PTLsim, a publicly available single core simulator, with a host of additional features to support hyperthreading within a core and multiple cores, with detailed models for caches, on-chip interconnections and the memory data flow. MPTLsim incorporates detailed simulation models for cache controllers, interconnections and has built-in implementations of a number of cache coherency protocols.