Many-core Chips Research Articles

Sharded Router: A novel on-chip router architecture employing bandwidth sharding and stealing

Abstract Packet-based networks-on-chip (NoC) are considered among the most viable candidates for the on-chip interconnection network of many-core chips. Unrelenting increases in the number of processing elements on a single chip die necessitate a scalable and efficient communication fabric. The resulting enlargement of the on-chip network size has been accompanied by an equivalent widening of the physical inter-router channels. However, the growing link bandwidth is not fully utilized, because the packet size is not always a multiple of the channel width. While slicing of the physical channel enhances link utilization, it incurs additional delay, because the number of flit per packet also increases. This paper proposes a novel router micro-architecture that employs fine-grained bandwidth “sharding” (i.e., partitioning) and stealing in order to mitigate the elevation in the zero-load latency caused by slicing. Consequently, the zero-load latency of the Sharded Router becomes identical with that of a conventional router, whereas its throughput is markedly improved by fully utilizing all available bandwidth. Detailed experiments using a full-system simulation framework indicate that the proposed router reduces the average network latency by up to 19% and the execution time of real multi-threaded workloads by up to 43%. Finally, hardware synthesis analysis verifies the modest area overhead of the Sharded Router over a conventional design.

DP&TB: a coherence filtering protocol for many-core chip multiprocessors

Future many-core chip multiprocessors (CMPs) will integrate hundreds of processor cores on chip. Two cache coherence protocols are the mainstream applied to current CMPs. The token-based protocol (Token) provides high performance, but it generates a prohibitive amount of network traffic, which translates into excessive power consumption. The directory-based protocol (Directory) reduces network traffic, yet trades off with the storage overhead of the directory as well as entails comparatively low performance caused by indirection limiting its applicability for many-core CMPs. In this work, we present DP&TB, a novel cache coherence protocol particularly suited to future many-core CMPs. In DP&TB, cache coherence is maintained at the granularity of a page, facilitating to filter out either unnecessary coherence inspections for blocks inside private pages or network traffic for blocks inside shared pages. We employ Directory to detect private and shared pages and Token to maintain the coherence of the blocks inside shared pages. DP&TB inherits the merit of Directory and Token and overcome their problems. Experimental results show that DP&TB comprehensively beyond Directory and Token with improvement by 9.1 % in performance over Token and by 13.8 % in network traffic over Directory. In addition, the storage overhead of DP&TB is less than half of that of Directory. Our proposal can fulfill the requirement of many-core CMPs to achieve high performance, power and area efficiency.

Core frequency adjustment to optimize Time Warp on many-core processors

Time Warp synchronized parallel discrete event simulators are organized to operate asynchronously and aggressively without explicit synchronization between the concurrently executing simulation objects. In place of an explicit synchronization mechanism, the concurrent simulators implement an independent but common virtual clock model and a rollback/recovery mechanism to restore causal order when out-of-order events are detected. When the critical path of execution of the simulation is balanced across this parallel threads of execution, this can result in a highly effective, lightweight synchronization mechanism to implement parallel simulation. However, imbalances in the workload across the threads can result in excessive rollback in some threads and slowed progress of the critical path. On small shared memory multi-core systems, a lowest time-stamp scheduling policy can effectively balance the workload. However, on larger many-core chips, conventional load balancing and workload migration will once again become necessary. Fortunately, emerging many-core chips contain some interesting features that can potentially be exploited to improve the performance of parallel simulations. In particular, the recently developed Intel Single-chip Cloud Computer (SCC) provides mechanisms for the runtime control of the frequency and voltage settings of the chip. Furthermore, the frequency and voltage settings are independently set within different regions (called islands) of the chip. Thus, in a Time Warp simulation, one could increase the frequency of the cores executing threads on the critical path (those experiencing infrequent rollback) and decrease the frequency of the cores executing threads off the critical path (those experiencing excessive rollback). This paper investigates the run-time control and adjustment of core frequency in some contemporary x86 multi-core processors to identify the platforms that can support the exploration of dynamic run-time control of core frequency settings. The results show that while all multi-core processors have software controllable core frequency modulation capabilities, they are generally not fully independent as the system comes under load and are therefore unsuitable for these studies. Fortunately, one processor, the AMD X6 line, provides software control for core frequencies that can be fixed (by software) even as the system operates under load.

REEact

With the shift to many-core chip multiprocessors (CMPs), a critical issue is how to effectively coordinate and manage the execution of applications and hardware resources to overcome performance, power consumption, and reliability challenges stemming from hardware and application variations inherent in this new computing environment. Effective resource and application management on CMPs requires consideration of user/application/hardware-specific requirements and dynamic adaption of management decisions based on the actual run-time environment. However, designing an algorithm to manage resources and applications that can dynamically adapt based on the run-time environment is difficult because most resource and application management and monitoring facilities are only available at the operating system level. This paper presents REEact, an infrastructure that provides the capability to specify user-level management policies with dynamic adaptation. REEact is a virtual execution environment that provides a framework and core services to quickly enable the design of custom management policies for dynamically managing resources and applications. To demonstrate the capabilities and usefulness of REEact, this paper describes three case studies--each illustrating the use of REEact to apply a specific dynamic management policy on a real CMP. Through these case studies, we demonstrate that REEact can effectively and efficiently implement policies to dynamically manage resources and adapt application execution.

3-D integrated heterogeneous intra-chip free-space optical interconnect

This paper presents the first chip-scale demonstration of an intra-chip free-space optical interconnect (FSOI) we recently proposed. This interconnect system provides point-to-point free-space optical links between any two communication nodes, and hence constructs an all-to-all intra-chip communication fabric, which can be extended for inter-chip communications as well. Unlike electrical and other waveguide-based optical interconnects, FSOI exhibits low latency, high energy efficiency, and large bandwidth density, and hence can significantly improve the performance of future many-core chips. In this paper, we evaluate the performance of the proposed FSOI interconnect, and compare it to a waveguide-based optical interconnect with wavelength division multiplexing (WDM). It shows that the FSOI system can achieve significantly lower loss and higher energy efficiency than the WDM system, even with optimistic assumptions for the latter. A 1×1-cm2 chip prototype is fabricated on a germanium substrate with integrated photodetectors. Commercial 850-nm GaAs vertical-cavity-surface-emitting-lasers (VCSELs) and fabricated fused silica microlenses are 3-D integrated on top of the substrate. At 1.4-cm distance, the measured optical transmission loss is 5 dB, the crosstalk is less than -20 dB, and the electrical-to-electrical bandwidth is 3.3 GHz. The latter is mainly limited by the 5-GHz VCSEL.

A scalable and fault-tolerant network routing scheme for many-core and multi-chip systems

Current on-chip network and inter-chip interconnection are designed separately. However, this traditional design methodology faces a great challenge: in a multi-chip system, each many-core chip contains hundreds or thousands of processors. The increasing number of on-chip processors must share one input/output unit to interface with the inter-chip interconnection. The increased network usage at the chip interface may create an uneven traffic load in the on-chip network. That is, traffic jams could occur in the chip area around the input/output unit. New technologies, such as through silicon via and silicon interposer, can directly connect networks on chips. These technologies can improve communication performance and reduce power consumption by omitting the input/output unit. This paper proposes a novel routing scheme to deal with the network scalability issues related to the many-core and multi-chip system-in-package paradigm. The proposed scheme can also enhance the fault-tolerance of nano-scale communication in deep-submicron designs.

3D NOC for many-core processors

With an increasing number of processors forming many-core chip multiprocessors (CMP), there exists a need for easily scalable, high-performance and low-power intra-chip communication infrastructure for emerging systems. In CMPs with hundreds of processing elements, 3D integration can be utilized to shorten long wires forming communication links. In this paper, we propose a Clos network-on-chip (CNOC) in conjunction with 3D integration as a viable network topology for many core CMPs. The primary benefit of 3D CNOC is scalability and a clear upper bound on power dissipation. We present the architectural and physical design of 3D CNOC and compare its performance with several other topologies. Comparisons are made among several topologies (fat tree, flattened butterfly, mesh and Clos) showing the power consumption of a 3D CNOC increases only minimally as the network size is scaled from 64 to 512 nodes relative to the other topologies. Furthermore, in a 512-node system, 3D CNOC consumes about 15% less average power than any other topology. We also compare 3D partitioning strategies for these topologies and discuss their effect on wire delay and the number of through-silicon vias.

Interconnect Network Analysis of Many-Core Chips

This paper presents the first intercore interconnect technology optimization and wiring demand calculation for mesh, concentrated mesh, flattened butterfly, and concentrated flattened butterfly network-on-chip topologies. Global wire dimensions are optimized to achieve maximum bandwidth and minimum delay. The core-to-core channel width is then determined by taking into account the available wiring area and area occupied by the router within each core. It is shown that the router area limits the channel width for all topologies. In mesh and concentrated mesh topologies, a smaller channel width results in underutilization of the available wiring area; less than 10% of the area available in two orthogonal wiring levels if the router area is constrained to 20% of the core area. For flattened butterfly and concentrated flattened butterfly topologies, the required wiring areas can be as high as 80% of two orthogonal wiring levels for the same routing area constraint. However, flattened butterfly topology does not scale for a larger number of cores due to the rapid increase in the number of ports and, consequently, router area.

Reduction of Variation-Induced Energy Overhead in Multi-Core Processors

Core-to-core variability in future many-core chip multi-processors (CMPs) negatively impacts energy. Under-performing cores necessitate increasing the system voltage to maintain homogeneous core performance, introducing an energy overhead. Multiple supply voltages can be used to mitigate the impact of delay variation in CMPs. In this paper, we carefully analyze the use of a local search algorithm to pick near-optimal supply voltages while meeting a fixed performance target. With two system voltages, we prove our algorithm selects the global optimum and in the more general multiple voltage case we develop quantitative bounds. Using a custom simulation methodology on a real processor core, we show that two system voltages provide the most incremental benefit, reducing the energy overhead relative to a single voltage by 59-75% and total energy by 6-16%. Additionally, the worst 5-15% of cores in such systems necessitate increasingly larger amounts of incremental energy for a constant incremental performance gain. Therefore, turning off or disabling these cores is beneficial to a joint performance-energy metric.

On Full-Connectivity Properties of Locally Connected Oscillatory Networks

The latest many-core chip technology advances foster highly parallel computing systems. Consequently, it is crucial to conceive hardware oriented architectures and to realize VLSI platforms, with kilo- or mega-processors, that are able to process and recognize spatial-temporal patterns without breaking them into frames. Oscillatory networks, their archetype being the Turing morphogenesis model, represent a suitable paradigm for processing spatial-temporal time-periodic patterns. In this manuscript we aim at pointing out full-connectivity properties of locally connected oscillatory networks (LCONs) with linear memoryless and space-invariant interactions. In particular, it is analytically shown that LCONs can implement any operators of globally connected networks with linear dynamical interactions, if some suitable components of the oscillator state vector are coupled. The key issue of our results is that the inverse of a banded matrix is almost always full, i.e., almost all of its entries are nonzero. Space-invariant local connectivity allows for hardware realizations with straightforward architectures.

Hardware/software co-design of global cloud system resolving models

[1] We present an analysis of the performance aspects of an atmospheric general circulation model at the ultra-high resolution required to resolve individual cloud systems and describe alternative technological paths to realize the integration of such a model in the relatively near future. Due to a superlinear scaling of the computational burden dictated by the Courant stability criterion, the solution of the equations of motion dominate the calculation at these ultra-high resolutions. From this extrapolation, it is estimated that a credible kilometer scale atmospheric model would require a sustained computational rate of at least 28 Petaflop/s to provide scientifically useful climate simulations. Our design study portends an alternate strategy for practical power-efficient implementations of next-generation ultra-scale systems. We demonstrate that hardware/software co-design of low-power embedded processor technology could be exploited to design a custom machine tailored to ultra-high resolution climate model specifications at relatively affordable cost and power considerations. A strawman machine design is presented consisting of in excess of 20 million processing elements that effectively exploits forthcoming many-core chips. The system pushes the limits of domain decomposition to increase explicit parallelism, and suggests that functional partitioning of sub-components of the climate code (much like the coarse-grained partitioning of computation between the atmospheric, ocean, land, and ice components of current coupled models) may be necessary for future performance scaling.

Light-weight communications on Intel's single-chip cloud computer processor

Many-core chips are changing the way high-performance computing systems are built and programmed. As it is becoming increasingly difficult to maintain cache coherence across many cores, manufacturers are exploring designs that do not feature any cache coherence between cores. Communications on such chips are naturally implemented using message passing, which makes them resemble clusters, but with an important difference. Special hardware can be provided that supports very fast on-chip communications, reducing latency and increasing bandwidth. We present one such chip, the Single-Chip Cloud Computer (SCC). This is an experimental processor, created by Intel Labs. We describe two communication libraries available on SCC: RCCE and Rckmb. RCCE is a light-weight, minimal library for writing message passing parallel applications. Rckmb provides the data link layer for running network services such as TCP/IP. Both utilize SCC's non-cache-coherent shared memory for transferring data between cores without needing to go off-chip. In this paper we describe the design and implementation of RCCE and Rckmb. To compare their performance, we consider simple benchmarks run with RCCE, and MPI over TCP/IP.

A 3-D Integrated Intrachip Free-Space Optical Interconnect for Many-Core Chips

This letter presents a new optical interconnect system for intrachip communications based on free-space optics. It provides all-to-all direct communications using dedicated lasers and photodetectors, hence avoiding packet switching while offering ultra-low latency and scalable bandwidth. A technology demonstration prototype is built on a circuit board using fabricated germanium photodetectors, micro-lenses, commercial vertical-cavity surface-emitting lasers, and micro-mirrors. Transmission loss in an optical link of 10-mm distance and crosstalk between two adjacent links are measured as 5 and -26 dB, respectively. The measured small-signal bandwidth of the link is 10 GHz.

On an efficient NoC multicasting scheme in support of multiple applications running on irregular sub-networks

When a number of applications simultaneously running on a many-core chip multiprocessor (CMP) chip connected through network-on-chip (NoC), significant amount of on-chip traffic is one-to-many (multicast) in nature. As a matter of fact, when multiple applications are mapped onto an NoC architecture with applicable traffic isolation constraints, the corresponding sub-networks of these applications are mapped onto actually tend to be irregular. In the literature, multicasting for irregular topologies is supported through either multiple unicasting or broadcasting, which, unfortunately, results in overly high power consumption and/or long network latency. To address this problem, a simple, yet efficient hardware-based multicasting scheme is proposed in this paper. First, an irregular oriented multicast strategy is proposed. Literally, following this strategy, an irregular oriented multicast routing algorithm can be designed based on any regular mesh based multicast routing algorithm. One such algorithm, namely, Alternative Recursive Partitioning Multicasting (AL + RPM), is proposed based on RPM, which was designed for regular mesh topology originally. The basic idea of AL + RPM is to find the output directions following the basic RPM algorithm and then decide to replicate the packets to the original output directions or the alternative (AL) output directions based on the shape of the sub-network. The experiment results show that the proposed multicast AL + RPM algorithm can consume, on average, 14% and 20% less power than bLBDR (a broadcasting-based routing algorithm) and the multiple unicast scheme, respectively. In addition, AL + RPM has much lower network latency than the above two approaches. To incorporate AL + RPM into a baseline router to support multicasting, the area overhead is fairly modest, less than 5.5%.

Economic Analysis of Testing Homogeneous Manycore Chips

The employment of a large number of structurally identical cores on a single silicon die is generally regarded as a promising solution for tera-scale computation, known as manycore chips. To ensure the product quality of such complex integrated circuits before shipping them to final users, extensive manufacturing tests are necessary and the associated test cost can account for a large share of the total production cost. By introducing spare cores on-chip, the burn-in test time can be shortened and the defect coverage requirements for core tests can be also relaxed, without sacrificing quality of the shipped products. If the above test cost reduction exceeds the manufacturing cost of the extra cores, the total production cost of manycore chips can be reduced. In this paper, we develop novel analytical models to study the above tradeoff and we verify the effectiveness of the proposed test economics model for hypothetical manycore chips with various configurations.

Landing stencil code on Godson-T

The advent of multi-core/many-core chip technology offers both an extraordinary opportunity and a profound challenge. In particular, computer architects and system software designers are faced with a unique opportunity to introducing new architecture features as well as adequate compiler technology -- together they may have profound impact. This paper presents a case study (using the 1-D Jacobi computation) of compiler-amendable performance optimization techniques on a many-core architecture Godson-T. Godson-T architecture has several unique features that are chosen for this study: 1) chip-level global addressable memory in particular the scratchpad memories (SPM) local to the processing cores; 2) fine-grain memory based synchronization (e.g., full-empty bit for fine-grain synchronization). Leveraging state-of-the-art performance optimization methods for 1-D stencil parallelization (e.g., timed tiling and variants), we developed and implement a number of many-core-based optimization for Godson-T. Our experimental study shows good performance in both execution time speedup and scalability, validate the value of globally accessed SPM and fine-grain synchronization mechanism (full-empty bits) under the Godson-T, and provides some useful guidelines for future compiler technology of many-core chip architectures.

Global On-Chip Coordination at Light Speed

On-chip optical links are an efficient means of designing the communication backbone for massive multicore chips. Using nanophotonic technology lets designers develop a low-power, low-latency interconnection infrastructure for many-core chips.

Silicon-photonic network architectures for scalable, power-efficient multi-chip systems

Scaling trends of logic, memories, and interconnect networks lead towards dense many-core chips. Unfortunately, process yields and reticle sizes limit the scalability of large single-chip systems. Multi-chip systems break free of these areal limits, but in turn require enormous chip-to-chip bandwidth. The "macrochip" concept presented here integrates multiple many-core processor chips in a single package with silicon-photonic interconnects. This design enables a multi-chip system to approach the performance of a single large die. In this paper we propose three silicon-photonic network designs that provide low-power, high-bandwidth inter-die communication: a static wavelength-routed point-to-point network, a "two-phase" arbitrated network, and a limited-connectivity point-to-point network. We also adapt two existing intra-chip silicon-photonic interconnects: a token-ring-based crossbar and a circuit-switched torus. We simulate a 64-die, 512-core cache-coherent macrochip using all of the above networks with synthetic kernels, and kernels from Splash-2 and PARSEC. We evaluate the networks on performance, optical power and complexity. Despite a narrow data-path width compared to the token-ring or torus, the point-to-point performs 3.3x and 3.9x better respectively. We show that the point-to-point is over 10x more power-efficient than the other networks. We also show that, contrary to electronic network designs, a point-to-point network has the lowest design complexity for an inter-chip silicon-photonic network.

Optimal Global Interconnects for Networks-on-Chip in Many-Core Architectures

While integrated circuits are moving toward many-core architectures, no circuit-aware interconnect technology optimization methodology has been reported for such chips. To utilize a many-core chip to its full potential, low-latency ultrahigh-bandwidth intercore interconnects are needed. In this letter, for the first time, interconnect dimensions in a network-on-chip (NoC) are optimized to achieve large bandwidth density and small latency simultaneously. The optimal wire width for a NoC is found to be more than ten times smaller than the previously obtained optimal global interconnect width. For a 1000-core chip implemented in the technology year 2015, the optimal wire width is found to be minimum-dimension limited.

Next-generation bioinformatics: using many-core processor architecture to develop a web service for sequence alignment

Bioinformatics algorithms and computing power are the main bottlenecks for analyzing huge amount of data generated by the current technologies, such as the 'next-generation' sequencing methodologies. At the same time, most powerful microprocessors are based on many-core chips, yet most applications cannot exploit such power, requiring parallelized algorithms. As an example of next-generation bioinformatics, we have developed from scratch a new parallelization of the Needleman-Wunsch (NW) sequence alignment algorithm for the 64-core Tile64 microprocessor. The unprecedented performance it offers for a standalone personal computer (PC) is discussed, optimally aligning sequences up to 20 times faster than the non-parallelized version, thus saving valuable time. This algorithm is available as a free web service for the scientific community at http://www.sicuma.uma.es/multicore. The open source code is also available on such site.