Many-core Chip Multiprocessors Research Articles

An efficient multi-level cache system for geometrically interconnected many-core chip multiprocessor

<p><span lang="EN-US">Many-core chip multiprocessor offers high parallel processing power for big data analytics; however, they require efficient multi-level cache and interconnection to achieve high system throughput. Using on-chip first level L1 and second level L2 per core fast private caches is expensive for large number of cores. In this paper, for moderate number of cores from 16 to 64, we present a cost and performance efficient multi-level cache system with per core L1 and last level shared bus cache on each bus line of a cost-efficient geometrically bus-based interconnection. In our approach, we extracted cache hit and miss concurrencies and applied concurrent average memory access time to more accurately determine the cache system performance. We conducted least recently used cache policy-based simulation for cache system with L1, with L1/L2, and with L1/shared bus cache. Our simulation results show that an average system throughput improvement of 2.5x can be achieved by using system with L1/shared bus cache system compared to using only first level L1 or L1/L2. Further, we show that the throughput degradation for the proposed cache system is only within 5% for a single bus fault, suggesting a good bus fault tolerance.</span></p>

A Table-Free Approximate Q-Learning-Based Thermal-Aware Adaptive Routing for Optical NoCs

Optical networks-on-chips (NoCs) based on silicon photonics have been proposed as an emerging communication architecture for many-core chip multiprocessors. However, the thermal sensitivity of silicon photonics is one of the major challenges. $Q$ -learning-based adaptive routing has been proposed in related work to mitigate the thermal issue. However, table overhead of the traditional table-based $Q$ -routing would scale up quickly with the increase of network size. In this article, we propose a table-free approximate $Q$ -learning-based thermal-aware adaptive routing to find optimal low-loss paths in the presence of on-chip temperature variations. The simulation results show that the proposed table-free approximate $Q$ -learning-based adaptive routing can converge faster and it can achieve similar optimization effect as compared to the best optimization effect of the traditional table-based $Q$ -routing. The performance gap between the proposed approximation method and the traditional table-based $Q$ -routing expands when the network size increases.

Efficient Mapping of Applications for Future Chip-Multiprocessors in Dark Silicon Era

The failure of Dennard scaling has led to the utilization wall that is the source of dark silicon and limits the percentage of a chip that can actively switch within a given power budget. To address this issue, a structure is needed to guarantee the limited power budget along with providing sufficient flexibility and performance for different applications with various communication requirements. In this article, we present a general-purpose platform for future many-core Chip-Multiprocessors (CMPs) that benefits from the advantages of clustering, Network-on-Chip (NoC) resource sharing among cores, and power gating the unused components of clusters. We also propose two task mapping methods for the proposed platform in which active and dark cores are dispersed appropriately, so that an excess of power budget can be obtained. Our evaluations reveal that the first and second proposed mapping mechanisms respectively reduce the execution time by up to 28.6% and 39.2% and the NoC power consumption by up to 11.1% and 10%, and gain an excess power budget of up to 7.6% and 13.4% over the baseline architecture.

Providing Balanced Mapping for Multiple Applications in Many-Core Chip Multiprocessors

This paper addresses the problem of balancing the on-chip packet latencies in a chip multi-processor (CMP), which is simultaneously executing multiple applications. Specifically, this paper presents a balanced application-to-core mapping algorithm that aims to minimize the maximum on-chip packet latency of all running applications. The paper starts by formulating the balanced mapping problem for CMPs and proving its NP-completeness. Next it presents an efficient heuristic algorithm for solving the aforesaid problem, which utilizes the characteristics of on-chip cache and memory accesses in CMPs and takes into account the workload variations among applications. Simulation results on PARSEC benchmark suite show that the proposed algorithm lowers the maximum average packet latency of all applications by 11 percent while cutting the standard deviation of on-chip packet latencies by 99 percent. This is achieved by very little overhead in terms of the overall packet latency and power consumption averaged over all packets.

Are distributed sharing codes a solution to the scalability problem of coherence directories in manycores? An evaluation study

The development of efficient and scalable cache coherence protocols is a key aspect in the design of manycore chip multiprocessors. In this work, we present a comprehensive evaluation of a kind of ...

Use It or Lose It

Moore's Law scaling continues to yield higher transistor density with each succeeding process generation, leading to today's many-core chip multiprocessors (CMPs) with tens or even hundreds of interconnected cores or tiles. Unfortunately, deep submicron CMOS process technology is marred by increasing susceptibility to wear. Prolonged operational stress gives rise to accelerated wearout and failure due to several physical failure mechanisms, including hot-carrier injection (HCI) and negative-bias temperature instability (NBTI). Each failure mechanism correlates with different usage-based stresses, all of which can eventually generate permanent faults. While the wearout of an individual core in many-core CMPs may not necessarily be catastrophic, a single fault in the interprocessor network-on-chip (NoC) fabric could render the entire chip useless, as it could lead to protocol-level deadlocks, or even partition away vital components such as the memory controller or other critical I/O. In this article, we study HCI- and NBTI-induced wear due to actual stresses caused by real workloads, applied onto the interconnect microarchitecture and develop a critical path model for NBTI-induced wearout. A key finding of this modeling is that, counter to prevailing wisdom, wearout in the CMP's on-chip interconnect is correlated with lack of load observed in the NoC routers rather than high load. We then develop a novel wearout-decelerating scheme in which routers under low load have their wear-sensitive components exercised without significantly impacting cycle time, pipeline depth, area, or power consumption of the overall router. A novel deterministic approach is proposed for the generation of appropriate exercise-mode data, ensuring design parameter targets are met. We subsequently show that the proposed design yields an ∼2,300× decrease in the rate of wear.

Leveraging dark silicon to optimize networks-on-chip topology

This paper presents a reconfigurable network-on-chip (NoC) for many-core chip multiprocessors (CMPs) in the dark silicon era, where a considerable part of high-end chips cannot be powered up due to the power and bandwidth walls. Core specialization, which trades off the cheaper silicon area with energy-efficiency, is a promising solution to the dark silicon challenge. This approach integrates a selection of many diverse application-specific cores into a single many-core chip. Each application then activates those cores that best match its processing requirements. Since active cores may not always form a contiguous active region in the chip, such a partially active many-core CMP requires some special on-chip communication support to optimize NoC parameters for the current set of active cores. In this paper, we propose a reconfigurable NoC that leverages inactive routers of a many-core chip to customize the topology for active cores. In this design, routers of the dark part of the chip are used as bypass switches that can set up virtual long links between distant active nodes in the network. Our experimental results show considerable reduction in NoC energy consumption and latency.

Many-Core Token-Based Adaptive Power Gating

Among power dissipation components, leakage power has become more dominant with each successive technology node. Leakage energy waste can be reduced by power gating. In this paper, we extend token-based adaptive power gating (TAP), a technique to power gate an actively executing core during memory accesses, to many-core Chip Multi-Processors (CMPs). TAP works by tracking every system memory request and its estimated time of arrival so that a core may power gate itself without performance or energy loss. Previous work on TAP [11] shows several benefits compared to earlier state-of-the-art techniques [10], including zero performance hit and 2.58 times average energy savings for out-of-order cores. We show that TAP can adapt to increasing memory contention by increasing power-gated time by 3.69 times compared to a low memory-pressure case. We also scale TAP to many-core architectures with a distributed wake-up controller that is capable of supporting staggered wake-ups and able to power gate each core for 99.07% of the time, achieved by a non-scalable centralized scheme.

DP&TB: a coherence filtering protocol for many-core chip multiprocessors

Future many-core chip multiprocessors (CMPs) will integrate hundreds of processor cores on chip. Two cache coherence protocols are the mainstream applied to current CMPs. The token-based protocol (Token) provides high performance, but it generates a prohibitive amount of network traffic, which translates into excessive power consumption. The directory-based protocol (Directory) reduces network traffic, yet trades off with the storage overhead of the directory as well as entails comparatively low performance caused by indirection limiting its applicability for many-core CMPs. In this work, we present DP&TB, a novel cache coherence protocol particularly suited to future many-core CMPs. In DP&TB, cache coherence is maintained at the granularity of a page, facilitating to filter out either unnecessary coherence inspections for blocks inside private pages or network traffic for blocks inside shared pages. We employ Directory to detect private and shared pages and Token to maintain the coherence of the blocks inside shared pages. DP&TB inherits the merit of Directory and Token and overcome their problems. Experimental results show that DP&TB comprehensively beyond Directory and Token with improvement by 9.1 % in performance over Token and by 13.8 % in network traffic over Directory. In addition, the storage overhead of DP&TB is less than half of that of Directory. Our proposal can fulfill the requirement of many-core CMPs to achieve high performance, power and area efficiency.

REEact

With the shift to many-core chip multiprocessors (CMPs), a critical issue is how to effectively coordinate and manage the execution of applications and hardware resources to overcome performance, power consumption, and reliability challenges stemming from hardware and application variations inherent in this new computing environment. Effective resource and application management on CMPs requires consideration of user/application/hardware-specific requirements and dynamic adaption of management decisions based on the actual run-time environment. However, designing an algorithm to manage resources and applications that can dynamically adapt based on the run-time environment is difficult because most resource and application management and monitoring facilities are only available at the operating system level. This paper presents REEact, an infrastructure that provides the capability to specify user-level management policies with dynamic adaptation. REEact is a virtual execution environment that provides a framework and core services to quickly enable the design of custom management policies for dynamically managing resources and applications. To demonstrate the capabilities and usefulness of REEact, this paper describes three case studies--each illustrating the use of REEact to apply a specific dynamic management policy on a real CMP. Through these case studies, we demonstrate that REEact can effectively and efficiently implement policies to dynamically manage resources and adapt application execution.

3D NOC for many-core processors

With an increasing number of processors forming many-core chip multiprocessors (CMP), there exists a need for easily scalable, high-performance and low-power intra-chip communication infrastructure for emerging systems. In CMPs with hundreds of processing elements, 3D integration can be utilized to shorten long wires forming communication links. In this paper, we propose a Clos network-on-chip (CNOC) in conjunction with 3D integration as a viable network topology for many core CMPs. The primary benefit of 3D CNOC is scalability and a clear upper bound on power dissipation. We present the architectural and physical design of 3D CNOC and compare its performance with several other topologies. Comparisons are made among several topologies (fat tree, flattened butterfly, mesh and Clos) showing the power consumption of a 3D CNOC increases only minimally as the network size is scaled from 64 to 512 nodes relative to the other topologies. Furthermore, in a 512-node system, 3D CNOC consumes about 15% less average power than any other topology. We also compare 3D partitioning strategies for these topologies and discuss their effect on wire delay and the number of through-silicon vias.

Reduction of Variation-Induced Energy Overhead in Multi-Core Processors

Core-to-core variability in future many-core chip multi-processors (CMPs) negatively impacts energy. Under-performing cores necessitate increasing the system voltage to maintain homogeneous core performance, introducing an energy overhead. Multiple supply voltages can be used to mitigate the impact of delay variation in CMPs. In this paper, we carefully analyze the use of a local search algorithm to pick near-optimal supply voltages while meeting a fixed performance target. With two system voltages, we prove our algorithm selects the global optimum and in the more general multiple voltage case we develop quantitative bounds. Using a custom simulation methodology on a real processor core, we show that two system voltages provide the most incremental benefit, reducing the energy overhead relative to a single voltage by 59-75% and total energy by 6-16%. Additionally, the worst 5-15% of cores in such systems necessitate increasingly larger amounts of incremental energy for a constant incremental performance gain. Therefore, turning off or disabling these cores is beneficial to a joint performance-energy metric.

Iris

On-chip communication, including short, often-multicast, latency-critical coherence and synchronization messages, and long, unicast, throughput-sensitive data transfers, limits the power efficiency and performance scalability of many-core chip-multiprocessor systems. This article analyzes on-chip communication challenges and studies the characteristics of existing electrical and emerging nanophotonic interconnect. Iris, a CMOS-compatible high-performance low-power nanophotonic on-chip network, is thus introduced. Iris's circuit-switched subnetwork supports throughput-sensitive data transfer. Iris's optical-antenna-array-based broadcast--multicast subnetwork optimizes latency-critical traffic and supports the path setup of circuit-switched communication. Overall, the proposed nanophotonic network design offers an on-chip communication backplane that is power efficient while demonstrating low latency and high throughput.

On an efficient NoC multicasting scheme in support of multiple applications running on irregular sub-networks

When a number of applications simultaneously running on a many-core chip multiprocessor (CMP) chip connected through network-on-chip (NoC), significant amount of on-chip traffic is one-to-many (multicast) in nature. As a matter of fact, when multiple applications are mapped onto an NoC architecture with applicable traffic isolation constraints, the corresponding sub-networks of these applications are mapped onto actually tend to be irregular. In the literature, multicasting for irregular topologies is supported through either multiple unicasting or broadcasting, which, unfortunately, results in overly high power consumption and/or long network latency. To address this problem, a simple, yet efficient hardware-based multicasting scheme is proposed in this paper. First, an irregular oriented multicast strategy is proposed. Literally, following this strategy, an irregular oriented multicast routing algorithm can be designed based on any regular mesh based multicast routing algorithm. One such algorithm, namely, Alternative Recursive Partitioning Multicasting (AL + RPM), is proposed based on RPM, which was designed for regular mesh topology originally. The basic idea of AL + RPM is to find the output directions following the basic RPM algorithm and then decide to replicate the packets to the original output directions or the alternative (AL) output directions based on the shape of the sub-network. The experiment results show that the proposed multicast AL + RPM algorithm can consume, on average, 14% and 20% less power than bLBDR (a broadcasting-based routing algorithm) and the multiple unicast scheme, respectively. In addition, AL + RPM has much lower network latency than the above two approaches. To incorporate AL + RPM into a baseline router to support multicasting, the area overhead is fairly modest, less than 5.5%.

Modeling and Stack Simulation of CMP Cache Capacity and Accessibility

Performance trade-offs between fast data access by local data replication and cache capacity maximization by global data sharing have been extensively studied for many-core Chip Multiprocessors (CMPs). Costly simulations over a wide spectrum of the design space are generally required to gain insight for a sound design. To lower the cost, we develop an abstract model for understanding the performance impact of data replication on CMP caches. To overcome the lack of real-time interactions among multiple cores in the model, we further develop an efficient single-pass stack simulation to study the performance of CMP cache organizations with various degrees of data replication. The global stack logically incorporates a shared stack and per-core private stacks; shared/private reuse (stack) distances can be collected in a single-pass simulation. With the reuse distances, one can calculate the performance of CMP cache organizations with various degrees of data replication. We verify both the model and the stack simulation against execution-driven simulations with commercial multithreaded workloads. The results show that the abstract model provides accurate information about performance trade-offs of data replication. The stack simulation accurately predicts the performance of various cache organizations with 2-9 percent error margins using only about 8 percent of the simulation time.

Parallel Scalability of Video Decoders

An important question is whether emerging and future applications exhibit sufficient parallelism, in particular thread-level parallelism, to exploit the large numbers of cores future chip multiprocessors (CMPs) are expected to contain. As a case study we investigate the parallelism available in video decoders, an important application domain now and in the future. Specifically, we analyze the parallel scalability of the H.264 decoding process. First we discuss the data structures and dependencies of H.264 and show what types of parallelism it allows to be exploited. We also show that previously proposed parallelization strategies such as slice-level, frame-level, and intra-frame macroblock (MB) level parallelism, are not sufficiently scalable. Based on the observation that inter-frame dependencies have a limited spatial range we propose a new parallelization strategy, called Dynamic 3D-Wave. It allows certain MBs of consecutive frames to be decoded in parallel. Using this new strategy we analyze the limits to the available MB-level parallelism in H.264. Using real movie sequences we find a maximum MB parallelism ranging from 4000 to 7000. We also perform a case study to assess the practical value and possibilities of a highly parallelized H.264 application. The results show that H.264 exhibits sufficient parallelism to efficiently exploit the capabilities of future manycore CMPs.

Transaction-Aware Network-on-Chip Resource Reservation

Performance and scalability are critically-important for on-chip interconnect in many-core chip-multiprocessor systems. Packet-switched interconnect fabric, widely viewed as the de facto on-chip data communication backplane in the many-core era, offers high throughput and excellent scalability. However, these benefits come at the price of router latency due to run-time multi-hop data buffering and resource arbitration. The network accounts for a majority of on-chip data transaction latency. In this work, we propose dynamic in-network resource reservation techniques to optimize run-time on-chip data transactions. This idea is motivated by the need to preserve existing abstraction and general-purpose network performance while optimizing for frequently-occurring network events such as data transactions. Experimental studies using multithreaded benchmarks demonstrate that the proposed techniques can reduce on-chip data access latency by 28.4% on average in a 16-node system and 29.2% on average in a 36-node system.