Multiprocessor Cache Research Articles

Modeling Remapping Based Fault Tolerance Techniques for Chip Multiprocessor Cache with Design Space Exploration

On top of the wear-out failures and external particle interventions, voltage scaling to mitigate the power consumption in multiprocessor makes cache more vulnerable to cell failures. For the indispensable voltage reduction to prolong the battery life of handheld devices, fault tolerance techniques are extremely important to ensure fault free execution in near-threshold voltage. Several fault tolerance techniques have been proposed and the remapping based techniques are found to be effective to address the issue of fault tolerance in single core systems. This work proposes an analytical model for remapping based fault tolerance techniques to evaluate the effectiveness of such schemes in multicore systems. The metrics Expected Miss Ratio in Multicore (EMRMC) and Expected Latency Ratio in Multicore (ELRMC), are introduced to characterize the behavior of remapping based techniques. The EMRMC and ELRMC are defined as the function of probability of cell failure (Pfail), block size, number of cores and threads. The system is simulated in Multi2sim 5.0, a multicore CPU-GPU simulator. The values of the metrics for different configuration parameters like probability of cell failure, number of cores, number of blocks, block size and number of threads are analysed for framing the guidelines of system configuration to deliver better performance in remapping based fault tolerance. It is observed that the EMRMC is proportional to Pfail and block size but inversely proportional to the number of cores and threads and it is not affected by the number of blocks. On the contrary, the ELRMC is inversely proportional to Pfail and block size and proportional to the number of cores and threads. It is also observed that the ELRMC is independent of the number of cores and blocks. EMRMC is best minimized for Pfail ≤ 1e-4, block size ≤ 64 bytes, number of cores ≥ 4 and number of threads ≥ 2. On the other hand, ELRMC is best observed for Pfail ≤ 1e-4, block size ≥ 64 bytes, number of cores ≥ 4 and number of threads 2.

Thread Criticality Assisted Replication and Migration for Chip Multiprocessor Caches

Non-Uniform Cache Architecture (NUCA) is a viable solution to mitigate the problem of large on-chip wire delay due to the rapid increase in the cache capacity of chip multiprocessors (CMPs). Through partitioning the last-level cache (LLC) into smaller banks connected by on-chip network, the access latency will exhibit non-uniform distribution. Various works have well explored the NUCA design, including block migration, block replication and block searching. However, all of the previous mechanisms designed for NUCA are thread-oblivious when multi-threaded applications are deployed on CMP systems. Due to the interference on shared resources, threads often demonstrate unbalanced progress wherein the lagging threads with slow progress are more critical to overall performance. In this paper, we propose a novel NUCA design called thread C riticality A ssisted R eplication and M igration (CARM). CARM exploits the runtime thread criticality information as hints to adjust the block replication and migration in NUCA. Specifically, CARM aims at boosting parallel application execution through prioritizing block replication and migration for critical threads. Full-system experimental results show that CARM reduces the execution time of a set of PARSEC workloads by 13.7 and 6.8 percent on average compared with the tradition D-NUCA and Re-NUCA respectively. Moreover, CARM also consumes much less energy compared with the evaluated schemes.

A cellular automata based self-correcting protocol processor for scalable CMPs

This work reports an effective design of protocol processor (PP), the key component of coherence controller, in a Chip Multiprocessors (CMPs) cache system. It caters to the need for determining the state of a data block in processors' private caches with high accuracy. An insignificant defect in the PP can introduce major inconsistencies in computing the states of cached copies of a data block. A realizable PP, therefore, is a necessity for accurate computation of cache line states. The proposed design is the outcome of radical change in the design approach of PP. The modeling tool of cellular automata (CA) is considered for the design. The special class of cellular automata with single length cycle attractors (SLCA) is tuned to replicate the PP for computing cache line states in CMPs. Theory has been developed to empower the SLCA based PP to sense its malfunctioning and that leads to the design of a self-correcting PP.

龙芯GS464E处理器核架构设计

Loongson GS464E is the most recent processor architecture introduced by Loongson Technology. In this paper, we describe the key aspects of the GS464E architecture. Compared to the previous GS464 architecture, GS464E focuses on improving the performance of memory access and branch prediction, using enlarged queues, caches, and TLBs in the processor. Support for the MIPS DSP instruction set and virtualization is also provided. Specifically, the memory subsystems have a 3-level cache hierarchy each with an LRU replacement policy, and also support multi-processor cache coherence. With the aforementioned optimization features, GS464E has become an innovative, high-performance processor architecture.

Protozoa

State-of-the-art multiprocessor cache hierarchies propagate the use of a fixed granularity in the cache organization to the design of the coherence protocol. Unfortunately, the fixed granularity, generally chosen to match average spatial locality across a range of applications, not only results in wasted bandwidth to serve an individual thread's access needs, but also results in unnecessary coherence traffic for shared data. The additional bandwidth has a direct impact on both the scalability of parallel applications and overall energy consumption. In this paper, we present the design of Protozoa, a family of coherence protocols that eliminate unnecessary coherence traffic and match data movement to an application's spatial locality. Protozoa continues to maintain metadata at a conventional fixed cache line granularity while 1) supporting variable read and write caching granularity so that data transfer matches application spatial granularity, 2) invalidating at the granularity of the write miss request so that readers to disjoint data can co-exist with writers, and 3) potentially supporting multiple non-overlapping writers within the cache line, thereby avoiding the traditional ping-pong effect of both read-write and write-write false sharing. Our evaluation demonstrates that Protozoa consistently reduce miss rate and improve the fraction of transmitted data that is actually utilized.

Automatic Constraint Based Test Generation for Behavioral HDL Models

With the emergence of complex high-performance microprocessors, functional test generation has become a crucial design step. Constraint-based test generation is a well-studied directed behavioral level functional test generation paradigm. The paradigm involves conversion of a given circuit model into a set of constraints and employing constraint solvers to generate tests for it. However, automatic extraction of constraints from a given behavioral hardware design language (HDL) model remained a challenging open problem. This paper proposes an approach for automatic extraction of word-level model constraints from the behavioral verilog HDL description. The scenarios to be tested are also expressed as constraints. The model and the scenario constraints are solved together using an integer solver to arrive at the necessary functional test. The effectiveness of the approach is demonstrated by automatically generating the constraint models for: 1) an exclusive-shared-invalid multiprocessor cache coherency model and 2) the 16-bit DLX-architecture, from their respective Verilog-based behavioral models. Experimental results that generate test vectors for high level scenarios like pipeline hazards, cache miss, etc., spanning over multiple time-frames are presented.

Using supplier locality in power-aware interconnects and caches in chip multiprocessors

Conventional snoopy-based chip multiprocessors take an aggressive approach broadcasting snoop requests to all nodes. In addition each node checks all received requests. This approach reduces the latency of cache to cache transfer misses at the expense of increasing power. In this paper we show that a large portion of interconnect/cache transactions are redundant as many snoop requests miss in the remote nodes. We exploit this inefficiency and introduce power optimization techniques for chip multiprocessors. Our optimizations rely on the observation that in a snoopy-based shared memory system the data supplier can be predicted with high accuracy. Our optimizations reduce power by eliminating unnecessary activity at both the requester and the supplier end of snoop requests. We reduce power as we (a) avoid broadcasting snoop requests to all processors and (b) avoid tag lookup for all nodes and for all requests arriving. In particular, we use supplier locality and introduce the following two optimizations. First, and at the requester end, we introduce speculative selective request (SSR) to reduce power dissipation in the binary tree interconnect. In SSR, we send the request only to the node more likely to have the missing data. We reduce power as we limit access only to the interconnect components between the requestor and the supplier node. Second, and at the supplier end, we propose speculative tag lookup (STL) to reduce power dissipation in data caches. We filter those accesses more likely to miss in the L1 cache. Using shared memory applications, we show that by limiting snoop requests to the speculated nodes we reduce interconnect power by 25% in a four-way multiprocessor. Moreover, we show that speculative tag lookup reduces power in tag arrays by 14.1% in a four-way multiprocessor. Both optimizations come with negligible performance loss and hardware overhead.

Environment for multiprocessor simulator development

AbstractTo develop multiprocessor systems for continuing the precipitous increase in performance seen in recent years, it is important to include performance prediction at a preimplementation stage. Since software simulation, which is one such performance prediction technique, has a high degree of flexibility, it is used in diverse situations. To simulate a multiprocessor system, a simulator is implemented for a specific architecture. However, this method has accompanying implementation costs for building a simulator for each different target architecture. In this paper, the authors propose the multiprocessor simulator library ISIS as a multiprocessor simulator construction support system. ISIS is a collection of multiprocessor system internal function block simulators. Multiprocessor simulators having any kind of architecture can be constructed by combining function block simulators. Compared with conventional techniques, implementation costs can be reduced without directly impairing simulator runtime costs. Several multiprocessor simulators were implemented by using ISIS, and their execution speeds and implementation costs were evaluated. Sufficient runtime speeds were obtained for practical uses at extremely low implementation costs. ISIS has actually been used in research for on‐chip multiprocessor cache systems, and various kinds of knowledge have been obtained from the evaluation results of that research. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(10): 74–84, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10122

Fresh Breeze

It is well-known that multiprocessor systems are vastly more difficult to program than systems that support sequential programming models. In a 1998 paper[11] this author argued that six important principles for supporting modular software construction are often violated by the architectures proposed for multiprocessor computer systems. The Fresh Breeze project concerns the architecture and design of a multiprocessor chip that can achieve superior performance while honoring these six principles.The envisioned multiprocessor chip will incorporate three ideas that are significant departures from mainstream thinking about multiprocessor architecture: (1) Simultaneous multithreading has been shown to have performance advantages relative to contemporary superscalar designs. This advantage can be exploited through use of a programming model that exposes parallelism in the form of multiple threads of computation. (2) The value of a shared address space is widely appreciated. Through the use of 64-bit pointers, the conventional distinction between "memory" and the file system can be abolished. This can provide a superior execution environment in support of program modularity and software reuse, as well as supporting multi-user data protection and security that is consistent with modular software structure. (3) No memory update; cycle-free heap. Data items are created, used, and released, but never modified once created. The allocation, release, and garbage collection of fixed-size chunks of memory will be implemented by efficient hardware mechanisms. A major benefit of this choice is that the multiprocessor cache coherence problem vanishes: any object retrieved from the memory system is immutable. In addition, it is easy to prevent the formation of pointer cycles, simplifying the design of memory management support.

Multiprocessor enhancements of the SimpleScalar tool set

This paper describes multiprocessor enhancements of the SimpleScalar tool set. The core simulation code has been modified to support multiprocessing, and a run-time library has been introduced for thread creation and synchronization. Measurements using the SPLASH-2 parallel benchmark suite [13] indicate that the multiprocessor enhancements introduce simulation overhead of 30%-50% relative to the original uniprocessor simulator. An idealized multiprocessor cache simulator has also been developed, and a dynamic visualization tool for cache coherence has also been developed. These multiprocessor enhancements are available at the WWW site for the SimpleScalar tool set.

Packaging technology for high performance CMOS server Fujitsu GS8900

Advanced packaging technologies for CMOS based high performance Fujitsu Global Server GS8900, released in late 1999, are introduced in this paper. Extending a new standard for technological leadership among large-scale enterprise servers, the GS8900 broke the 2000 MIPS barrier in performance for the first time by taking advantages of Fujitsu advanced 0.18 /spl mu/m copper wiring process and chip/MCM/system packaging capabilities, delivering a doubled performance in comparison to its predecessor. The packaging technologies are uniquely characterized in several aspects. First, the high density stacked via type MCM-D technology features four pairs of CPU tightly coupled multiprocessor and large capacity second caches, the maximum processor terminal count is more than 10,000. The processors are wired onto a multilayer thin film MCM substrate with 153 /spl mu/m pitch high-density area array lead-free bumps. Secondly, maximum four CPU-MCMs, including 16 CPU processors, and 64 GB main memory modules are mounted on one multilayer system board of high frequency transmission properties. Each MCM is held through a high-density ZIF connector of around 3000 I/Os in a 1.27 mm pitch full matrix, which is assembled on the system board with lead-free solders. Thirdly, advanced cooling technologies are developed for improving the system performance and reliability.

Location consistency-a new memory model and cache consistency protocol

Existing memory models and cache consistency protocols assume the memory coherence property which requires that all processors observe the same ordering of write operations to the same location. In this paper, we address the problem of defining a memory model that does not rely on the memory coherence assumption and also the problem of designing a cache consistency protocol based on such a memory model. We define a new memory consistency model, called Location Consistency (LC), in which the state of a memory location is modeled as a partially ordered multiset (pomset) of write and synchronization operations. We prove that LC is strictly weaker than existing memory models, but is still equivalent to stronger models for the common case of parallel programs that have no data races. We also describe a new multiprocessor cache consistency protocol based on the LC memory model. We prove that this LC protocol obeys the LC memory model. The LC protocol does not need to enforce single write ownership of memory blocks. As a result, the LC protocol is simpler and more scalable than existing snooping and directory-based cache consistency protocols.

Adaptive memory management and optimism control in time warp

It is widely believed that the Time Warp protocol for parallel discrete event simulation is prone to two potential problems: an excessive amount of wasted, rolled back computation resulting from “rollback thrashing” behaviors, and inefficient use of memory, leading to poor performance of virtual memory and/or multiprocessor cache systems. An adaptive mechanism is proposed based on the Cancelback memory management protocol for shared-memory multiprocessors that dynamically controls the amount of memory used in the simulation in order to maximize performance. The proposed mechanism is adaptive in the sense that it monitors the execution of the Time Warp program, and using simple models, automatically adjusts the amount of memory used to reduce Time Warp overheads (fossil collection, Cancelback, the amount of rolled back computation, etc.) to a manageable level. We describe an implementation of this mechanism on a shared memory, Kendall Square Research KSR-1, multiprocessor and demonstrate its effectiveness in automatically maximizing performance while minimizing memory utilzation, for several synthetic and benchmark discrete event simulation applications. We also demonstrate the adaptive ability of the mechanism by showing that it “tracks” the time-varying nature of a communication network simulation.

Cache-Based Synchronization in Shared Memory Multiprocessors

In shared memory multiprocessors, efficient synchronization is imperative to assure good performance. There are two aspects to the “cost” of a synchronization operation: the first is the waiting time at synchronization points, and the second is the intrinsic overhead in performing the operation. The overhead has two components. The first component deals with contention resolution for synchronization operation among competing processors. The second component deals with the shared data accesses that the processor has to perform once it enters a synchronization region. We present a mechanism to reduce the overhead of performing synchronization operations in a cache-based shared memory multiprocessor. The mechanism is based on the intuitive notion that parallel programs invariably use synchronization operations to govern the access to shared data. Traditional multiprocessor cache protocols treat synchronization accesses the same way as normal read/write memory accesses, leading to inefficiencies in performing synchronization operations which ultimately limit the scalability of such systems. The key idea in our approach is to combine synchronization with the coherence maintenance for the cached data. Each cache line maintains states for synchronization as well as for cache coherence, and the cache protocol ensures the correctness of the synchronization operations and the coherence of the data at these synchronization points. To assess the performance gain due to the proposed mechanism, simulation studies are performed using a workload model that represents a dynamic scheduling paradigm which forms the core of several parallel programs. Results from simulation studies show that the proposed cache-based synchronization performs significantly better than traditional cache coherence approaches.

Will parallel simulation come to an end?

In 1987, I worked on the performance issues of the multi-processor caches under the supervision of Professor Jean-Loup Baer. We used the trace-driven simulation approach and found that the process might be too time consuming. Then we investigated the possibility of utilizing multiple processors to speed up the simulation procedure. At this point, my thesis advisor Professor Edward Lazowska joined in the project. By exploiting the communication patterns of the cache protocols under study, we observed the number of synchronization operations among the processors (that execute the cache simulation) can be significantly reduced. Thus, good speedup could be achieved. We published this exciting results in the Proceedings of the SCS Multi-Conferences on Distributed Simulation. After we finished this project, I felt that "It is too expensive to develop a parallel simulation program just for one application. It is desirable to develop a general simulation system." Jean-Loup showed me two papers:1. Chandy, K. M., and Misra, J., "Distributed Simulation: A Case Study in Design and Verification of Distributed Programs." IEEE Trans. on Software Engineering, SE-5, 1979.2. Jefferson, D., "Virtual Time." ACM Transactions on Programming Languages and Systems, July, 1985.

Stack evaluation of arbitrary set-associative multiprocessor caches

We propose a simple solution to the problem of efficient stack evaluation of LRU multiprocessor cache memories with arbitrary set-associative mapping. It is an extension of the existing stack evaluation techniques for all set-associative LRU uniprocessor caches. Special marker entries are used in the stack to represent data blocks (or lines) deleted by an invalidation-based cache coherence protocol. A method of marker-splitting is employed when a data block below a marker in the stack is accessed. Using this technique, one-pass trace evaluation of memory access trace yields hit ratios for all cache sizes and set-associative mappings of multiprocessor caches in a single pass over a memory reference trace. Simulation experiments on some multiprocessor trace data show an order-of-magnitude speed-up in simulation time using this one-pass technique.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

False sharing and spatial locality in multiprocessor caches

The performance of the data cache in shared-memory multiprocessors has been shown to be different from that in uniprocessors. In particular, cache miss rates in multiprocessors do not show the sharp drop typical of uniprocessors when the size of the cache block increases. The resulting high cache miss rate is a cause of concern, since it can significantly limit the performance of multiprocessors. Some researchers have speculated that this effect is due to false sharing, the coherence transactions that result when different processors update different words of the same cache block in an interleaved fashion. While the analysis of six applications in the paper confirms that false sharing has a significant impact on the miss rate, the measurements also show that poor spatial locality among accesses to shared data has an even larger impact. To mitigate false sharing and to enhance spatial locality, we optimize the layout of shared data in cache blocks in a programmer-transparent manner. We show that this approach can reduce the number of misses on shared data by about 10% on average.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An adaptive memory management protocol for Time Warp parallel simulation

It is widely believed that Time Warp is prone to two potential problems: an excessive amount of wasted, rolled back computation resulting from “rollback thrashing” behaviors, and inefficient use of memory, leading to poor performance of virtual memory and/or multiprocessor cache systems. An adaptive mechanism is proposed based on the Cancelback memory management protocol that dynamically controls the amount of memory used in the simulation in order to maximize performance. The proposed mechanism is adaptive in the sense that it monitors the execution of the Time Warp program, automatically adjusts the amount of memory used to reduce Time Warp overheads (fossil collection, Cancelback, the amount of rolled back computation, etc.) to a manageable level. The mechanism is based on a model that characterizes the behavior of Time Warp programs in terms of the flow of memory buffers among different buffer pools. We demonstrate that an implementation of the adaptive mechanism on a Kendall Square Research KSR-1 multiprocessor is effective in automatically maximizing performance while minimizing memory utilization of Time Warp programs, even for dynamically changing simulation models.

Working sets, cache sizes, and node granularity issues for large-scale multiprocessors

The distribution of resources among processors, memory and caches is a crucial question faced by designers of large-scale parallel machines. If a machine is to solve problems with a certain data set size, should it be built with a large number of processors each with a small amount of memory, or a smaller number of processors each with a large amount of memory? How much cache memory should be provided per processor for cost-effectiveness? And how do these decisions change as larger problems are run on larger machines? In this paper, we explore the above questions based on the characteristics of five important classes of large-scale parallel scientific applications. We first show that all the applications have a hierarchy of well-defined per-processor working sets, whose size, performance impact and scaling characteristics can help determine how large different levels of a multiprocessor's cache hierarchy should be. Then, we use these working sets together with certain other important characteristics of the applications—such as communication to computation ratios, concurrency, and load balancing behavior—to reflect upon the broader question of the granularity of processing nodes in high-performance multiprocessors. We find that very small caches whose sizes do not increase with the problem or machine size are adequate for all but two of the application classes. Even in the two exceptions, the working sets scale quite slowly with problem size, and the cache sizes needed for problems that will be run in the foreseeable future are small. We also find that relatively fine-grained machines, with large numbers of processors and quite small amounts of memory per processor, are appropriate for all the applications.

Verification of a multiprocessor cache protocol using simulation relations and higher-order logic

We present a formal verification method for concurrent systems. The technique is to show a correspondence between state machines representing an implementation and specification behaviour. The correspondence is called a simulation relation, and is particularly well-suited for theorem-provers. Since the method does not rely on enumerating all the states, it can be applied to systems with an infinite or unknown number of states. This substantially expands the class of hardware designs that can be formally verified. The method is illustrated by proving the correctness of a particularly subtle example which is likely to be of increasing importance: a directory-based multiprocessor cache protocol. The proof is carried out using the HOL (“higher-order logic”) theorem-prover.