Level Data Cache Research Articles

Early miss prediction based periodic cache bypassing for high performance GPUs

The aim of the hierarchical cache memories that are equipped for GPUs is the management of irregular memory access patterns for general purpose workloads. The level-1 data cache (L1D) of the GPU plays an important role for its ability in the provision of high bandwidth and low-latency data accesses. Unfortunately, the GPU L1D may become a performance bottleneck due to facing many performance challenges such as cache contention and resource congestion. These critical issues come from a large number of simultaneous requests from the SIMT cores to the limited-capacity L1D. We observe that many applications have a large number of requests with a very low reuse probability, resulting in the GPU performance degradation. To overcome these challenges, we propose an efficient cache bypassing mechanism that can periodically filter the access stream and make an accurate bypassing decision to improve the efficiency of the L1D. The proposed technique uses a small storage amount to save the tag array of the L1D for the early miss prediction before it makes the bypassing decision. The experiment results reveal that the proposed technique significantly increases the cache efficiency and the GPU performance.

MeRLiN

Early reliability assessment of hardware structures using microarchitecture level simulators can effectively guide major error protection decisions in microprocessor design. Statistical fault injection on microarchitectural structures modeled in performance simulators is an accurate method to measure their Architectural Vulnerability Factor (AVF) but requires excessively long campaigns to obtain high statistical significance. We propose MeRLiN1, a methodology to boost microarchitecture level injection-based reliability assessment by several orders of magnitude and keep the accuracy of the assessment unaffected even for large injection campaigns with very high statistical significance. The core of MeRLiN is the grouping of faults of an initial list in equivalent classes. All faults in the same group target equivalent vulnerable intervals of program execution ending up to the same static instruction that reads the faulty entries. Faults in the same group occur in different times and entries of a structure and it is extremely likely that they all have the same effect in program execution; thus, fault injection is performed only on a few representatives from each group. We evaluate MeRLiN for different sizes of the physical register file, the store queue and the first level data cache of a contemporary microarchitecture running MiBench and SPEC CPU2006 benchmarks. For all our experiments, MeRLiN is from 2 to 3 orders of magnitude faster than an extremely high statistical significant injection campaign, reporting the same reliability measurements with negligible loss of accuracy. Finally, we theoretically analyze MeRLiN's statistical behavior to further justify its accuracy.

Designing a practical data filter cache to improve both energy efficiency and performance

Conventional Data Filter Cache (DFC) designs improve processor energy efficiency, but degrade performance. Furthermore, the single-cycle line transfer suggested in prior studies adversely affects Level-1 Data Cache (L1 DC) area and energy efficiency. We propose a practical DFC that is accessed early in the pipeline and transfers a line over multiple cycles. Our DFC design improves performance and eliminates a substantial fraction of L1 DC accesses for loads, L1 DC tag checks on stores, and data translation lookaside buffer accesses for both loads and stores. Our evaluation shows that the proposed DFC can reduce the data access energy by 42.5% and improve execution time by 4.2%.

A SystemC Cache Simulator for a Multiprocessor Shared Memory System

In this research we built a SystemC Level-1 data cache system in a distributed shared memory architectural environment, with each processor having its own local cache. Using a set of Fast-Fourier Transform and Random trace files we evaluated the cache performance, based on the number of cache hits/misses, of the caches using snooping and directory-based cache coherence protocols. A series of experiments were carried out, with the results of the experiments showing that the directory-based MOESI cache coherency protocol has a performance edge over the snooping Valid-Invalid cache coherency protocol.

SS-SERA: An improved framework for architectural level soft error reliability analysis

Integrated with an improved architectural vulnerability factor (AVF) computing model, a new architectural level soft error reliability analysis framework, SS-SERA (soft error reliability analysis based on SimpleScalar), was developed. SS-SERA was used to estimate the AVFs for various on-chip structures accurately. Experimental results show that the AVFs of issue queue (IQ), register update units (RUU), load store queue (LSQ) and functional unit (FU) are 38.11%, 22.17%, 23.05% and 24.43%, respectively. For address-based structures, i.e., level1 data cache (L1D), DTLB, level2 unified cache (L2U), level1 instruction cache (L1I) and ITLB, AVFs of their data arrays are 22.86%, 27.57%, 14.80%, 8.25% and 12.58%, lower than their tag arrays’ AVFs which are 30.01%, 28.89%, 17.69%, 10.26% and 13.84%, respectively. Furthermore, using the AVF values obtained with SS-SERA, a qualitative and quantitative analysis of the AVF variation and predictability was performed for the structures studied. Experimental results show that the AVF exhibits significant variations across different structures and workloads, and is influenced by multiple microarchitectural metrics and their interactions. Besides, AVFs of SPEC2K floating point programs exhibit better predictability than SPEC2K integer programs.

REDUCING CACHE HIERARCHY ENERGY CONSUMPTION BY PREDICTING FORWARDING AND DISABLING ASSOCIATIVE SETS

The first level data cache in modern processors has become a major consumer of energy due to its increasing size and high frequency access rate. In order to reduce this high energy consumption, we propose in this paper a straightforward filtering technique based on a highly accurate forwarding predictor. Specifically, a simple structure predicts whether a load instruction will obtain its corresponding data via forwarding from the load-store structure — thus avoiding the data cache access — or if it will be provided by the data cache. This mechanism manages to reduce the data cache energy consumption by an average of 21.5% with a negligible performance penalty of less than 0.1%. Furthermore, in this paper we focus on the cache static energy consumption too by disabling a portion of sets of the L2 associative cache. Overall, when merging both proposals, the combined L1 and L2 total energy consumption is reduced by an average of 29.2% with a performance penalty of just 0.25%.

Applying architectural vulnerability Analysis to hard faults in the microprocessor

In this paper, we present a new metric, Hard-Fault Architectural Vulnerability Factor (H-AVF), to allow designers to more effectively compare alternate hard-fault tolerance schemes. In order to provide intuition on the use of H-AVF as a metric, we evaluate fault-tolerant level-1 data cache and register file implementations using error correcting codes and a fault-tolerant adder using triple-modular redundancy (TMR). For each of the designs, we compute its H-AVF. We then use these H-AVF values in conjunction with other properties of the design, such as die area and power consumption, to provide composite metrics. The derived metrics provide simple, quantitative measures of the cost-effectiveness of the evaluated designs.

Exploiting Narrow Values for Soft Error Tolerance

Soft errors are an important challenge in contemporary microprocessors. Particle hits on the components of a processor are expected to create an increasing number of transient errors with each new microprocessor generation. In this paper we propose simple mechanisms that effectively reduce the vulnerability to soft errors In a processor. Our designs are generally motivated by the fact that many of the produced and consumed values in the processors are narrow and their upper order bits are meaningless. Soft errors canted by any particle strike to these higher order bits can be avoided by simply identifying these narrow values. Alternatively soft errors can be detected or corrected on the narrow values by replicating the vulnerable portion of the value inside the storage space provided for the upper order bits of these operands. We offer a variety of schemes that make use of narrow values and analyze their efficiency in reducing soft error vulnerability of level-1 data cache of the processor

Design and Fabrication of 3D Microprocessors

ABSTRACTStacking multiple device strata can improve system performance of a microprocessor (μP) by reducing interconnect length. This enables latency improvement, power reduction, and improved memory bandwidth. In this paper we review some of our recent design analysis and process results which quantitatively show the benefits of stacking applied to μPs.We report on two applications for stacking which take advantage of reduced wire length- “logic+logic” stacking and “logic+memory” stacking. In addition to optimizing minimum wire length, we considered carefully the thermal ramifications of the new designs. For the logic+memory application, we considered the case of reducing off-die wiring by stacking a DRAM cache (32 to 64MB) onto a high performance μP. Simulations showed 3x reduced off-die bandwidth, Cycles Per Memory Access (CPMA) reduction of 13%, and a 66% average bus power reduction. For logic+logic applications, we considered a high performance μP where the unit blocks were repartitioned into two strata. For this case, simulations showed that stacking can simultaneously reduce power by 15% while increasing performance by 15% with a minor 14° C increase in peak temperature compared to the planar design. Using voltage scaling, this translates to 34% power reduction and 8% performance improvement with no temperature increase. We found that these results can be further improved by a secondary splitting of the individual blocks. As an example, we split a 32KB first level data cache resulting in 25% power reduction, 10% latency reduction, and 20% area reduction.We also discuss the fabrication of stacked structures with two complimentary process flows. In one case, we developed a 300mm wafer stacking process using Cu-Cu bonding, wafer thinning, and through-silicon vias (TSVs). This technology provides reliable bonding with non-detectable bonding-interface resistance and inter-strata via pitch below 8μm. We investigated the impact of this wafer stacking process to the transistor and interconnect layers built using a 65nm strained-Si/Cu-Low-K process technology and found no impact to either discrete N- and P-MOS devices or to thin 4Mb SRAMs. We verified fully functional SRAMs on thinned wafers with thicknesses down to 5μm. Although wafer stacking leads itself well to tight-pitch same-die-size stacking, die stacking enables integration of different size dies and includes opportunity to improve yield by stacking known good dies. We demonstrated a die stack process flow with 75μm thinned die, TSV, and inter-strata via pitch below 100μm. We also found negligible impact to transistors using this process flow. Multiple stacks of up to seven 75μm thin dies with TSVs were fabricated and tested. Prospects for high volume integration of 3D into μPs are discussed.

Static energy reduction techniques for microprocessor caches

Microprocessor performance has been improved by increasing the capacity of on-chip caches. However, the performance gain comes at the price of increased static energy consumption due to sub-threshold leakage current. This paper compares three techniques for reducing static energy consumption in on-chip level-1 and level-2 caches. One technique employs low-leakage transistors in the memory cell. Another technique, power supply switching can be used to turn off the memory cells and discard their contents. A third alternative is dynamic threshold modulation, which places the memory cells in a standby state that preserves cell contents. In our experiments, we explore the energy/performance trade-offs of these techniques and find that the dynamic threshold modulation achieves the best results for level-1 caches, improving the energy-delay product by 2% in a level-1 instruction cache and 7% in a level-1 data cache. Low-leakage transistors perform best for the level-2 cache as they reduce the static energy by up to 98% and improve the energy-delay product by more than a factor of 50.