SPEC CPU2006 Benchmarks Research Articles

An SSA-based algorithm for optimal speculative code motion under an execution profile

To derive maximum optimization benefits from partial redundancy elimination (PRE),it is necessary to go beyond its safety constraint. Algorithms for optimal speculative code motion have been developed based on the application of minimum cut to flow networks formed out of the control flow graph. These previous techniques did not take advantage of the SSA form, which is a popular program representation widely used in modern-day compilers. We have developed the MC-SSAPRE algorithm that enables an SSA-based compiler to take full advantage of SSA to perform optimal speculative code motion efficiently when an execution profile is available. Our work shows that it is possible to form flow networks out of SSA graphs, and the min-cut technique can be applied equally well on these flow networks to find the optimal code placement. We provide proofs of the correctness and computational and lifetime optimality of MC-SSAPRE. We analyze its time complexity to show its efficiency advantage. We have implemented MC-SSAPRE in the open-sourced Path64 compiler. Our experimental data based on the full SPEC CPU2006 Benchmark Suite show that MC-SSAPRE can further improve program performance over traditional SSAPRE, and that our sparse approach to the problem does result in smaller problem sizes.

Global-aware and multi-order context-based prefetching for high-performance processors

Data prefetching is widely used in high-end computing systems to accelerate data accesses and to bridge the increasing performance gap between processor and memory. Context-based prefetching has become a primary focus of study in recent years due to its general applicability. However, current context-based prefetchers only adopt the context analysis of a single order, which suffers from low prefetching coverage and thus limits the overall prefetching effectiveness. Also, existing approaches usually consider the context of the address stream from a single instruction but not the context of the address stream from all instructions, which further limits the context-based prefetching effectiveness. In this study, we propose a new context-based prefetcher called the Global-aware and Multi-order Context-based (GMC) prefetcher. The GMC prefetcher uses multi-order, local and global context analysis to increase prefetching coverage while maintaining prefetching accuracy. In extensive simulation testing of the SPEC-CPU2006 benchmarks with an enhanced CMP$im simulator, the proposed GMC prefetcher was shown to outperform existing prefetchers and to reduce the data-access latency effectively. The average Instructions Per Cycle (IPC) improvement of SPEC CINT2006 and CFP2006 benchmarks with GMC prefetching was over 55% and 44% respectively.

WiDGET

The recent paradigm shift to multi-core systems results in high system throughput within a specified power budget. However, future systems still require good single thread performance--no longer the predominant design priority--to mitigate sequential bottlenecks and/or to guarantee service-level agreements. Unfortunately, near saturation in voltage scaling necessitates a long-term alternative to dynamic voltage and frequency scaling. We propose an energy-proportional computing infrastructure, called WiDGET, that decouples thread context management from a sea of simple execution units (EUs). WiDGET's decoupled design provides flexibility to alter resource allocation for a particular power-performance target while turning off unallocated resources. In other words, WiDGET enables dynamic customization of different combinations of small and/or powerful cores on a single chip, consuming power in proportion to the delivered performance. Over all SPEC CPU2006 benchmarks, WiDGET provides average per-thread performance that is 26% better than a Xeon-like processor while using 8% less power. WiDGET can also scale down to a level comparable to an Atom-like processor, turning off resources to reduce average power by 58%. WiDGET achieves high power efficiency (BIPS 3 /W), exceeding Xeon-like and Atom-like processors by up to 2x and 21x, respectively.

Using SPEC CPU2006 to evaluate the sequential and parallel code generated by commercial and open-source compilers

The role of the compiler is fundamental to exploit the hardware capabilities of a system running a particular application, minimizing the sequential execution time and, in some cases, offering the possibility of parallelizing part of the code automatically. This paper relies on the SPEC CPU2006 v1.1 benchmark suite to evaluate the performance of the code generated by three widely-used compilers (Intel C++/Fortran Compiler 11.0, Sun Studio 12 and GCC 4.3.2). Performance is measure in terms of base speed for reference problem sizes. Both sequential and automatic parallel performance obtained is analyzed, using different hardware architectures and configurations. The study includes a detailed description of the different problems that arise while compiling SPEC CPU2006 benchmarks with these tools, an information difficult to obtain elsewhere.

Multi-core scalability-impacting compiler optimizations

In a multi-core system, while the processor core pipelines and local caches are replicated in each core, other resources, such as shared cache and memory bus, are shared across all cores in a processor. Running multiple copies of memory intensive applications on different cores often leads to poor scaling, because these shared resources can become bottlenecks to throughput performance. Such issues have been traditionally studied under the design and evaluation of processors, platforms, and operating systems. We have identified a set of compiler optimizations that have measurable impact on the scaling of applications on multi-core systems and evaluated them based on the standard rate run of the SPEC CPU2006 benchmark suite, where throughput is measured by running multiple copies of a program in a multi-core and multi-processor system. We have also collected data and analyzed how these compiler optimizations affect the utilization and behaviors of the shared resources. Through our experimental results, we show that conventional compiler optimizations can play an important role in improving the scaling of running memory intensive application threads on multi-core systems.

Dynamic Program Behavior Identification for High Performance CMPs with Private LLCs

Chip Multi-Processors (CMPs) emerge as a mainstream architectural design alternative for high performance parallel and distributed computing. Last Level Cache (LLC) management is critical to CMPs because off-chip accesses often require a long latency. Due to its short access latency, well performance isolation and easy scalability, private cache is an attractive design alternative for LLC of CMPs. This paper proposes program Behavior Identification-based Cache Sharing (BICS) for LLC management. BICS is based on a private cache organization for the shorter access latency. Meanwhile, BICS tries to simulate a shared cache organization by allowing evicted blocks of one private LLC to be saved at peer LLCs. This technique is called spilling. BICS identifies cache behavior types of applications at runtime. When a cache block is evicted from a private LLC, cache behavior characteristics of the local application are evaluated so as to determine whether the block is to be spilled. Spilled blocks are allowed to replace some valid blocks of the peer LLCs as long as the interference is within a reasonable level. Experimental results using a full system CMP simulator show that BICS improves the overall throughput by as much as 14.5%, 12.6%, 11.0%and 11.7% (on average 8.8%, 4.8%, 4.0% and 6.8%) over private cache, shared cache, Utility-based Cache Partitioning (UCP) scheme and the baseline spilling-based organization Cooperative Caching (CC) respectively on a 4-core CMP for SPEC CPU2006 benchmarks.

Chip Multiprocessor Design Space Exploration through Statistical Simulation

Developing fast chip multiprocessor simulation techniques is a challenging problem. Solving this problem is especially valuable for design space exploration purposes during the early stages of the design cycle where a large number of design points need to be evaluated quickly. This paper studies statistical simulation as a fast simulation technique for chip multiprocessor (CMP) design space exploration. The idea of statistical simulation is to measure a number of program execution characteristics from a real program execution through profiling, to generate a synthetic trace from it, and simulate that synthetic trace as a proxy for the original program. The important benefit is that the synthetic trace is much shorter compared to a real program trace, which leads to substantial simulation speedups. This paper enhances state-of-the-art statistical simulation: 1) by modeling the memory address stream behavior in a more microarchitecture-independent way and 2) by modeling a program's time-varying execution behavior. These two enhancements enable accurately modeling resource conflicts in shared resources as observed in the memory hierarchy of contemporary chip multiprocessors when multiple programs are coexecuting on the CMP. Our experimental evaluation using the SPEC CPU benchmarks demonstrates average prediction error of 7.3 percent across a range of CMP configurations while varying the number of cores and memory hierarchy configurations.

Access region cache with register guided memory reference partitioning

Wide-issue and high-frequency processors require not only a low-latency but also high-bandwidth memory system to achieve high performance. Previous studies have shown that using multiple small single-ported caches instead of a monolithic large multi-ported one for L1 data cache can be a scalable and inexpensive way to provide higher bandwidth. Various schemes on how to direct the memory references have been proposed in order to achieve a close match to the performance of an ideal multi-ported cache. However, most existing designs seldom take dynamic data access patterns into consideration, thus suffer from access conflicts within one cache and unbalanced loads between the caches. It is observed in this paper that if one can group data references defined in a program into several regions (access regions) to allow parallel accesses, providing separate small caches – access region cache for these regions may prove to have better performance. A register-guided memory reference partitioning approach is proposed and it effectively identifies these semantic regions and organizes them into multiple caches adaptively to maximize concurrent accesses. The base register name, not its content, in the memory reference instruction is used as a basic guide for instruction steering. With the initial assignment to a specific access region cache per the base register name, a reassignment mechanism is applied to capture the access pattern when program is moving across its access regions. In addition, a distribution mechanism is introduced to adaptively enable access regions to extend or shrink among the physical caches to reduce potential conflicts further. The simulations of SPEC CPU2000 benchmarks have shown that the semantic-based scheme can reduce the conflicts effectively, and obtain considerable performance improvement in terms of IPC; with 8 access region caches, 25–33% higher IPC is achieved for integer benchmark programs than a comparable 8-banked cache, while the benefit is less for floating-point benchmark programs, 19% at most.

Exploiting Memory Soft Redundancy for Joint Improvement of Error Tolerance and Access Efficiency

Technology roadmap projects nanoscale multibillion- transistor integration in the coming years. However, on-chip memory becomes increasingly exposed to the dual challenges of device-level reliability degradation and architecture-level performance gap. In this paper, we propose to exploit the inherent memory soft (transient) redundancy for on-chip memory design. Due to the mismatch between fixed cache line size and runtime variations in memory spatial locality, many irrelevant data are fetched into the memory thereby wasting memory spaces. The proposed soft-redundancy allocated memory detects and utilizes these memory spaces for jointly achieving efficient memory access and effective error control. A runtime reconfiguration scheme is also proposed to further enhance the soft-redundancy allocation. Simulation results demonstrate 74.8% average error-control coverage ratio on the SPEC CPU2000 benchmarks with average of 59.5% and 41.3% reduction in memory miss rate and bandwidth usage, respectively, as compared to the existing memory techniques. Furthermore, the proposed technique is fully scalable with respect to various memory configurations.

Techniques for efficient placement of synchronization primitives

Harnessing the hardware parallelism of the emerging multi-cores systems necessitates concurrent software. Unfortunately, most of the existing mainstream software is sequential in nature. Although one could auto-parallelize a given program, the efficacy of this is largely limited to floating-point codes. One of the ways to alleviate the above limitation is to parallelize programs, which cannot be auto-parallelized, via explicit synchronization. In this regard, efficient placement of the synchronization primitives - say, post, wait - plays a key role in achieving high degree of thread-level parallelism ( TLP ). In this paper, we propose novel compiler techniques for the above. Specifically, given a control flow graph ( CFG ), the proposed techniques place a post as early as possible and place a wait as late as possible in the CFG, subject to dependences. We demonstrate the efficacy of our techniques, on a real machine, using real codes, specifically, from the industry-standard SPEC CPU benchmarks, the Linux kernel and other widely used open source codes. Our results show that the proposed techniques yield significantly higher levels of TLP than the state-of-the-art.

Exploiting Memory Soft Redundancy for Joint Improvement of Error Tolerance and Access Efficiency

Technology roadmap projects nanoscale multibillion- transistor integration in the coming years. However, on-chip memory becomes increasingly exposed to the dual challenges of device-level reliability degradation and architecture-level performance gap. In this paper, we propose to exploit the inherent memory soft ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">transient</i> ) <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">redundancy</i> for on-chip memory design. Due to the mismatch between fixed cache line size and runtime variations in memory spatial locality, many irrelevant data are fetched into the memory thereby wasting memory spaces. The proposed soft-redundancy allocated memory detects and utilizes these memory spaces for jointly achieving efficient memory access and effective error control. A runtime reconfiguration scheme is also proposed to further enhance the soft-redundancy allocation. Simulation results demonstrate 74.8% average error-control coverage ratio on the SPEC CPU2000 benchmarks with average of 59.5% and 41.3% reduction in memory miss rate and bandwidth usage, respectively, as compared to the existing memory techniques. Furthermore, the proposed technique is fully scalable with respect to various memory configurations.

BAT: Performance-Driven Crosstalk Mitigation Based on Bus-Grouping Asynchronous Transmission

Crosstalk delay within an on-chip bus can induce severe transmission performance penalties. The Bus-grouping Asynchronous Transmission (BAT) scheme is proposed to mitigate the performance degradation. Furthermore, considering the distinct spatial locality of transition distribution on some types of buses, we use the locality to optimize the BAT. In terms of the implementation, we propose the Differential Counter Cluster (DCC) synchronous mechanism to synchronize the data transmission, and the Delay Active Shielding (DAS) to protect some critical signals from crosstalk and optimize the routing area overhead. The BAT is scalable with the variation of bus width with little extra implementation complexity. The effectiveness of the BAT is evaluated by focusing on the on-chip buses of a superscalar microprocessor simulator using the SPEC CPU2000 benchmarks. When applied to a 64-bit on-chip instruction bus, the BAT scheme, compared with the conservative approach, Codec and Variable Cycle Transmission (DYN) approaches, improves performance by 55+%, 10+%, 30+%, respectively, at the expense of 13% routing area overhead.

Improving Error Tolerance for Multithreaded Register Files

Chip multithreaded computing is exposed to the dual challenges of increasing system complexity and error sensitivity. It is critical to develop effective solutions that achieve better error tolerance without inducing performance degradation. In this paper, we propose a new error-tolerant memory design based on a unique computing phenomenon referred to as the dynamic multithreading redundancy (DMR). The proposed technique exploits the interplay between the concurrent threads for runtime error control. We also present two DMR enhancements, immediate write-back and self-recovery, to address the error accumulation effect. A multithreaded register file was implemented to demonstrate the proposed DMR technique. Simulation results on the SPEC CPU2000 benchmarks demonstrate significant overhead reduction in performance and energy efficiency related to error recovery. In addition, the proposed technique features good scalability with respect to the instruction-level and thread-level parallelism for next-generation processor design, where the soft error problem is expected to get worse due to technology scaling and architecture-affecting trends.

Self-Adaptive Data Caches for Soft-Error Reliability

Soft-error induced reliability problems have become a major challenge in designing new generation microprocessors. Due to the on-chip caches' dominant share in die area and transistor budget, protecting them against soft errors is of paramount importance. Recent research has focused on the design of cost-effective reliable data caches in terms of performance, energy, and area overheads, based on the assumption of fixed error rates. However, for systems in operating environments that vary with time or location, those schemes will be either insufficient or overdesigned for the changing error rates. In this paper, we explore the design of a self-adaptive reliable data cache that dynamically adapts its employed reliability schemes to the changing operating environments thus to maintain a target reliability. The proposed data cache is implemented with three levels of error protection schemes, a monitoring mechanism, and a control component that decides whether to upgrade, downgrade, or keep the current protection level based on the feedback from the monitor. Our experimental evaluation using a set of SPEC CPU2000 benchmarks shows that our self-adaptive data cache achieves similar reliability to a cache protected by the most reliable scheme, while simultaneously minimizing the performance and power overheads.

On the effectiveness of phase based regression models to trade power and performance using dynamic processor adaptation

Microarchitecture optimizations, in general, exploit the gross program behavior for performance improvement. Programs may be viewed as consisting of different ''phases'' which are characterized by variation in a number of processor performance metrics. Previous studies have shown that many of the performance metrics remain nearly constant within a ''phase''. Thus, the change in program ''phases'' may be identified by observing the change in the values of these metrics. This paper aims to exploit the time varying behavior of programs for processor adaptation. Since the resource usage is not uniform across all program ''phases'', the processor operates at varying levels of underutilization. During phases of low available Instruction Level Parallelism (ILP), resources may not be fully utilized while in other phases, more resources may be required to exploit all the available ILP. Thus, dynamically scaling the resources based on program behavior is an attractive mechanism for power-performance trade-off. In this paper we develop per-phase regression models to exploit the phase behavior of programs and adequately allocate resources for a target power-performance trade-off. Modeling processor performance-power using such a regression model is an efficient method to evaluate an architectural optimization quickly and accurately. We also show that the per-phase regression model is better suited than an ''unified'' regression model that does not use phase information. Further, we describe a methodology to allocate processor resources dynamically by using regression models which are developed at runtime. Our simulation results indicate that average energy savings of 20% can be achieved with respect to a maximally configured system with negligible impact on performance for most of the SPEC-CPU and MEDIA benchmarks.

Dispersing proprietary applications as benchmarks through code mutation

Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not representative of the real-life applications of interest. This paper proposes code mutation, a novel technique that mutates a proprietary application to complicate reverse engineering so that it can be distributed as a benchmark. The benchmark mutant then serves as a proxy for the proprietary application. The key idea in the proposed code mutation approach is to preserve the proprietary application's dynamic memory access and/or control flow behavior in the benchmark mutant while mutating the rest of the application code. To this end, we compute program slices for memory access operations and/or control flow operationstrimmed through constant value and branch profiles; and subsequently mutate the instructions not appearing in these slices through binary rewriting. Our experimental results using SPEC CPU2000 and MiBench benchmarks show that code mutation is a promising technique that mutates up to 90% of the static binary, up to 50% of the dynamically executed instructions, and up to 35% of the at run time exposed inter-operation data dependencies. The performance characteristics of the mutant are very similar to those of the proprietary application across a wide range of microarchitectures and hardware implementations.

Dispersing proprietary applications as benchmarks through code mutation

Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not representative of the real-life applications of interest. This paper proposes code mutation, a novel technique that mutates a proprietary application to complicate reverse engineering so that it can be distributed as a benchmark. The benchmark mutant then serves as a proxy for the proprietary application. The key idea in the proposed code mutation approach is to preserve the proprietary application's dynamic memory access and/or control flow behavior in the benchmark mutant while mutating the rest of the application code. To this end, we compute program slices for memory access operations and/or control flow operationstrimmed through constant value and branch profiles; and subsequently mutate the instructions not appearing in these slices through binary rewriting. Our experimental results using SPEC CPU2000 and MiBench benchmarks show that code mutation is a promising technique that mutates up to 90% of the static binary, up to 50% of the dynamically executed instructions, and up to 35% of the at run time exposed inter-operation data dependencies. The performance characteristics of the mutant are very similar to those of the proprietary application across a wide range of microarchitectures and hardware implementations.

Dispersing proprietary applications as benchmarks through code mutation

Industry vendors hesitate to disseminate proprietary applications to academia and third party vendors. By consequence, the benchmarking process is typically driven by standardized, open-source benchmarks which may be very different from and likely not representative of the real-life applications of interest. This paper proposes code mutation, a novel technique that mutates a proprietary application to complicate reverse engineering so that it can be distributed as a benchmark. The benchmark mutant then serves as a proxy for the proprietary application. The key idea in the proposed code mutation approach is to preserve the proprietary application's dynamic memory access and/or control flow behavior in the benchmark mutant while mutating the rest of the application code. To this end, we compute program slices for memory access operations and/or control flow operationstrimmed through constant value and branch profiles; and subsequently mutate the instructions not appearing in these slices through binary rewriting. Our experimental results using SPEC CPU2000 and MiBench benchmarks show that code mutation is a promising technique that mutates up to 90% of the static binary, up to 50% of the dynamically executed instructions, and up to 35% of the at run time exposed inter-operation data dependencies. The performance characteristics of the mutant are very similar to those of the proprietary application across a wide range of microarchitectures and hardware implementations.

Analysing and improving clustering based sampling for microprocessor simulation

We propose a set of statistical metrics for making a comprehensive, fair, and insightful evaluation of features, clustering algorithms, and distance measures in representative sampling techniques for microprocessor simulation. Our evaluation of different clustering algorithms using these metrics shows that CLARANS clustering algorithm produces better quality clusters in the feature space and more homogeneous phases for CPI compared to the popular k-means algorithm. We also propose a new micro-architecture independent data locality based feature, Reuse Distance Distribution (RDD), for finding phases in programs, and show that the RDD feature consistently results in more homogeneous phases than the Basic Block Vector (BBV) feature for many SPEC CPU2000 benchmark programs.

Memory Data Flow Modeling in Statistical Simulation for the Efficient Exploration of Microprocessor Design Spaces

Microprocessor design is both complex and time consuming: exploring a huge design space for identifying the optimal design under a number of constraints is infeasible using detailed architectural simulation of entire benchmark executions. Statistical simulation is a recently introduced approach for efficiently culling the microprocessor design space. The basic idea of statistical simulation is to collect a number of important program characteristics and to generate a synthetic trace from it. Simulating this synthetic trace is extremely fast as it contains only a million instructions. This paper improves the statistical simulation methodology by proposing accurate memory data flow models. We propose 1) cache miss correlation, or measuring cache statistics conditionally dependent on the global cache hit/miss history, for modeling cache miss patterns and memory-level parallelism, 2) cache line reuse distributions for modeling accesses to outstanding cache lines, and 3) through-memory read-after-write dependency distributions for modeling load forwarding and bypassing. Our experiments using the SPEC CPU2000 benchmarks show substantial improvements compared to current state-of-the-art statistical simulation methods. For example, for our baseline configuration, we reduce the average instructions per cycle (IPC) prediction error from 10.9 to 2.1 percent; the maximum error observed equals 5.8 percent. In addition, we show that performance trends are predicted very accurately, making statistical simulation enhanced with accurate data flow models a useful tool for efficient and accurate microprocessor design space explorations.