SPEC CPU2006 Benchmarks Research Articles

Multiple Function Merging for Code Size Reduction

Resource-constrained environments, such as embedded devices, have limited amounts of memory and storage. Practical programming languages such as C++ and Rust tend to output multiple similar functions by monomorphizing polymorphic functions. An optimization technique called Function Merging, which merges similar functions into a single function, has been studied. However, in the state-of-the-art approach, the number of functions that can be merged at once is limited to two; thus, efficiently merging three or more functions, which are often generated from polymorphic functions, has been impossible. In this study, we propose Multiple Function Merging optimization, which targets merging three or more similar functions into a single function using a multiple sequence alignment algorithm. With multiple aligned information, Multiple Function Merging can increase merge opportunities and reduce extra branching overheads at the code generation stage. We evaluated it using the SPEC CPU benchmark suite and some large-scale C/C++ programs, and the results show that it reduces code size by as much as 7.61% compared with the state-of-the-art approach.

Designing a Deep Neural Network engine for LLC block reuse prediction to mitigate Soft Error in Multicore

Last level cache (LLC), a major contender of chip area, exhibits the highest sensitivity to soft error. Block reuse prediction is used to exhibit selective protection of LLC blocks. However, traditional reuse predictors fail to accurately understand the cache access pattern. This work explores machine learning for block reuse prediction that realizes the features that affect reuse likelihood and therefore, better predicts the reusable blocks. To off-load the reuse calculation from the processing cores, a dedicated reuse predictor engine is designed. This engine implements a deep neural network with unsupervised learning and calculates reuse probabilities offline without affecting the execution path. Priority blocks with high reuse likelihood are protected using replications in shared LLC. Low priority blocks are invalidated to reduce their vulnerable time. A replication aware replacement policy is also developed to protect the replicated/reserved blocks from eviction. The scheme is evaluated in Multi2Sim 5.0 simulation framework with SPEC-CPU benchmarks considering bulk-CMOS, FDSOI and FinFET technologies. The results indicate 41.29% and 43.14% reductions in miss rate, 92.29% and 89.56% reductions in vulnerability, 5.73% and 6.31% increase in write-back rate, 17.55% and 16.27% reductions in average memory access time, 2.17% and 1.54% increase in Instructions-per-cycle (IPC) and 1.31% and 0.77% reductions in normalized-execution-cycle over the baseline (with SECDED protection) for integer and floating point benchmarks respectively. Marginal increase in power consumption by 1.33% and dynamic energy by 0.43% are observed at 6.32% area overhead.

REDB: Real-time enhancement of Docker containers via memory bank partitioning in multicore systems

Docker is a lightweight virtualization technology adopted by numerous companies to develop, deploy, and manage applications. However, All Docker containers running on the same system share system resources, leading to performance interference among them. This interference adversely affects the predictability of program response times within the containers. Thus, shared resource isolation between containers has become an important issue. Efforts have been made to improve isolation across various dimensions, such as file systems, I/O, etc. However, consideration of the effect of memory bank interference remains open. We propose a novel approach, REDB, as an effective solution to achieve memory isolation among containers, thereby enhancing the real-time performance of Docker containers. The fundamental strategy is partitioning DRAM banks and allocating specific banks to particular containers. The results reveal that the slowdown ratios of SPEC CPU2006 benchmarks show an average improvement of 7.7% (up to 18.4%) for Docker with REDB. We utilize the Cyclictest tool to characterize the Docker container latency, and our evaluations indicate that the maximum latency, a critical indicator of real-time performance, decreases significantly from 564 us for Docker without bank partitioning to just 26 us for Docker with REDB, resulting in an impressive reduction of 95.3%.

Performance analysis of congestion-aware Q-routing algorithm for network on chip

<p>A network on chip’s performance is greatly impacted by network congestion due to the substantial increase in latency and energy utilized. Designing routing strategies that keep the network informed of the status of traffic is made easier by machine learning techniques. In this work, a reinforcement-based congestion-aware Q-routing (CAQR) technique has been presented. The proposed algorithm performed better in comparison to the conventional XY routing method tested against the SPEC CPU2006 benchmark suite in the gem5 NoC simulator tool. The suite used has 4 benchmarks, namely, namd, lbm, leslie3d and bzip2 which can be used for the cores in the network in any combination. The tests were run with 16 cores on a 44 network with the maximum instruction count supported by the system (here 5,000). The proposed Q-routing algorithm showed an average of 19% reduction for benchmark simulation as compared to the Dimension-ordered (X-Y) routing for readings of average packet latency which is a crucial factor in determining a network’s efficiency. The analysis also shows an average reduction of 24%, 10%, 23% and 47% in terms of average packet network latency, average flit latency, average flit network latency and average energy consumption across various benchmarks.</p>

Energy efficiency in multicore shared cache by fault tolerance using a genetic algorithm based block reuse predictor

Aggressive voltage scaling to reduce energy consumption in Multicore causes exponential cell failures in SRAM. Last-level-cache (LLC), the major contender of chip area, exhibits highest sensitivity to low voltage parametric failures and limits energy efficiency. To break the energy barrier, exclusive fault protection mechanism is solicited to ensure on-chip data recovery out of the low voltage failures. This work proposes Cache evolution for energy efficiency by protecting cache blocks from SRAM cell failures due to voltage scaling. A set of coherence and reuse aware in-cache selective replication policies are proposed to ensure on-chip data recovery. Reuse likelihood is predicted through a vector optimization technique using Genetic Algorithm (GA). Reuse aware selective invalidation is employed to balance cache load due to replications. A replication aware replacement policy is also developed that victimizes the lowest reusable cache block. The proposed scheme is evaluated in Multi2Sim 5.0 simulation framework with SPEC CPU benchmark programs. Experimental results claim 43.66% and 38.80% reductions in miss rate and 59.10% and 52.73% reductions in vulnerability for integer and floating point benchmarks respectively. Energy reduction of 39.21% is observed for integer and 25.73% is observed for floating point benchmarks. Up to 34.78% and 26.98% power reductions in integer and floating point benchmarks are also observed. The minimum supply voltage of 350 mV is achieved at the cost of 7.05% area overhead and 3% performance drop-off.

Architecture slack exploitation for phase classification and performance estimation in server-class processors

In this paper, we present a highly accurate performance estimation methodology that accounts for architecture slack in workloads. Our work leverages the advanced instrumentation available in POWER8 processor that monitors core pipeline activity in relation to off-core memory accesses to build metrics for architecture slack characterization for workloads. Using these metrics, we construct a workload classifier that classifies workloads as core-bound and memory-bound and propose a performance prediction model for change in processor frequency for each class of workload – cPerf and mPerf, respectively. We evaluated these models with SPECCPU and PARSEC benchmark suites on a POWER8 based OpenPOWER system. We observed that the predicted performance with our models has high accuracy (97%) for both CPU and memory intensive benchmarks. We validated that the classifier is suitable to accurately classify phase of workloads during execution intervals. We developed an algorithm that uses classifier for phase classification and prediction models for performance estimation at runtime. We leveraged this algorithm and evaluated the execution time impacts of CPU and memory classified benchmarks.

A Fast Cross-Layer Dynamic Power Estimation Method by Tracking Cycle-Accurate Activity Factors With Spark Streaming

The advent of autonomous power-limited systems poses a new challenge for early design space exploration. The existing architecture-level power evaluation tools lose accuracy due to ignoring features of circuit-level behaviors and influences of process, voltage, and temperature variations. Although power estimations based on SPICE or PrimeTime PX (PTPX) are accurate enough, they come at the cost of long simulation time and are available only in very late phases of design flow. In this article, a fast and accurate dynamic power evaluation method is proposed, which estimates activity factors at the circuit level. The impact of process variation at the gate level is considered through the proposed effective capacitance model. Activity factors are then estimated by the model and input vectors of the circuit. Input vectors are generated by architecture-level simulations in the form of streaming. For real-time and high-speed power evaluation, a data streaming framework is proposed for massive parallelism. The cross-layer estimation is verified based on the functional units of PULPino processor running SPEC CPU2006 benchmarks. Compared with the SPICE results using SMIC 28-nm PDK, our cycle-by-cycle dynamic power analysis shows an average error of 5.4%. Meanwhile, our approach realizes 65.2% faster than the traditional PTPX simulation and 48.8% faster compared with the state-of-art cross-level evaluation method.

Profile inference revisited

Profile-guided optimization (PGO) is an important component in modern compilers. By allowing the compiler to leverage the program’s dynamic behavior, it can often generate substantially faster binaries. Sampling-based profiling is the state-of-the-art technique for collecting execution profiles in data-center environments. However, the lowered profile accuracy caused by sampling fully optimized binary often hurts the benefits of PGO; thus, an important problem is to overcome the inaccuracy in a profile after it is collected. In this paper we tackle the problem, which is also known as profile inference and profile rectification . We investigate the classical approach for profile inference, based on computing minimum-cost maximum flows in a control-flow graph, and develop an extended model capturing the desired properties of real-world profiles. Next we provide a solid theoretical foundation of the corresponding optimization problem by studying its algorithmic aspects. We then describe a new efficient algorithm for the problem along with its implementation in an open-source compiler. An extensive evaluation of the algorithm and existing profile inference techniques on a variety of applications, including Facebook production workloads and SPEC CPU benchmarks, indicates that the new method outperforms its competitors by significantly improving the accuracy of profile data and the performance of generated binaries.

Evaluation of Hardware Data Compression in Interprocessor Links of Elbrus Processors

The tendency to increase core count in modern processor systems leads to a higher strain on memory subsystem. In particular, one of the most critical points in terms of throughput is interprocessor links, where bandwidth is significantly less than in processor data buses. Hardware data compression can be considered as one of the ways to increase throughput in interprocessor links, as it allows to decrease the amount of information transmitted over the links. This paper presents the evaluation of hardware data compression in interprocessor links of Elbrus processors. BΔI*-HL compression algorithm is chosen for the evaluation. The results are obtained of FPGA prototype of “Elbrus-16C” processor for the tasks of SPEC CPU2000 benchmark suite. They show that by using hardware data compression 38,0% of all data packets were compressed and that the amount of information transmitted overall has decreased by 13,4%. These results demonstrate that the use of hardware data compression in interprocessor links of Elbrus processors is justified and has potential to significantly increase memory subsystem performance.

Performance counter based online pipeline bugs detection using machine learning techniques

The growing complexity of new features in multicore processors imposes significant pressure towards functional verification. Although a large amount of time and effort are spent on it, functional design bugs escape into the products and cause catastrophic effects. Hence, online design bug detection is needed to detect the functional bugs in the field. In this work, we propose a novel approach by leveraging Performance Monitoring Counters (PMC) and machine learning to detect and locate pipeline bugs in a processor. We establish the correlation between PMC events and pipeline bugs in order to extract the features to build and train machine learning models. We design and implement a synthetic bug injection framework to obtain datasets for our simulation. To evaluate the proposal, Multi2Sim simulator is used to simulate the x86 architecture model. An x86 fault model is developed to synthetically inject bugs in x86 pipeline stages. PMC event values are collected by executing the SPEC CPU2006 and MiBench benchmarks for both bug and no-bug scenarios in the x86 simulator. This training data obtained through simulation is used to build a Bug Detection Model (BDM) that detects a pipeline bug and a Bug Location Model (BLM) that locates the pipeline unit where the bug occurred. Simulation results show that both BDM and BLM provide an accuracy of 97.3% and 91.6% using Decision tree and Random forest, respectively. When compared against other state of art approaches, our solution can locate the pipeline unit where the bug occurred with a high accuracy and without using additional hardware.

A Reusable Characterization of the Memory System Behavior of SPEC2017 and SPEC2006

The SPEC CPU Benchmarks are used extensively for evaluating and comparing improvements to computer systems. This ubiquity makes characterization critical for researchers to understand the bottlenecks the benchmarks do and do not expose and where new designs should and should not be expected to show impact. However, in characterization there is a tradeoff between accuracy and reusability: The more precisely we characterize a benchmark’s performance on a given system, the less usable it is across different micro-architectures and varying memory configurations. For SPEC, most existing characterizations include system-specific effects (e.g., via performance counters) and/or only look at aggregate behavior (e.g., averages over the full application execution). While such approaches simplify characterization, they make it difficult to separate the applications’ intrinsic behavior from the system-specific effects and/or lose the diverse phase-based behaviors. In this work we focus on characterizing the applications’ intrinsic memory behaviour by isolating them from micro-architectural configuration specifics. We do this by providing a simplified generic system model that evaluates the applications’ memory behavior across multiple cache sizes, with and without prefetching, and over time. The resulting characterization can be reused across a range of systems to understand application behavior and allow us to see how frequently different behaviors occur. We use this approach to compare the SPEC 2006 and 2017 suites, providing insight into their memory system behaviour beyond previous system-specific and/or aggregate results. We demonstrate the ability to use this characterization in different contexts by showing a portion of the SPEC 2017 benchmark suite that could benefit from giga-scale caches, despite aggregate results indicating otherwise.

SaVioR: Thwarting Stack-Based Memory Safety Violations by Randomizing Stack Layout

Stack-based memory corruption vulnerabilities have been exploited, allowing attackers to execute arbitrary code and read/write arbitrary memory. Although several solutions have been proposed to prevent memory errors on the stack, they are either limited to a specific type of attack (either spatial or temporal attacks) or cause significant performance degradation. In this article, we introduce SaVioR, an efficient and comprehensive stack protection mechanism. The key technique involves randomization of the stack layout to reduce its predictability and exploitability. SaVioR isolates an individual object from spatially and temporally adjacent vulnerable objects and randomizes each object's location, which prevents attackers from predicting the stack layout and thus reduces the likelihood of memory errors being exploited. We implemented SaVioR based on the LLVM compiler framework and applied it to the SPEC CPU2006 benchmarks and real-world applications. Our security evaluation showed that SaVioR provides a high degree of randomness in the stack layout and thus reduces the likelihood of successful exploitation of spatial and temporal memory errors on the stack. Our performance evaluation also demonstrated that it incurs a modest performance overhead (14 percent) with the SPEC CPU2006 benchmark suite, which improves performance compared to the state-of-the-art stack protection while achieving a comparable security level.

Polymorphic Memory: A Hybrid Approach for Utilizing On-Chip Memory in Manycore Systems

The key challenges of manycore systems are the large amount of memory and high bandwidth required to run many applications. Three-dimesnional integrated on-chip memory is a promising candidate for addressing these challenges. The advent of on-chip memory has provided new opportunities to rethink traditional memory hierarchies and their management. In this study, we propose a polymorphic memory as a hybrid approach when using on-chip memory. In contrast to previous studies, we use the on-chip memory as both a main memory (called M1 memory) and a Dynamic Random Access Memory (DRAM) cache (called M2 cache). The main memory consists of M1 memory and a conventional DRAM memory called M2 memory. To achieve high performance when running many applications on this memory architecture, we propose management techniques for the main memory with M1 and M2 memories and for polymorphic memory with dynamic memory allocations for many applications in a manycore system. The first technique is to move frequently accessed pages to M1 memory via hardware monitoring in a memory controller. The second is M1 memory partitioning to mitigate contention problems among many processes. Finally, we propose a method to use M2 cache between a conventional last-level cache and M2 memory, and we determine the best cache size for improving the performance with polymorphic memory. The proposed schemes are evaluated with the SPEC CPU2006 benchmark, and the experimental results show that the proposed approaches can improve the performance under various workloads of the benchmark. The performance evaluation confirms that the average performance improvement of polymorphic memory is 21.7%, with 0.026 standard deviation for the normalized results, compared to the previous method of using on-chip memory as a last-level cache.

Improved Basic Block Reordering

Basic block reordering is an important step for profile-guided binary optimization. The state-of-the-art goal for basic block reordering is to maximize the number of fall-through branches. However, we demonstrate that such orderings may impose suboptimal performance on instruction and I-TLB caches. We propose a new algorithm that relies on a model combining the effects of fall-through and caching behavior. As details of modern processor caching is quite complex and often unknown, we show how to use machine learning in selecting parameters that best trade off different caching effects to maximize binary performance. An extensive evaluation on a variety of applications, including Facebook production workloads, the open-source compilers Clang and GCC, and SPEC CPU benchmarks, indicate that the new method outperforms existing block reordering techniques, improving the resulting performance of applications with large code size. We have open sourced the code of the new algorithm as a part of a post-link binary optimization tool, BOLT.

Smart selection of optimizations in dynamic compilers

SummaryDynamic compilers perform compilation and generation of target code during runtime, implying that the compilation time is added into the program runtime. Thus, to build a high‐performing dynamic compilation system, it is crucial to be able to generate high‐quality code and, at the same time, have a small compilation cost. In this article, we present an approach that uses machine learning to select sequences of optimization for dynamic compilation that considers both code quality and compilation overhead. Our approach starts by training a model, offline, with a knowledge bank of those sequences with low overhead and high‐quality code generation capability using a genetic heuristic. Then, this bank is used to guide the smart selection of optimizations sequences for the compilation of code fragments during the emulation of an application. We evaluate the proposed strategy in two LLVM‐based dynamic binary translators, namely OI‐DBT and HQEMU, and show that these two translators can achieve average speedups of 1.26x and 1.15x in MiBench and Spec Cpu benchmarks, respectively.

Leakage-Aware Dynamic Thermal Management of 3D Memories

3D memory systems offer several advantages in terms of area, bandwidth, and energy efficiency. However, thermal issues arising out of higher power densities have limited their widespread use. While prior works have looked at reducing dynamic power through reduced memory accesses, in these memories, both leakage and dynamic power consumption are comparable. Furthermore, as the temperature rises, the leakage power increases, creating a thermal-leakage loop. We study the impact of leakage power on 3D memory temperature and propose turning OFF specific memory channels to meet thermal constraints. Data is migrated to a 2D memory before closing a 3D channel. We introduce an analytical model to assess the 2D memory delay and use the model to guide data migration decisions. The above strategy is referred to as FastCool and provides an improvement of 22%, 19%, and 32% on average (up to 57%, 72%, and 82%) in performance, memory energy, and energy-delay product (EDP), respectively, on different workloads consisting of SPEC CPU2006 benchmarks. We further propose a thermal management strategy named Energy-Efficient FastCool (EEFC) , which improves upon FastCool by selecting the channels to be closed by considering temperature, leakage, access rate, and position of various 3D memory channels at runtime. Our experiments demonstrate that EEFC leads to an additional improvement of up to 30%, 30%, and 51% in performance, memory energy, and EDP compared to FastCool. Finally, we analyze the effects of process variations on the efficiency of the proposed FC and EEFC strategies. Variation in the manufacturing process causes changes in the leakage power and temperature profile. Since EEFC considers both while selecting channels for closure, it is more resilient to process variations and achieves a lower application execution time and memory energy compared to FastCool.

Extending Performance-Energy Trade-offs Via Dynamic Core Scaling

Modern processors often need to switch among different power states based on usage scenarios, energy availability, and thermal conditions. Dynamic voltage and frequency scaling (DVFS) is a commonly used power management strategy to trade off performance and energy. As transistor scaling is reaching its limit, the viable supply voltage range where DVFS can operate is shrinking, which limits its effectiveness. To extend the performance-energy trade-off capabilities in modern processors, this article proposes dynamic core scaling (DCS) that does not rely on voltage scaling. DCS dynamically adjusts the active superscalar datapath resources so that programs run at a given percentage of their maximum speed while minimizing energy consumption at the same time. Since DCS does not need voltage scaling, it can be combined with DVFS to achieve greater energy savings. To effectively manage performance-energy trade-offs using a combination of DCS and DVFS, this article proposes an oracle controller that demonstrates the optimal control strategy, and two practical controllers that are applicable in real implementations. Evaluations using an 8-way superscalar processor implemented in a 45-nm circuit show that DCS is more effective in performance-energy trade-offs than DVFS at the high performance end for a number of SPEC CPU2000 benchmarks. When used together with DVFS, DCS saves an additional 20 percent of a full-size core’s energy on average. At the minimum operating voltage, DVFS hits its limit, while DCS is still able to achieve an average of 46 percent further energy reduction.

Page Reusability-Based Cache Partitioning for Multi-Core Systems

Most modern multi-core processors provide a shared last level cache (LLC) where data from all cores are placed to improve performance. However, this opens a new challenge for cache management, owing to cache pollution. With cache pollution, data with weak temporal locality can evict other data with strong temporal locality when both are mapped into the same cache set. In this article, we propose page reusability-based cache partitioning (PRCP) for multi-core systems to maximize cache utilization by minimizing cache pollution. To achieve this, PRCP divides pages into two groups: (1) highly-reused pages and (2) lowly-reused pages. The reusability of each page is collected online via periodic page table scans. PRCP then dynamically partitions the shared cache into two corresponding areas using page coloring technique. We have implemented PRCP in Linux kernel and evaluated it using SPEC CPU2006 benchmarks. The results show that our scheme can achieve comparable performance to the optimal offline MRC-guided process-based cache partitioning scheme without a priori knowledge of workloads.

IMPULP: A Hardware Approach for In-Process Memory Protection via User-Level Partitioning

In recent years many security attacks occur when malicious codes abuse in-process memory resources. Due to the increasing complexity, an application program may call third-party code which cannot be controlled by programmers but may contain security vulnerabilities. As a result, the users have the risk of suffering information leakage and control flow hijacking. However, current solutions like Intel memory protection extensions (MPX) severely degrade performance, while other approaches like Intel memory protection keys (MPK) lack flexibility in dividing security domains. In this paper, we propose IMPULP, an effective and efficient hardware approach for in-process memory protection. The rationale of IMPULP is user-level partitioning that user code segments are divided into different security domains according to their instruction addresses, and accessible memory spaces are specified dynamically for each domain via a set of boundary registers. Each instruction related to memory access will be checked according to its security domain and the corresponding boundaries, and illegal in-process memory access of untrusted code segments will be prevented. IMPULP can be leveraged to prevent a wide range of in-process memory abuse attacks, such as buffer overflows and memory leakages. For verification, an FPGA prototype based on RISC-V instruction set architecture has been developed. We present eight tests to verify the effectiveness of IMPULP, including five memory protection function tests, a test to defense typical buffer overflow, a test to defense famous memory leakage attack named Heartbleed, and a test for security benchmark. We execute the SPEC CPU2006 benchmark programs to evaluate the efficiency of IMPULP. The performance overhead of IMPULP is less than 0.2% runtime on average, which is negligible. Moreover, the resource overhead is less than 5.5% for hardware modification of IMPULP.

Implementation of Memory Subsystem of Cycle-Accurate Application-Level Simulator of the Elbrus Microprocessors

Performance characteristics of any modern microprocessor largely depend on its memory subsystem. Naturally, the memory subsystem software model is an important component of the cycle-accurate simulator, and its validity and quality have high impact on the overall accuracy of the simulation. In this paper the cycle-accurate application-level simulator of the Elbrus microprocessor family is introduced. The structure of the cycle-accurate simulator is briefly explained. After that the software model of memory subsystem and its integration as a part of the cycle-accurate application-level simulator are described. We evaluate accuracy of the application-level cycle-accurate simulator on the SPEC CPU2006 benchmark and analyze the simulation errors. Finally, a brief comparison of different Elbrus architecture simulators is given.