Voltage Guardbands Research Articles

MoRS: An Approximate Fault Modeling Framework for Reduced-Voltage SRAMs

On-chip memory (usually based on Static RAMs-SRAMs) are crucial components for various computing devices including heterogeneous devices, e.g, GPUs, FPGAs, ASICs to achieve high performance. Modern workloads such as Deep Neural Networks (DNNs) running on these heterogeneous fabrics are highly dependent on the on-chip memory architecture for efficient acceleration. Hence, improving the energy-efficiency of such memories directly leads to an efficient system. One of the common methods to save energy is undervolting i.e., supply voltage underscaling below the nominal level. Such systems can be safely undervolted without incurring faults down to a certain voltage limit. This safe range is also called voltage guardband. However, reducing voltage below the guardband level without decreasing frequency causes timing-based faults. In this paper, we propose MoRS, a framework that generates the first approximate undervolting fault model using real faults extracted from experimental undervolting studies on SRAMs to build the model. We inject the faults generated by MoRS into the on-chip memory of the DNN accelerator to evaluate the resilience of the system under the test. MoRS has the advantage of simplicity without any need for high-time overhead experiments while being accurate enough in comparison to a fully randomly-generated fault injection approach. We evaluate our experiment in popular DNN workloads by mapping weights to SRAMs and measure the accuracy difference between the output of the MoRS and the real data. Our results show that the maximum difference between real fault data and the output fault model of MoRS is 6.21%, whereas the maximum difference between real data and random fault injection model is 23.2%. In terms of average proximity to the real data, the output of MoRS outperforms the random fault injection approach by 3.21x.

Exceeding Conservative Limits: A Consolidated Analysis on Modern Hardware Margins

Modern large-scale computing systems (data centers, supercomputers, cloud and edge setups and high-end cyber-physical systems) employ heterogeneous architectures that consist of multicore CPUs, general-purpose many-core GPUs, and programmable FPGAs. The effective utilization of these architectures poses several challenges, among which a primary one is power consumption. Voltage reduction is one of the most efficient methods to reduce power consumption of a chip. With the galloping adoption of hardware accelerators (i.e., GPUs and FPGAs) in large datacenters and other large-scale computing infrastructures, a comprehensive evaluation of the safe voltage reduction levels for each different chip can be employed for efficient reduction of the total power. We present a survey of recent studies in voltage margins reduction at the system level for modern CPUs, GPUs and FPGAs. The pessimistic voltage guardbands inserted by the silicon vendors can be exploited in all devices for significant power savings. On average, voltage reduction can reach 12% in multicore CPUs, 20% in manycore GPUs and 39% in FPGAs.

Predictive Guardbanding: Program-Driven Timing Margin Reduction for GPUs

The energy efficiency of GPU architectures has emerged as an essential aspect of computer system design. In this article, we explore the energy benefits of reducing the GPU chip's voltage to the safe limit, i.e., V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</sub> point, using predictive software techniques. We perform such a study on several commercial off-the-shelf GPU cards. We find that there exists about 20% voltage guardband on those GPUs spanning two architectural generations, which, if “eliminated” entirely, can result in up to 25% energy savings on one of the studied GPU cards. Our measurement results unveil a program dependent V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</sub> behavior across the studied applications, and the exact improvement magnitude depends on the program's available guardband. We make fundamental observations about the program-dependent V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</sub> behavior. We experimentally determine that the voltage noise has a more substantial impact on V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</sub> compared to the process and temperature variation, and the activities during the kernel execution cause large voltage droops. From these findings, we show how to use kernels' microarchitectural performance counters to predict its V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</sub> value accurately. The average and maximum prediction errors are 0.5% and 3%, respectively. The accurate V <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min</sub> prediction opens up new possibilities of a crosslayer dynamic guardbanding scheme for GPUs, in which software predicts and manages the voltage guardband, while the functional correctness is ensured by a hardware safety net mechanism.

A Unified Clock and Switched-Capacitor-Based Power Delivery Architecture for Variation Tolerance in Low-Voltage SoC Domains

Correctly operating digital SoC domains at their target frequencies require the addition of supply voltage ( $V_{\mathrm{ dd}}$ ) guardbands to account for supply droop events and temperature variation. These guardbands degrade processor energy efficiency, especially in low-voltage sensor and IoT applications due to increased delay sensitivity to temperature and $V_{\mathrm{ dd}}$ variation. In this paper, we present an all-digital unified clock and power (UniCaP-SC) architecture that combines switched-capacitor (SC)-based voltage control and clock frequency regulation into a single loop to significantly reduce required $V_{\mathrm{ dd}}$ guardbands. A UniCaP-SC test chip consisting of a near-threshold voltage (NTV) ARM Cortex-M0 processor was fabricated in 65-nm CMOS. The fully integrated system enables all-digital construction, aggressive $V_{\mathrm{ dd}}$ margin reduction, and continuous $V_{\mathrm{ dd}}$ scalability using SC-based voltage converters while no additional decoupling capacitance (decap). Test-chip measurements demonstrate a 16% $V_{\mathrm{ dd}}$ reduction corresponding to a 94% $V_{\mathrm{ dd}}$ margin recovery or an equivalent $3.2\times $ increase in the operating clock frequency ( $f_{\mathrm{ clk}}$ ).

DNN Engine: A 28-nm Timing-Error Tolerant Sparse Deep Neural Network Processor for IoT Applications

This paper presents a 28-nm system-on-chip (SoC) for Internet of things (IoT) applications with a programmable accelerator design that implements a powerful fully connected deep neural network (DNN) classifier. To reach the required low energy consumption, we exploit the key properties of neural network algorithms: parallelism, data reuse, small/sparse data, and noise tolerance. We map the algorithm to a very large scale integration (VLSI) architecture based around an single-instruction, multiple-data data path with hardware support to exploit data sparsity by completely eliding unnecessary computation and data movement. This approach exploits sparsity, without compromising the parallel computation. We also exploit the inherent algorithmic noise-tolerance of neural networks, by introducing circuit-level timing violation detection to allow worst case voltage guard-bands to be minimized. The resulting intermittent timing violations may result in logic errors, which conventionally need to be corrected. However, in lieu of explicit error correction, we cope with this by accentuating the noise tolerance of neural networks. The measured test chip achieves high classification accuracy (98.36% for the MNIST test set), while tolerating aggregate timing violation rates > 10−1. The accelerator achieves a minimum energy of 0.36 $\mu \text{J}$ /inference at 667 MHz; maximum throughput at 1.2 GHz and 0.57 $\mu \text{J}$ /inference; or a 10% margined operating point at 1 GHz and 0.58 $\mu \text{J}$ /inference.

Aggressive Voltage and Temperature Control for Power Saving in Mobile Application Processors

DVFS is a widely used methodology for reducing the power consumption of mobile devices. This scheme involves frequency scaling in accordance with a specific governor and the establishment of an operating voltage to be paired with frequency. Incorporated into the settings for operating voltage is a guardband that ensures safe processor operation even at the worst conditions of on-chip temperature. Typically, the processor temperature remains at a normal range (i.e., not the worst), hence the voltage guardband set to guarantee safe operation is overly protected. In this paper, we propose a temperature-aware DVS (T-DVS) that aggressively reduces voltage guardband. We explore the opportunity to provide minimum operating voltages for frequencies at different temperatures and realize a dynamic voltage control scheme that reduces power consumption. The T-DVS manages temperature so that it remains in the “green zone” where maximum voltage gain is enabled for power-efficient operation. We validate the effectiveness of the T-DVS under various thermal conditions by using mobile application processors and different operating scenarios. Experimental results show that the T-DVS leads to power gain without degrading performance regardless of thermal conditions and chip characteristics. By examining the real-world applications of and off-the-shelf smartphone, we show that the voltage gains generated by the T-DVS results in battery lifetime increment.

Edge-TM

Scaling of semiconductor devices has enabled higher levels of integration and performance improvements at the price of making devices more susceptible to the effects of static and dynamic variability. Adding safety margins (guardbands) on the operating frequency or supply voltage prevents timing errors, but has a negative impact on performance and energy consumption. We propose Edge-TM , an adaptive hardware/software error management policy that ( i ) optimistically scales the voltage beyond the edge of safe operation for better energy savings and ( ii ) works in combination with a Hardware Transactional Memory (HTM)-based error recovery mechanism. The policy applies dynamic voltage scaling (DVS) (while keeping frequency fixed) based on the feedback provided by HTM, which makes it simple and generally applicable. Experiments on an embedded platform show our technique capable of 57% energy improvement compared to using voltage guardbands and an extra 21-24% improvement over existing state-of-the-art error tolerance solutions, at a nominal area and time overhead.

Postsilicon Voltage Guard-Band Reduction in a 22 nm Graphics Execution Core Using Adaptive Voltage Scaling and Dynamic Power Gating

In high volume manufacturing, conventional approach to deal with inverse-temperature dependence (ITD) and aging is to add a postsilicon flat voltage guard band to all dies based on testing a small random sample of dies. Although this scheme guarantees error-free operation, it significantly degrades energy efficiency, as it penalize all dies for the maximum delay degradation due to ITD and aging as seen by the worst case die, while also assuming maximum aging condition. In this paper, a graphics execution core implemented in 22 nm trigate process uses per-die tunable replica circuit (TRC) to monitor delay degradation due to ITD and actual aging conditions. TRC triggers adaptive voltage scaling to dynamically adjust $V_{\mathrm {CC}}$ as needed during run time to maintain correct operation at minimum additional voltage. Measured data show up to 33% (14%) energy savings at 0.4 V (0.8 V) compared with baseline scheme. The TRC is also utilized in a dynamic power gating (DPG) scheme to lower energy overhead due to fast droop guard band. DPG introduces a load line effect during normal operation, thus saving energy, while deactivating this load line upon droop detection by the TRC to maintain ISO performance as baseline. Silicon data show that DPG can improve energy efficiency by 14.5% (7%) at 0.8 V (0.6 V).

A Low-Power 1-GHz Razor FIR Accelerator With Time-Borrow Tracking Pipeline and Approximate Error Correction in 65-nm CMOS

The unrelenting demands of wireless/multimedia DSP workloads necessitate specialized hardware to achieve higher performance and power efficiency. Razor systems offer even greater power efficiency by minimizing static supply voltage (VDD) guardbands for process/voltage/temperature (PVT) variation, while also providing a degree of resilience to general delay faults (e.g. SEUs). To date, Razor has only been demonstrated on silicon in the context of microprocessor pipelines [1][2]. Reported Algorithmic Noise Tolerance (ANT) circuits [3][4] operate at very high error rates, but rely on imbalanced ripple-carry adders and hence clock frequency (Fclk) is limited (50-88MHz). ANT also requires additional datapaths for error detection/correction, which cannot be clock gated in the absence of errors, increasing baseline area and power. Combining Razor error detection with algorithm-level correction enables high-Fclk datapaths and low-overheads. A 0.19mm2 16-tap Razor FIR datapath is fabricated in 65nm LP CMOS, with input and output SRAMs, tunable pulse-clock generator, BIST logic and an AHB slave on-chip bus interface (Fig. 24.5.1), demonstrating: 1) two distinct fixed-latency Razor error-correction techniques for real-time DSP datapaths: time-borrow tracking (TBT) and interpolation-based approximate error correction (AEC); 2) a Razor latch (RZL) circuit with reduced pessimism; 3) a 1GHz datapath, an order of magnitude improvement over [3][4] due to elimination of ripple-carry adders; 4) energy efficiency improvement of up to 37%.

Dynamic reduction of voltage margins by leveraging on-chip ECC in Itanium II processors

Lowering supply voltage is one of the most effective approaches for improving the energy efficiency of microprocessors. Unfortunately, technology limitations, such as process variability and circuit aging, are forcing microprocessor designers to add larger voltage guardbands to their chips. This makes supply voltage increasingly difficult to scale with technology. This paper presents a new mechanism for dynamically reducing voltage margins while maintaining the chip operating frequency constant. Unlike previous approaches that rely on special hardware to detect and recover from timing violations caused by low-voltage execution, our solution is firmware-based and does not require additional hardware. Instead, it relies on error correction mechanisms already built into modern processors. The system dynamically reduces voltage margins and uses correctable error reports raised by the hardware to identify the lowest, safe operating voltage. The solution adapts to core-to-core variability by tailoring supply voltage to each core's safe operating level. In addition, it exploits variability in workload vulnerability to low voltage execution. The system was prototyped on an HP Integrity Server that uses Intel's Itanium 9560 processors. Evaluation using SPECjbb2005 and SPEC CPU2000 workloads shows core power savings ranging from 18% to 23%, with minimal performance impact.

Circuit-Level Timing Error Tolerance for Low-Power DSP Filters and Transforms

In this paper, we present a novel circuit-level timing error mitigation technique, which aims to increase energy-efficiency of digital signal processing datapaths without loss of robustness. Timing errors are detected using razor flip-flops on critical-paths, and the error-rate feedback is used to control a dynamic voltage scaling control loop. In place of conventional razor error correction by replay, we propose a new approach to bound the magnitude of intermittent timing errors at the circuit level. A timing guard-band is created by shaping the path delay distribution such that the critical paths correspond to a group of least-significant bit registers. These end-points are ensured to be critical by modifying the topology of the final stage carry-merge adder, and by using tool-based device sizing. Hence, timing violations lead to weakly correlated logical errors of small magnitude in a mean-squared-error sense. We examine this approach in an finite-impulse response (FIR) filter and a 2-D discrete cosine transform implementation, in 32-nm CMOS. Power saving compared to a conventional design at iso-frequency is 21%-23% at the typical corner, while retaining a voltage guard-band to protect against fast transient changes in switching activity and supply noise. The impact on minimum clock period is small (16%-20%), as it does not necessitate the use of ripple-carry adders and also requires only a bare minimum of additional design effort.

The Next Generation 64b SPARC Core in a T4 SoC Processor

The SPARC T4 processor introduces the next generation multi-threaded 64b core to deliver up to 5x integer and 7x floating-point single-thread performance improvement over its predecessor. The chip integrates eight cores, an 8-Bank 4 MB L3 Cache, a 768 GB/sec crossbar, a memory controller, PCI Gen2.0, 10 Gb Ethernet and cache coherency with 2.4 Tb/s bandwidth high-speed I/Os. The dual-issue, out-of-order execution core (S3) features a new 16-stage integer pipeline, extensive branch predictions, dynamic threading and an enhanced cryptographic processing unit. The 406 mm2 die contains 855 million transistors and 2.6 million flip-flops in TSMC's 40nm process utilizing 11 Cu metals and four transistor types. Enhanced physical design methodologies and extensive power management features enable 3.0 GHz operation in the same power envelope of its predecessor. Logically complex SRAMs deploy techniques to support out-of-order execution core while addressing area, timing and power challenges. The power supply calibration circuit improves yield by reducing 70% of conventional voltage guard-band for the speed and power constrained design.

VRSync

Power consumption is a primary concern for microprocessor designers. Lowering the supply voltage of processors is one of the most effective techniques for improving their energy efficiency. Unfortunately, low-voltage operation faces multiple challenges going forward. One such challenge is increased sensitivity to voltage fluctuations, which can trigger so-called "voltage emergencies" that can lead to errors. These fluctuations are caused by abrupt changes in power demand, triggered by processor activity variation as a function of workload. This paper examines the effects of voltage fluctuations on future many-core processors. With the increase in the number of cores in a chip, the effects of chip-wide activity fluctuation -- such as that caused by global synchronization in multithreaded applications -- overshadow the effects of core-level workload variability. Starting from this observation, we developed VRSync, a novel synchronization methodology that uses emergency-aware scheduling policies that reduce the slope of load fluctuations, eliminating emergencies. We show that VRSync is very effective at eliminating emergencies, allowing voltage guardbands to be significantly lowered, which reduces energy consumption by an average of 33%.

Pushing Adaptive Voltage Scaling Fully on Chip

In this paper, we investigate the possibility for low-power applications to integrate an efficient Adaptive Voltage Scaling (AVS) system on chip. Therefore the impact of process (both global and local), voltage and temperature variations is firstly studied on two typical low-power circuits i.e., a mobile processor and a wireless sensor node. In order to ensure safe operation under all conditions, it is required to increase the supply voltage by up to 22% leading to a nearly 50% increase in dynamic power consumption when compared to theoretical operating conditions. AVS allows for reducing the supply voltage guard band. To be able to include the AVS system on chip using standard CMOS, this paper proposes to use a switched-capacitor network for DC/DC conversion from the higher battery voltage. A critical path replica is used for both sensing the circuit maximum operating frequency and generating its clock signal. We show that the voltage ripple induced by the DC/DC converter does not significantly contribute to the supply voltage guard band, and that overall the proposed AVS system allows a reduction by up to 80% of this guard band while consuming less than 33% of the total circuit area.