Energy-delay Product Improvement Research Articles

Ytopt: Autotuning Scientific Applications for Energy Efficiency at Large Scales

ABSTRACTAs we enter the exascale computing era, efficiently utilizing power and optimizing the performance of scientific applications under power and energy constraints has become critical and challenging. We propose a low‐overhead autotuning framework to autotune performance and energy for various hybrid MPI/OpenMP scientific applications at large scales and to explore the tradeoffs between application runtime and power/energy for energy efficient application execution, then use this framework to autotune four ECP proxy applications—XSBench, AMG, SWFFT, and SW4lite. Our approach uses Bayesian optimization with a Random Forest surrogate model to effectively search parameter spaces with up to 6 million different configurations on two large‐scale HPC production systems, Theta at Argonne National Laboratory and Summit at Oak Ridge National Laboratory. The experimental results show that our autotuning framework at large scales has low overhead and achieves good scalability. Using the proposed autotuning framework to identify the best configurations, we achieve up to 91.59% performance improvement, up to 21.2% energy savings, and up to 37.84% EDP (energy delay product) improvement on up to 4096 nodes.

A 90 nm area and power efficient Carry Select Adder using 2–1 multiplexer based Excess-1 block

This paper proposes a novel architecture of excess-1 adder-based Carry Select Adder (M2CSA) using single leaf cell i.e., 2–1 Multiplexer. M2CSA is designed using a new type of Excess-1 block. The Excess-1 block is designed in a distinct way using 2–1 multiplexers. The architectures of the proposed carry select adder and its internal excess-1 blocks are completely distinct when compared to existing carry select adders. The proposed 4-, 8-, 16-, 32-, and 64-bit M2CSAs use a 2–1 multiplexer only. The complex gates such as XOR gates are eliminated which exist in existing Carry Select Adders (CaSeAs). A 64-bit M2CSA that utilises one kind of 2-1 Mux cell is clearly decomposed to retain outstanding cell regularity. In previous designs, ripple carry adder blocks are substituted with half adders by limiting carry propagation to a particular degree. By avoiding carry propagation over adder blocks, particularly in 32- and 64-bit adders, the performance of M2CSA is initially increased. For CaSeAs, the area is of primary significance; for M2CSAs, it is also diminished by preserving cell regularity. Verilog HDL is utilised in the design of the M2CSA and current CaSeAs. Using Cadence NCLaunch, all of the designs are functionally tested. At a 90nm technology node, all of the designs are generated and executed using Genus and Innovus tools, respectively. As a comparison to prior designs, the 64-bit M2CSA final ASIC architecture is on average 17% less in terms of space. The comparison and result analysis show that 64-bit M2CSA performs 20% and 19% faster than 64-bit SQCSC in terms of performance and power dissipation, respectively.. The proposed 4-, 8-, 16-, 32-, 64- bit M2CSA exhibits less PDP (Power-Delay Product) ranging from 25%–30% concerning SQCSCs. Similarly, EDP (Energy- Delay Product) values for all the designs are calculated. The improvement in EDP for the 4-, 8-, 16-, 32-, 64- bit M2CSA is 41, 38, 42, 43, and 45 percentages compared to 4-, 8-, 16-, 32-, 64- bit SQCSC respectively. Therefore, the M2CSAs outperform the existing designs in terms of EDP by 38%–45%.

SAMA: Self-adjusting multi-cycle approximate adder

Approximate computing is a promising approach that leads to a better compromise between power and delay by trading an acceptable level of accuracy in inherently error resilient applications. In this paper, we present a runtime accuracy configurable approximate adder that can operate in both approximate and exact modes. It provides a better compromise between power, delay, and accuracy compared with state-of-the-art approximate adders at high accuracies. Our evaluations show that the proposed adder can make up to 45% and 79% improvements in power-delay product (PDP) and energy-delay product (EDP), respectively, compared with state-of-the-art approximate adders, at high accuracies. In the accurate mode, the improvements in the PDP and EDP are up to 58% and 82%, respectively. The use of the proposed adder in an image processing application and in a mean-squares prediction algorithm confirms its superiority over existing circuits in terms of PDP and EDP.

F-LEMMA: Fast Learning-Based Energy Management for Multi-/Many-Core Processors

Over the last two decades, as microprocessors have evolved to achieve higher computational performance, their power density has also increased at an accelerated rate. Improving energy efficiency and reducing power consumption are therefore critically important to modern computing systems. One effective technique for improving energy efficiency is dynamic voltage and frequency scaling (DVFS). With the emergence of integrated voltage regulators (IVRs), the speed of DVFS can reach microsecond ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu \text{s}$ </tex-math></inline-formula> ) timescales. However, a practical and effective strategy to guide fast DVFS remains a challenge. In this article, we propose F-LEMMA: a fast, learning-based, hierarchical DVFS framework consisting of a global power allocator in the kernel space, a reinforcement learning-based power management scheme at the architecture level, and a swift controller at the digital circuit level. This hierarchical approach leverages computation at the system and architecture levels with the short response time of the swift controller to achieve effective and rapid <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu \text{s}$ </tex-math></inline-formula> -level power management supported by the IVR. Our experimental results demonstrate that F-LEMMA can achieve significant energy savings (35.2%) across a broad range of workloads. Conservatively compared with existing state-of-the-art DVFS-based power management schemes that can only operate at millisecond timescales, F-LEMMA can provide notable (up to 11%) energy-delay product (EDP) improvements across benchmarks. Compared with state-of-the-art nonlearning-based power management, our method has a universally positive effect on evaluated benchmarks, proving its adaptability.

Margin and Yield Optimization of Single Flux Quantum Logic Cells Using Swarm Optimization Techniques

Single flux quantum (SFQ) logic family is an attractive alternative to CMOS technology with the promise of more than two–three orders of magnitude improvement in energy-delay product. However, component-level parameter variations that arise during the fabrication process of SFQ logic circuits tend to be very high, which means that the fabricated circuits may run into major functional and/or performance issues. Therefore, optimizing SFQ logic cells to maximize their operating parameter margins under different variability sources is required. In this article, a swarm optimization technique combining the best of automatic niching particle swarm optimization and fireworks algorithm is presented where the objective is to maximize the summation of the upper and lower bound margins over all design parameters of an SFQ logic cell. The proposed method eliminates all functional errors and improves the critical margin range and parametric yield values for six different logic cells by 22.83 and 15.22% on average, when compared to a previously optimized open-source cell library.

Energy Efficient Dim and Dark Cache for Temperature Reduction of Chip Multiprocessors

Background: For high computation, parallel and distributed processing chip multiprocessors are widely used. In chip multiprocessors, large on chip cache memories are used for performance improvement and speedy execution of the task. These cache memories consume 13% to 45% of the overall power consumed by the entire chip. As chip size shrinks with technology scaling, static energy consumed by the chip increases. This results in heating and chip malfunctioning. To solve this problem we apply static energy reduction techniques to Last Level Cache (LLC) to reduce static energy consumption and thereby chip temperature rise. Methods: Static energy reduction is achieved with two methods. DIM_OLY_SEL Policy:- In this policy, static Energy Reduction is achieved by dimming cache banks. DIM_DARK_PAT Policy:- In this method, dimming as well as darkening techniques are used to save static energy and thereby, temperature reduction. Results: With the implementation of DIM_OLY_SEL policy LLC, Temperature Reduction up to 1.4 Kelvin, improvements in EDP (Energy Delay Product) gains up to 28% and leakage energy Saving up to 23% is observed. With the implementation of DIM_DARK_PAT, we observed LLC temperature reduction up to 4.7 Kelvin, EDP gain improvement of 33 % and leakage energy saving 23%. Conclusion: DIM_OLY_SEL and DIM_DARK_PAT Policies are observed to be very effective policies for energy saving and chip temperature reduction. Hence, they can be used effectively for safe chip multiprocessor functioning.

Cost-effective fixed-point hardware support for RISC-V embedded systems

With the ever-increasing energy-efficiency requirements for the computing platforms at the edge, precision tuning techniques highlight the possibility of improving the efficiency of floating-point computations by selectively lowering the precision of intermediate operations without affecting the accuracy of the final result. Recent trends also demonstrated the possibility of successfully employing fixed-point computations in place of floating-point ones to further optimize the efficiency in the high-performance computing (HPC) domain. However, the use of integer functional units to support the execution of fixed-point operations in embedded platforms can severely degrade the energy-delay product (EDP). This work presents a cost-effective architecture to efficiently support fixed-point computations in embedded systems with two goals. On the one hand, it allows replacing the floating-point computations, with meaningful area and EDP improvements. On the other hand, it can complement the FPU, providing flexibility in selecting the best arithmetic for the target applications depending on their accuracy and performance requirements. Experimental results were collected from a representative set of floating-point-intensive applications executed on six variants of the baseline system-on-chip (SoC). We compared the efficiency of different floating- and fixed-point architectures in terms of accuracy, area, and EDP. Our fixed-point solution demonstrated a 0.651 EDP, normalized with respect to a SoC that featured a binary32 FPU, while achieving a negligible accuracy loss, i.e., 0.0003% on average (0.004% peak), compared to binary32 execution. In contrast, a SoC employing the integer functional units for fixed-point execution reports a 1.796 normalized EDP to achieve the same accuracy loss. Compared to the baseline SoC implementing only the integer units, the proposed architecture shows an area overhead limited to 4%, while the SoC featuring binary32 floating-point hardware support requires 32% more resources.

Multi-Input Logic-in-Memory for Ultra-Low Power Non-Von Neumann Computing.

Logic-in-memory (LIM) circuits based on the material implication logic (IMPLY) and resistive random access memory (RRAM) technologies are a candidate solution for the development of ultra-low power non-von Neumann computing architectures. Such architectures could enable the energy-efficient implementation of hardware accelerators for novel edge computing paradigms such as binarized neural networks (BNNs) which rely on the execution of logic operations. In this work, we present the multi-input IMPLY operation implemented on a recently developed smart IMPLY architecture, SIMPLY, which improves the circuit reliability, reduces energy consumption, and breaks the strict design trade-offs of conventional architectures. We show that the generalization of the typical logic schemes used in LIM circuits to multi-input operations strongly reduces the execution time of complex functions needed for BNNs inference tasks (e.g., the 1-bit Full Addition, XNOR, Popcount). The performance of four different RRAM technologies is compared using circuit simulations leveraging a physics-based RRAM compact model. The proposed solution approaches the performance of its CMOS equivalent while bypassing the von Neumann bottleneck, which gives a huge improvement in bit error rate (by a factor of at least 108) and energy-delay product (projected up to a factor of 1010).

PEPS: predictive energy-efficient parallel scheduler for multi-core processors

In multi-core processors, energy efficiency and performance consideration are essential issues. Usually, energy-saving techniques result in performance loss and vice versa. Therefore, energy delay product (EDP) is used broadly in many applications as a trade-off between energy saving and performance improvement. This paper presents a technique to perform work-stealing scheduling in the operating system kernel without needing any modification to the user-space program. The proposed scheduling uses predictive models to determine the optimal active number of cores and clock frequency of the processor as an optimum configuration at runtime for any running program to achieve the minimum EDP value. Since EDP is considered as a long-term metric, at runtime, in each specific time frame, PEPS uses the instruction per watt (IPW) to determine the best configuration. By using performance and power predicting models, PEPS finds the optimal configuration in terms of energy efficiency for the next time interval. Because different workloads at runtime have different behaviors and programs with different degrees of parallelization acted variously, the proposed method uses performance counters as a factor for workload characterization. Compared to the Linux scheduler, the proposed algorithm has up to 25% improvement in energy saving at the cost of 7% performance loss. Moreover, while reducing the temperature by 24%, it results in 19% improvement in EDP.

Ultra-Compact Ternary Logic Gates Based on Negative Capacitance Carbon Nanotube FETs

Ternary logic has been studied for several decades as it can offer significant advantages to reduce the number of interconnects and the complexity of operations. However, the excessive transistor count of the existing ternary logic gates can diminish these advantages in practice. In this brief, based on the negative capacitance (NC) feature of the ferroelectric materials and the well-proven electronic properties of the carbon nanotube field-effect transistor (CNTFET), we have proposed ultra-compact ternary logic gates. After developing a NC-CNTFET model, we have designed a 2-transistor ternary inverter, a 4-transistor ternary NAND, and a 4-transistor ternary NOR with the structures and transistor counts similar to the binary complementary metal-oxide-semiconductor (CMOS) logic. The simulation results ascertain the correct and robust functionality of the proposed ternary gates, even in the presence of process variations. Our proposed ternary inverter, NAND, and NOR gates lead to on average 65%, 60%, and 60% improvements in transistor count, 79%, 83%, and 77% improvements in the area, and 34%, 61%, and 54% improvements in energy-delay product (EDP) as compared to the previous state-of-the-art ternary gates. Our approach accentuates that the proposed ternary gates are the potential candidates for demonstrating more complex multi-valued arithmetic-logic units.

Energy-Efficient Monolithic 3-D SRAM Cell With BEOL MoS2 FETs for SoC Scaling

In this article, we propose an energy-efficient monolithic 3-D (M3D) three-tier SRAM cell with back-end-of-the-line (BEOL) back-gated (BG) MoS2 FETs. The impacts of wire routing resistance and capacitance, gate topology of MoS2 FETs, and the layout optimization of multitier 6T SRAM cells have been comprehensively analyzed for SoC scaling through system-technology co-optimization. SRAM plays an integral role in the performance of SoCs, and the performance can be improved by SRAM on logic integration. Compared with one-tier BG SRAM cell design, the proposed monolithic three-tier BG SRAM cell releases the impact of metal line resistance and shows a 44.3% reduction in cell area, 28.4% improvement in read access time, 21.3% improvement in dynamic energy, and 43.6% improvement in energy-delay product. The energy- and area-efficient three-tier BG SRAM cell enables intelligent functionalities for the area- and energy-constrained edge computing devices.

PARMA: Parallelization-Aware Run-Time Management for Energy-Efficient Many-Core Systems

Performance and energy efficiency considerations have shifted computing paradigms from single-core to many-core architectures. At the same time, traditional speedup models such as Amdahl's Law face challenges in the run-time reasoning for system performance and energy efficiency, because these models typically assume limited variations of the parallel fraction. Moreover, the parallel fraction, which varies dynamically in workloads, is generally unknown at run-time without application-level instrumentation. This article describes novel performance/energy trade-off models based on realistic architectural considerations, which describe the parallel fraction and speedup as functions of performance counter values available in modern processors, removing the need for application-level instrumentation. These are then used to develop a Parallelization-Aware Run-time Management (PARMA) approach. PARMA aims at controlling core allocations and operating voltage/frequency points for energy efficiency, according to the varying workload parallel fractions. The efficacy of our models and the PARMA approach is extensively validated using a number of PARSEC benchmark applications, involving two performance/energy trade-off metrics: energy-delay-product (EDP), typically used in high-performance applications and energy per instruction (EPI), suitable for energy-aware applications. Up to 48 and 68 percent improvements in EDP and EPI have been observed using the PARMA approach compared with parallelization-agnostic methods.

CASH-RAM: Enabling In-Memory Computations for Edge Inference Using Charge Accumulation and Sharing in Standard 8T-SRAM Arrays

Machine Learning (ML) workloads being memory- and compute-intensive, consume large amounts of power running on conventional computing systems, restricting their implementations to large-scale data centers. Transferring large amounts of data from the edge devices to the data centers is not only energy expensive, but sometimes undesirable in security-critical applications. Thus, there is a need for building domain-specific hardware primitives for energy-efficient ML processing at the edge. One such approach - in-memory computing , eliminates frequent and unnecessary data-transfers between the memory and the compute units, by directly computing the data where it is stored. However, the analog nature of computations introduces non-idealities, which degrades the overall accuracy of neural networks. In this paper, we propose an in-memory computing primitive for accelerating dot-products within standard 8T-SRAM caches, using charge-sharing. The inherent parasitic capacitance of the bitlines and sourcelines is used for accumulating analog voltages, which can be sensed for an approximate dot product. The charge sharing approach involves a self-compensation technique which reduces the effects of non-idealities, thereby reducing the errors. Our results for ternary weight neural networks show that using the proposed compensation approaches, the accuracy degradation is within 1% and 5% of the baseline accuracy, for the MNIST and CIFAR-10 dataset, respectively, with an energy-delay product improvement of $38\times $ over the standard von-Neumann computing system. We believe that this work can be used in conjunction with existing mitigation techniques, such as re-training approaches, to further enhance system performance.

Energy Efficient Low Latency Multi-issue Cores for Intelligent Always-On IoT Applications

Advanced Internet-of-Things applications require control-oriented codes to be executed with low latency for fast responsivity while their advanced signal processing and decision making tasks require computational capabilities. For this context, we propose three multi-issue core designs featuring an exposed datapath architecture with high performance, while retaining energy-efficiency. These features are achieved with exploitation of instruction-level parallelism, fast branching and the use of an instruction register file. With benchmarks in control-flow and signal processing application domains we measured in the best case 64% reduced energy consumption compared to a state-of-the-art RISC core, while consuming less silicon area. A high-performance design point reaches nearly 2.6 GHz operating frequency in the best case, over 2× improvement, while simultaneously achieving a 14% improvement in system energy-delay product.

Design Space Exploration for Chiplet-Assembly-Based Processors

Recent advancements in 2.5-D integration technologies have made chiplet assembly a viable system design approach. Chiplet assembly is emerging as a new paradigm for heterogeneous design at lower cost, design effort, and turnaround time and enables low-cost customization of hardware. However, the success of this approach depends on identifying a minimum chiplet set which delivers these benefits. We develop the first microarchitectural design space exploration framework for chiplet assembly-based processors which enables us to identify the minimum set of chiplets to design and manufacture. Since chiplet assembly makes heterogeneous technology and cost-effective application-dependent customization possible, we show the benefits of using multiple systems built from multiple chiplets to service diverse workloads (up to 35% improvement in energy-delay product over a single best system) and advantages of chiplet assembly approaches over system-on-chip (SoC) methodology in terms of total cost (up to 72% improvement in cost) while satisfying the energy and performance constraints of individual applications.

Energy Efficient On-Demand Dynamic Branch Prediction Models

The branch predictor unit (BPU) is among the main energy consuming components in out-of-order (OoO) processors. For integer applications, we find 16 percent of the processor energy is consumed by the BPU. BPU is accessed in parallel with the instruction cache before it is known if a fetch group contains control instructions. We find 85 percent of BPU lookups are done for non-branch operations, and of the remaining lookups, 42 percent are done for highly biased branches that can be predicted statically with high accuracy. We evaluate two variants of a branch prediction model that combines dynamic and static branch prediction to achieve energy improvements for power-constrained applications. These models, named on-demand branch prediction (ODBP) and path-based on-demand branch prediction (ODBP-PATH), are two novel prediction techniques that eliminate unnecessary BPU lookups using compiler generated hints to identify instructions that can be more accurately predicted statically. ODBP-PATH is an implementation of ODBP that combines static and dynamic branch prediction based on the program path of execution. For a 4-wide OoO processor, ODBP-PATH delivers 11 percent average energy-delay (ED) product improvement, and 9 percent core average energy saving on the SPEC Int 2006 benchmarks.

Runtime Efficiency-Accuracy Tradeoff Using Configurable Floating Point Multiplier

Many applications, such as machine learning and sensor data analysis, are statistical in nature and can tolerate some level of inaccuracy in their computation. Approximate computing is a viable method to save energy and increase performance by controllably trading off energy for accuracy. In this paper, we propose a tiered approximate floating point multiplier, called CFPU, which significantly reduces energy consumption and improves the performance of multiplication at a slight cost in accuracy. The floating point multiplication is approximated by replacing the costly mantissa multiplication step of the operation with lower energy alternatives. We process the data by using one of the three modes: a basic approximate mode, an intermediate approximate mode, or on the exact hardware, depending on the accuracy requirements. We evaluate the efficiency of the proposed CFPU on a wide range of applications including twelve general OpenCL ones and three machine learning applications. Our results show that using the first CFPU approximation mode results in $3.5\times $ energy-delay product (EDP) improvement, compared to a GPU using traditional floating point units (FPUs), while ensuring less than 10% average relative error. Adding the second mode further increases the EDP improvement to $4.1\times $ , compared to an unmodified FPU, for less than 10% error. In addition, our results show that the proposed CFPU can achieve $2.8\times $ EDP improvement for multiply operations as compared to state-of-the-art approximate multipliers.

Design of MWCNT based Through Silicon Vias with Polymer Liners to Reduce the Crosstalk Effects

This paper presents a unique method to reduce the crosstalk noise, power consumption, power delay product (PDP) and energy delay product (EDP) in coupled multi-walled carbon nanotube (MWCNT) based through silicon vias (TSVs) with polymer liners. A novel structure of TSVs is proposed, where the MWCNT bundles are placed as conducting material and polymer liners are placed as dielectric material. The electrical equivalent model of the proposed TSVs is used and simulated in HSPICE to evaluate the performance. The performance analysis determines that there is a significant improvement in crosstalk noise, power consumption, PDP and EDP for polymer liners such as polyimide, polypropylene carbonate (PPC) and benzocyclobutene (BCB) over the conventional silicon dioxide (SiO2) liner. Moreover, the performance is also analyzed by varying the pitch between the TSVs from 100 μm to 3000 μm. It is noted that the induced crosstalk noise issues are reduced as the TSV pitch is increased. Similarly, the power consumption, PDP and EDP of the proposed TSVs are also reduced for the high pitch value. From the simulation results, it is observed that the proposed MWCNT TSVs with BCB liner show improvements up to 18.01%, 35.37% and 49.06%, respectively in terms of power consumption, PDP and EDP over the TSVs with SiO2 liner.

A Dynamic Programming Framework for DVFS-Based Energy-Efficiency in Multicore Systems

Per-core Dynamic Voltage and Frequency (V/F) Scaling (DVFS) is a well-known methodology for achieving energy efficiency in multicore systems. Heuristic DVFS techniques provide fast, suboptimal V/F predictions while Dynamic Programming (DP) methods solve smaller sub-problems iteratively and use their outcomes to evaluate V/F levels globally, but at the cost of overhead delays. We propose an efficient DP framework using the Viterbi algorithm, which uses the Energy-Delay Product (EDP) as an objective function to predict the best V/F levels using applications' profiled information, to minimize energy consumption and execution time. Experimental results show that our framework outperforms heuristics using the EDP criteria and provides near-optimal solutions when maximizing energy saving is as, or more, important than minimizing execution time penalty. In fact, across several benchmarks, our proposed algorithm provides from a 12 to 75 percent improvement in EDP compared to heuristic methods. Furthermore, using a Pareto frontier to evaluate solutions of the algorithms under study, we demonstrate that our framework's energy-time solution is on average only 9 percent worse than the optimal solution. In addition, we show that our dynamic programming solution is 3 to 18 percent closer to a theoretical lower-bound when compared to the studied heuristic methods.

Energy-Efficient Convolutional Neural Network Based on Cellular Neural Network Using Beyond-CMOS Technologies

In this article, we perform a uniform benchmarking for the convolutional neural network (CoNN) based on the cellular neural network (CeNN) using a variety of beyond-CMOS technologies. Representative charge-based and spintronic device technologies are implemented to enable energy-efficient CeNN related computations. To alleviate the delay and energy overheads of the fully connected layer, a hybrid spintronic CeNN-based CoNN system is proposed. It is shown that low-power FETs and spintronic devices are promising candidates to implement energy-efficient CoNNs based on CeNNs. Specifically, more than $10\times $ improvement in energy-delay product (EDP) is demonstrated for the systems using spin diffusion-based devices and tunneling FETs compared to their conventional CMOS counterparts.