Energy Overhead Research Articles

Design of Ultracompact Content Addressable Memory Exploiting 1T-1MTJ Cell

Content addressable memories (CAMs) are a promising category of computing-in-memory (CiM) elements that can perform highly parallel and efficient search operations for routers, pattern matching, and other data-intensive applications. Various magnetic tunnel junction (MTJ) based content addressable memory (CAM) designs have been proposed to realize zero standby power and high performance search. However, due to the relatively small tunnel magneto-resistance (TMR) ratio, MTJ based CAMs require extra transistors and differential MTJ branches to distinguish between the parallel and anti-parallel resistance states, resulting in significant area and energy overhead. In this paper, we propose a device-circuit co-design approach for an ultra-compact CAM design by only exploiting an 1T-1MTJ structure in each cell. We propose a 2-step search scheme to enable the parallel in-memory search operation across the proposed CAM array, and demonstrate the sufficient sensing margin of the array in a successful search operation. Evaluation results suggest that our proposed 1T-1MTJ based CAM design improves 179X/301X area efficiency compared with the state-of-the-art 15T-4MTJ/20T-6MTJ CAM design. Application benchmarking on hyperdimensional computing (HDC) inference shows a 54.6X/12.8X speedup compared with GPU/20T-6MTJ CAM based approaches.

Layout Aware Optimization Methodology for SOT-MRAM Based on Technically Feasible Top-Pinned Magnetic Tunnel Junction Process

The emerging spin-orbit torque magnetic random-access memory (SOT-MRAM) shows promising prospects in high-level cache applications due to its sub-nanosecond switching speed and high reliability. However, SOT-MRAM faces the issue of large bit-cell layout area, which is currently the focus of attention. Although many design and evaluation works have emerged, the lack of a unified standard for realistic SOT process has hindered the development of relevant research towards practicality. In this paper, the bit-cell area of the SOT-MRAM will be evaluated and optimized based on the technically-feasible process. First of all, based on the state-of-the-art top-pinned SOT nanopillar process, the SOT-MRAM design rules are proposed. On this basis, this paper systematically summarizes four basic device layout modes and provides optimized layout suggestions for conventional SOT bit-cells with different types and sizes of devices. In addition, a series of area-efficient SOT bit-cell designs based on the common area (CA) and dual common (DC) solutions are proposed, which can reduce the layout area of SOT bit-cells by up to 38.4% with reasonable write latency and energy overhead.

Local bit line 8T SRAM based in-memory computing architecture for energy-efficient linear error correction codec implementation

Memory reliability is a critical issue in SRAM-based In-Memory Computing (IMC) architecture. The rapid advance in transistor technology makes SRAM more sensitive to soft errors. Various developments have recommended different methods to prevent store data corruption using Hamming Code. In order to encode or decode the information, a piece of Error Correction Code (ECC) hardware is necessary, which results in an area and energy overhead. In this paper, an IMC-based linear Error Correction Codec (IMC-ECC) hardware for parity check bit generation (encoding) and syndrome vector generation (decoding) is implemented by using the proposed Transmission Pass Gate (TPG) Local Bit-line (LB-8T) SRAM structure based IMC architecture. The local bit line structure resolves the half-select issues that occur for bit-interleave architecture. The TPG improves the write stability by delivering both strong VDD and GND. Furthermore, the proposed IMC-ECC scheme, along with bit-interleave architecture, makes the IMC architecture more resilient to multi-bit soft error. The proposed IMC-ECC scheme is validated by implementing a 4 Kb LB-8T SRAM array in 65 nm CMOS technology and Hamming code (7, 4) as an example. The proposed IMC-ECC scheme achieves the average energy consumption of 0.294 pJ/bit and 0.075 pJ/bit for the encoding and decoding process, respectively, at a supply voltage of 1 V.

Gradient distribution-aware INT8 training for neural networks

Recently, low bit-width quantization (e.g., INT8) has been commonly used in deep neural network inference acceleration, but fewer researchers have focused on low-precision training quantization techniques. Considering that the backward propagation in deep neural network training is more computationally intensive and has a heavier energy overhead than the inference process, the quantization of backward propagation is of great interest for the training of very large-scale neural networks as well as for low-power devices with online training requirements. However, the shape specificity and continuous variability of the gradient distribution make gradient quantization difficult, and many studies propose various complex quantization methods for the gradient to reduce training accuracy loss. In this paper, we propose two innovative techniques mainly for INT8 quantization training, including the Data-aware Dynamic Segmentation Quantization scheme to quantize various special gradient distributions and the Update Direction Periodic Search strategy to achieve lower quantization errors. Then, we build a distribution-aware INT8 quantization training framework based on these two methods and conduct experiments on various models and tasks. Experimental results show that our proposed INT8 quantization training method achieves a negligible loss in final training accuracy compared to the full-precision floating-point counterpart on different models, including ResNet, MobileNetV2, VGG, AlexNet, and LSTM. By replacing floating-point computing with 8-bit integer computing for network training, this INT8 quantization training framework provides the possibility of deploying online training directly on low-power devices in the future.

An efficient read approach for memristive crossbar array

Resistive random access memories (ReRAM) have drawn attention of researchers due to their unique properties with applications in in-memory computing, which allows storage and computation in the same unit. This mitigates one of the major limitations in current computing architectures, where for each computation we require to move data from memory to processor or vice versa, which incurs immense amount of energy overheads. Among the various technologies for implementing ReRAM, memristor is considered to be one of the most desirable candidates due to its small size, low power consumption, and high data retention. Such ReRAM systems are often fabricated in the form of crossbar for compact layout. However, they suffer from various challenges, one of the major ones being the sneak-path problem during reading of cell values. The read operation is mostly disturbed by sneak-path currents that can result in incorrect reading of the cell. This paper presents a new approach for reading the cell values in memristive crossbars, which is capable of avoiding erroneous read operations caused by sneak-paths. It also supports parallel operations whereby multiple memristor states can be read in a single cycle. A straightforward approach for reading all the cells in an n×n crossbar, where the read operation is performed sequentially, requires O(n2) cycles, whereas the proposed approach requires O(n) cycles.

Routing optimization algorithm based on mobile agent for wireless sensor networks

A routing model about mobile agent in sensor networks was drawn out, and an optimization problem of mobile agent static route was derived. In order to optimize the route of mobile agent, a target location algorithm in wireless sensor networks was proposed. All the sensor nodes which know target compete to get the mobile agent in order to track the location of targets. Competition rules are defined by the remaining value of the energy and the distance between the nodes and the target. Simulation results showed that this algorithm could provide less energy consumption compared to other schemes and prolong survival cycle time. It balances the network energy, reduces the amount of data transmission in the network, reduces the network energy overhead, prolongs the lifetime of the entire network, and effectively reduces the network delay.

A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm

The energy efficiency of deep neural network (DNN) inference can be improved with custom accelerators. DNN inference accelerators often employ specialized hardware techniques to improve energy efficiency, but many of these techniques result in catastrophic accuracy loss on transformer-based DNNs, which have become ubiquitous for natural language processing (NLP) tasks. This article presents a DNN accelerator designed for efficient execution of transformers. The proposed accelerator implements per-vector scaled quantization (VSQ), which employs an independent scale factor for each 64-element vector to enable the use of 4-bit arithmetic with little accuracy loss and low energy overhead. Using a multilevel dataflow to maximize reuse, the 5-nm prototype achieves 95.6 tera-operations per second per Watt (TOPS/W) at 0.46 V on a 4-bit benchmarking layer with VSQ. At a nominal voltage of 0.67 V, the accelerator achieves 1734 inferences/s/W (38.7 TOPS/W) with only 0.7% accuracy loss on BERT-Base and 4714 inferences/s/W (38.6 TOPS/W) with 0.15% accuracy loss on ResNet-50 by using quantization-aware fine-tuning to recover accuracy, demonstrating a practical accelerator design for energy-efficient DNN inference.

Detection of Thermal Covert Channel Attacks Based on Classification of Components of the Thermal Signal Features

In response to growing security challenges facing many-core systems imposed by thermal covert channel (TCC) attacks, a number of threshold-based detection methods have been proposed. In this paper, we show that these threshold-based detection methods are inadequate to detect TCCs that harness advanced signaling and specific modulation techniques. Since the frequency representation of a TCC signal is found to have multiple side lobes, this important feature shall be explored to enhance the TCC detection capability. To this end, we present a pattern-classification-based TCC detection method using an artificial neural network that is trained with a large volume of spectrum traces of TCC signals. After proper training, this classifier is applied at runtime to infer TCCs, should they exist. The proposed detection method is able to achieve a detection accuracy of 99%, even in the presence of the stealthiest TCCs ever discovered. Because of its low runtime overhead ( <inline-formula><tex-math notation="LaTeX">$&lt; 0.187\%$</tex-math></inline-formula> ) and low energy overhead ( <inline-formula><tex-math notation="LaTeX">$&lt; 0.072\%$</tex-math></inline-formula> ), this proposed detection method can be indispensable in fighting against TCC attacks in many-core systems. With such a high accuracy in detecting TCCs, powerful countermeasures, like the ones based on dynamic voltage and frequency scaling (DVFS), can be rightfully applied to neutralize any malicious core participating in a TCC attack.

Trainable quantization for Speedy Spiking Neural Networks.

Spiking neural networks are considered as the third generation of Artificial Neural Networks. SNNs perform computation using neurons and synapses that communicate using binary and asynchronous signals known as spikes. They have attracted significant research interest over the last years since their computing paradigm allows theoretically sparse and low-power operations. This hypothetical gain, used from the beginning of the neuromorphic research, was however limited by three main factors: the absence of an efficient learning rule competing with the one of classical deep learning, the lack of mature learning framework, and an important data processing latency finally generating energy overhead. While the first two limitations have recently been addressed in the literature, the major problem of latency is not solved yet. Indeed, information is not exchanged instantaneously between spiking neurons but gradually builds up over time as spikes are generated and propagated through the network. This paper focuses on quantization error, one of the main consequence of the SNN discrete representation of information. We argue that the quantization error is the main source of accuracy drop between ANN and SNN. In this article we propose an in-depth characterization of SNN quantization noise. We then propose a end-to-end direct learning approach based on a new trainable spiking neural model. This model allows adapting the threshold of neurons during training and implements efficient quantization strategies. This novel approach better explains the global behavior of SNNs and minimizes the quantization noise during training. The resulting SNN can be trained over a limited amount of timesteps, reducing latency, while beating state of the art accuracy and preserving high sparsity on the main datasets considered in the neuromorphic community.

Memristor-Based Light-Weight Transformer Circuit Implementation for Speech Recognizing

Transformer network (TN) is a promising model widely used for natural language processing (NLP), computer vision (CV), and audio processing (AP). However, the large number of multiples and additions (multi-adds) in TN leads to a massive number of parameters in the TN model, causing an increasing number of hardware resource overhead and thus making it difficult to deploy the TN model to edge devices. To alleviate this problem, this paper proposes a circuit implementation scheme for a memristor-based lightweight transformer network (MLTN). In addition, the speech recognition task based on the speech command dataset is implemented through the modular hardware design approach, which provides an effective lightweight hardware implementation scheme for the edge devices. Further, the retention, noise immunity, variability, and stability of the designed circuit are analyzed by simulation experiments with LTSPICE. The results show that the proposed MLTN circuit not only has a lighter network model but also has significant recognition accuracy with different hardware scales. Specifically, MLTN-40 can achieve 96.18% simulation accuracy, while the energy overhead is only 3.8% of the ARM framework, which is commonly used for edge devices, and 39.9% of the traditional server (GPU), making it a promising solution for implementing speech recognition on edge devices.

A Practical System for 3-D Hand Pose Tracking Using EMG Wearables With Applications to Prosthetics and User Interfaces

Ubiquitous finger motion tracking enables a number of exciting applications in augmented reality, sports analytics, rehabilitation-healthcare, haptics, etc. This article presents NeuroPose, a system that shows the feasibility of 3-D finger motion tracking using a platform of wearable electromyography (EMG) sensors. EMG sensors can sense electrical potential from muscles due to finger activation, thus offering rich information for fine-grained finger motion sensing. However, converting the sensor information to 3-D finger poses is non trivial since signals from multiple fingers superimpose at the sensor in complex patterns. Toward solving this problem, NeuroPose fuses information from anatomical constraints of finger motion with machine learning architectures on recurrent neural networks (RNNs), encoder–decoder networks, and ResNets to extract 3-D finger motion from noisy EMG data. The generated motion pattern is temporally smooth as well as anatomically consistent. Furthermore, a transfer learning algorithm is leveraged to adapt a pretrained model on one user to a new user with minimal training overhead. A systematic study with 12 users demonstrates a median error of 6.24° and a 90%-ile error of 18.33° in tracking 3-D finger joint angles. The accuracy is robust to natural variation in sensor mounting positions as well as changes in wrist positions of the user. In addition, this article validates the feasibility of mirrored bilateral training approach with applications in prosthetic devices. Finally, NeuroPose is comprehensively evaluated on both low-end and recent smartphones with a processing latency of 0.019 s and low energy overhead.

Sub-10 fJ/bit radiation-hard nanoelectromechanical non-volatile memory

With the exponential growth of the semiconductor industry, radiation-hardness has become an indispensable property of memory devices. However, implementation of radiation-hardened semiconductor memory devices inevitably requires various radiation-hardening technologies from the layout level to the system level, and such technologies incur a significant energy overhead. Thus, there is a growing demand for emerging memory devices that are energy-efficient and intrinsically radiation-hard. Here, we report a nanoelectromechanical non-volatile memory (NEM-NVM) with an ultra-low energy consumption and radiation-hardness. To achieve an ultra-low operating energy of less than 10 {{{{{{rm{fJ; bit}}}}}}}^{-1}, we introduce an out-of-plane electrode configuration and electrothermal erase operation. These approaches enable the NEM-NVM to be programmed with an ultra-low energy of 2.83 {{{{{{rm{fJ; bit}}}}}}}^{-1}. Furthermore, due to its mechanically operating mechanisms and radiation-robust structural material, the NEM-NVM retains its superb characteristics without radiation-induced degradation such as increased leakage current, threshold voltage shift, and unintended bit-flip even after 1 Mrad irradiation.

A Threshold Implementation-Based Neural Network Accelerator With Power and Electromagnetic Side-Channel Countermeasures

With the recent advancements in machine learning (ML) theory, a lot of energy-efficient neural network (NN) accelerators have been developed. However, their associated side-channel security vulnerabilities pose a major concern. There have been several proof-of-concept attacks demonstrating the extraction of their model parameters and input data. This work introduces a threshold implementation (TI) masking-based NN accelerator that secures model parameters and inputs against power and electromagnetic (EM) side-channel attacks. The 0.159 mm2 demonstration in 28 nm runs at 125 MHz at 0.95 V and limits the area and energy overhead to 64% and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.5\times $ </tex-math></inline-formula> , respectively, while demonstrating security even greater than 2M traces. The accelerator also secures model parameters through encryption and the inputs against horizontal power analysis (HPA) attacks.

Reducing the Energy Overheads of a Manufacturing Company

The paper deals with reducing energy costs. Nowadays, with ever-increasing energy prices, it is important for a company to minimize the consumption of all energy. In the course of its activities, the company comes into contact with other entities of different nature. It is connected to this environment by inputs and outputs. Every company has certain goals. A very important group of objectives consists of economic objectives such as achieving profit, ensuring financial stability and liquidity of the company, efficiency of production and other activities.

Edge Computing Offloading Method Based on Deep Reinforcement Learning for Gas Pipeline Leak Detection

Traditional gas pipeline leak detection methods require task offload decisions in the cloud, which has low real time performance. The emergence of edge computing provides a solution by enabling offload decisions directly at the edge server, improving real-time performance; however, energy is the new bottleneck. Therefore, focusing on the gas transmission pipeline leakage detection scenario in real time, a novel detection algorithm that combines the benefits of both the heuristic algorithm and the advantage actor critic (AAC) algorithm is proposed in this paper. It aims at optimization with the goal of real-time guarantee of pipeline mapping analysis tasks and maximizing the survival time of portable gas leak detectors. Since the computing power of portable detection devices is limited, as they are powered by batteries, the main problem to be solved in this study is how to take into account the node energy overhead while guaranteeing the system performance requirements. By introducing the idea of edge computing and taking the mapping relationship between resource occupation and energy consumption as the starting point, the optimization model is established, with the goal to optimize the total system cost (TSC). This is composed of the node’s transmission energy consumption, local computing energy consumption, and residual electricity weight. In order to minimize TSC, the algorithm uses the AAC network to make task scheduling decisions and judge whether tasks need to be offloaded, and uses heuristic strategies and the Cauchy–Buniakowsky–Schwarz inequality to determine the allocation of communication resources. The experiments show that the proposed algorithm in this paper can meet the real-time requirements of the detector, and achieve lower energy consumption. The proposed algorithm saves approximately 56% of the system energy compared to the Deep Q Network (DQN) algorithm. Compared with the artificial gorilla troops Optimizer (GTO), the black widow optimization algorithm (BWOA), the exploration-enhanced grey wolf optimizer (EEGWO), the African vultures optimization algorithm (AVOA), and the driving training-based optimization (DTBO), it saves 21%, 38%, 30%, 31%, and 44% of energy consumption, respectively. Compared to the fully local computing and fully offloading algorithms, it saves 50% and 30%, respectively. Meanwhile, the task completion rate of this algorithm reaches 96.3%, which is the best real-time performance among these algorithms.

An Energy-Efficient Convolutional Neural Network Processor Architecture Based on a Systolic Array

Deep convolutional neural networks (CNNs) have shown strong abilities in the application of artificial intelligence. However, due to their extensive amount of computation, traditional processors have low energy efficiency when executing CNN algorithms, which is unacceptable for portable devices with limited hardware cost and battery capacity, so designing a CNN-specific processor is necessary. In this paper, we propose an energy-efficient CNN processor architecture for lightweight devices with a processing elements (PEs) array consisting of 384 PEs. Using the systolic array-based PE array, it realizes parallel operations between filter rows and between channels of output feature maps, supporting the acceleration of 3D convolution and fully connected computation with various parameters by configuring internal instruction registers. The computing strategy based on the proposed systolic dataflow achieves less hardware overhead compared with other strategies, and the reuse of image values and weight values, which effectively reduce the power of memory access. A memory system with a multi-level storage structure combined with register file (RF) and SRAM is used in the proposed CNN processor, which further reduces the energy overhead of computing. The proposed CNN processor architecture has been verified on a ZC706 FPGA platform using VGG-16 based on the proposed image segmentation method, the evaluation results indicate that the peak throughput achieves 115.2 GOP/s consuming 3.801 W at 150 MHz, energy efficiency and DSP efficiency reaches 30.32 GOP/s/W and 0.26 GOP/s/DSP, respectively.

A Spintronic 2M/7T Computation-in-Memory Cell

Computing data-intensive applications on the von Neumann architecture lead to significant performance and energy overheads. The concept of computation in memory (CiM) addresses the bottleneck of von Neumann machines by reducing the data movement in the computing system. Emerging resistive non-volatile memory technologies, as well as volatile memories (SRAM and DRAM), can be used to realize architectures based on the CiM paradigm. In this paper, we propose a hybrid cell design to provide the opportunity for CiM by combining the magnetic tunnel junction (MTJ) and the conventional 6T-SRAM cell. The cell performs CiM operations based on stateful in-array computation, which has better scalability for multiple operands compared with stateless computation in the periphery. Various logic operations such as XOR, OR, and IMP can be performed with the proposed design. In addition, the proposed cell can also operate as a conventional memory cell to read and write volatile as well as non-volatile data. The obtained simulation results show that the proposed CiM-A design can increase the performance of regular memory architectures by reducing the delay by 8 times and the energy by 13 times for database query applications consisting of consecutive bitwise operations with minimum overhead.

A pairing-free data authentication and aggregation mechanism for Intelligent Healthcare System

The Intelligent Healthcare System (IHS), a prominent medical area underlying the Internet of Things (IoT), might be deployed to gather and analyze health information across a range of sources to deliver improved as well as more economical medical treatment. Managing all the health-related information over the internet becomes a challenging chore as numerous people rely on it implicitly or explicitly. The significant challenges of IHS comprise the privacy and security along with protection of aggregation of patients’ data as sensors obtain and transmit patient’s health information. The few of surveyed existing works based on security and privacy goals for IHS are inefficient due to cumbersome of huge computational cost and non-resilience of different attacks. Therefore, to resolve such challenges: to provide the robustness, preserve security goals and improve the efficiency, we introduced an efficient data authentication and aggregation protocol for IHS without pairing and ID-based cryptography. Provable security has been provided with Random Oracle model (ROM), which proofs the robustness of the presented scheme. Thus, the defined security requirements of our proposed work have been achieved. The performance analysis through MIRACL shows that the scheme presented outperforms the similar prior mechanisms in terms of computational-communicational cost and energy overheads.

RANCE: A Randomly Centralized and On-Demand Clustering Protocol for Mobile Ad Hoc Networks

LEACH-like clustering protocols focus mainly on the low-power, low-rate, and low-wakeup network applications, and work in a multiround clustering strategy that causes frequent handovers of cluster heads (CHs), thus less support for real-time services that require stable cluster topologies. Besides, these protocols are faced with respective drawbacks, such as suboptimality of selected heads, costly node-base station (BS) energy overheads, lack of runtime cluster maintenance, etc. This article proposes RANCE, a randomly centralized and on-demand clustering protocol, aiming at prolonging nodes’ clustered time to support internodes collaboration while being energy efficient in mobile ad hoc networks. First, RANCE designs a randomly centralized CH selection mechanism in which every node in the local wireless network is eligible to initiate the centralized CH selection, so that the self-organizing characteristics of mobile ad hoc nodes can be utilized for head selection optimization. Second, taking into account the wireless volatility caused by changes of topology, obstacles, signal strength, etc., the fine-grained cluster relationships maintenance is provided by means of multilevel aliveness and adaptive bidirectional heartbeat packets. Third, RANCE works in an event-driven and on-demand manner instead of a time-triggered manner in LEACH-like protocols, to reduce the impact on continuous services caused by frequent CH handovers among all nodes. Simulation results show that RANCE provides longer clustered time (over 99% of nodes’ lifetime in networks more than 100 nodes) and good clustering scalability with high consistency at minimum energy cost, and exhibits good potentials in mobile wireless environments that are infrastructureless/poor for continuous missions.

Leveraging the Properties of mmWave Signals for 3D Finger Motion Tracking for Interactive IoT Applications

mmWave signals form a critical component of 5G and next-generation wireless networks, which are also being increasingly considered for sensing the environment around us to enable ubiquitous IoT applications. In this context, this paper leverages the properties of mmWave signals for tracking 3D finger motion for interactive IoT applications. While conventional vision-based solutions break down under poor lighting, occlusions, and also suffer from privacy concerns, mmWave signals work under typical occlusions and non-line-of-sight conditions, while being privacy-preserving. In contrast to prior works on mmWave sensing that focus on predefined gesture classification, this work performs continuous 3D finger motion tracking. Towards this end, we first observe via simulations and experiments that the small size of fingers coupled with specular reflections do not yield stable mmWave reflections. However, we make an interesting observation that focusing on the forearm instead of the fingers can provide stable reflections for 3D finger motion tracking. Muscles that activate the fingers extend through the forearm, whose motion manifests as vibrations on the forearm. By analyzing the variation in phases of reflected mmWave signals from the forearm, this paper designs mm4Arm, a system that tracks 3D finger motion. Nontrivial challenges arise due to the high dimensional search space, complex vibration patterns, diversity across users, hardware noise, etc. mm4Arm exploits anatomical constraints in finger motions and fuses them with machine learning architectures based on encoder-decoder and ResNets in enabling accurate tracking. A systematic performance evaluation with 10 users demonstrates a median error of 5.73° (location error of 4.07 mm) with robustness to multipath and natural variation in hand position/orientation. The accuracy is also consistent under non-line-of-sight conditions and clothing that might occlude the forearm. mm4Arm runs on smartphones with a latency of 19 ms and low energy overhead.