Energy-efficient Hardware Implementation Research Articles

An ultra-fast and energy-efficient CNTFET-based image corner detection hardware for real-time image processing applications

In image processing and machine vision, corner detection is pivotal in diverse applications, including computer vision, 3D reconstruction, face detection, object tracking, and video technologies. Despite the wide usage, the real-time and energy-efficient hardware implementation of corner detection algorithms remains a critical challenge because of the computational resource limitations. On the other hand, owing to the complicated nature of corner detection algorithms, their hardware implementation has been limited to the graphics processing unit (GPU) and field programmable gate arrays (FPGA) platforms. In this regard, this work aims to propose a novel and ultra-efficient carbon nanotube field-effect transistor (CNTFET)-based hardware for image corner detection. Thanks to the proposed corner detection algorithm, the designed hardware has been realized using 2742 transistors with competitive accuracy. The proposed corner detection hardware indicated a remarkable salt-and-pepper noise immunity without using any noise reduction circuit. Our comprehensive simulations demonstrate 78%, 87%, and 94.5% total average improvements in delay, power, and energy compared to the other related corner detection hardware. Moreover, the proposed CNTFET-based corner detection hardware shows a 43 ps propagation delay, demonstrating its real-time operation. The proposed corner detection algorithm at the system level shows suitable accuracy metrics such as Recall, Precision, and error of detection (EoD) compared to the other well-known corner detectors. Our method has established a new pathway for real-time circuit-level hardware design for image processing and machine vision applications.

Forward-forward training of an optical neural network.

Neural networks (NNs) have demonstrated remarkable capabilities in various tasks, but their computation-intensive nature demands faster and more energy-efficient hardware implementations. Optics-based platforms, using technologies such as silicon photonics and spatial light modulators, offer promising avenues for achieving this goal. However, training multiple programmable layers together with these physical systems poses challenges, as they are difficult to fully characterize and describe with differentiable functions, hindering the use of error backpropagation algorithm. The recently introduced forward-forward algorithm (FFA) eliminates the need for perfect characterization of the physical learning system and shows promise for efficient training with large numbers of programmable parameters. The FFA does not require backpropagating an error signal to update the weights, rather the weights are updated by only sending information in one direction. The local loss function for each set of trainable weights enables low-power analog hardware implementations without resorting to metaheuristic algorithms or reinforcement learning. In this paper, we present an experiment utilizing multimode nonlinear wave propagation in an optical fiber demonstrating the feasibility of the FFA approach using an optical system. The results show that incorporating optical transforms in multilayer NN architectures trained with the FFA can lead to performance improvements, even with a relatively small number of trainable weights. The proposed method offers a new path to the challenge of training optical NNs and provides insights into leveraging physical transformations for enhancing the NN performance.

Energy-Efficient Hardware Implementation of Fully Connected Artificial Neural Networks Using Approximate Arithmetic Blocks.

In this paper, we explore efficient hardware implementation of feedforward artificial neural networks (ANNs) using approximate adders and multipliers. Due to a large area requirement in a parallel architecture, the ANNs are implemented under the time-multiplexed architecture where computing resources are re-used in the multiply accumulate (MAC) blocks. The efficient hardware implementation of ANNs is realized by replacing the exact adders and multipliers in the MAC blocks by the approximate ones taking into account the hardware accuracy. Additionally, an algorithm to determine the approximate level of multipliers and adders due to the expected accuracy is proposed. As an application, the MNIST and SVHN databases are considered. To examine the efficiency of the proposed method, various architectures and structures of ANNs are realized. Experimental results show that the ANNs designed using the proposed approximate multiplier have a smaller area and consume less energy than those designed using previously proposed prominent approximate multipliers. It is also observed that the use of both approximate adders and multipliers yields, respectively, up to 50% and 10% reduction in energy consumption and area of the ANN design with a small deviation or better hardware accuracy when compared to the exact adders and multipliers.

Energy-Efficient Recurrent Neural Network With MRAM-Based Probabilistic Activation Functions

Herein, we develop a programmable energy-efficient hardware implementation for Recurrent Neural Networks (RNNs) with Resistive Random-Access Memory (ReRAM) synapses and ultra-low power, area-efficient spin-based activation functions. To attain high energy-efficiency while maintaining accuracy, a novel Computing-in-Memory (CiM) architecture is proposed to leverage data-level parallelism during the evaluation phase. We employ an MRAM-based Adjustable Probabilistic Activation Function (APAF) via a low-power tunable activation mechanism, providing adjustable levels of accuracy to mimic ideal sigmoid and tanh thresholding along with a matching algorithm to regulate the neuron properties. Our hardware/software cross-layer simulation shows that our proposed design achieves up to 74.5× energy-efficiency with ∼11× area reduction compared to its counterpart designs while keeping the accuracy comparable.

Word-Based Processor Structure for Montgomery Modular Multiplier Suitable for Compact IoT Edge Devices

The Internet of Things (IoT) is an emerging technology that forms a huge network of different objects and intelligent devices. IoT Security is becoming more important due to the exchange of sensitive sensor data and the potential for incorporating the virtual and real worlds. IoT edge devices create serious security threats to network systems. Due to their limited resources, it is challenging to implement cryptographic protocols on these devices to secure them. To address this problem, we should perform compact implementation of cryptographic algorithms on these devices. At the heart of most cryptographic algorithms is the modular multiplication operation. Therefore, efficient implementation of this operation will have a great impact on the implementation of the whole cryptographic protocol. In this paper, we will focus on the resource and energy efficient hardware implementation of the adopted Montgomery modular multiplication algorithm over GF(2m). The main building block of the proposed word-based processor structure is a processor array that has a modular structure with local connectivity between its processing elements. The ability to manage the saving amounts of area, delay, and consumed energy is the main benefit of the suggested hardware structure. We used ASIC technology to implement the suggested word-based processor structure. The final results show an average reduction in the area of 86.3% when compared with the competitive word-based multiplier structures. Additionally, the recommended design achieves significant average savings in area-time product, power, and consumed energy of 53.7%, 83.2%, and 72.6%, receptively, over the competitive ones. The obtained results show that the provided processor structure is best suited for application in compact IoT edge devices with limited resources.

Efficient Implementation of Max-Pooling Algorithm Exploiting History-Effect in Ferroelectric-FinFETs

Convolutional neural networks (CNNs) have become the state-of-the-art tool for image classification, object detection, and segmentation. The max-pooling layer in the CNN architecture leads to a significant enhancement in the classification accuracy by extracting the most prominent features from the feature maps produced by the convolutional layers, reducing the number of computations, and preventing overfitting. However, the conventional digital/analog implementations of the max-pooling layer are energy-hungry. Moreover, compact and energy-efficient hardware implementation of the max-pooling layer is essential for realizing CNNs which may handle complex artificial intelligence (AI)/machine learning (ML) workloads on Internet of Things (IoT) edge devices. To this end, in this work, for the first time, we have proposed a highly scalable, compact, and energy-efficient implementation of the max-pooling algorithm utilizing a single Ferroelectric (Fe)-FinFET. We have designed a novel feature-to-pulse mapping scheme and exploit the history-effect in the polarization state of Fe-FinFETs (which is otherwise undesirable for memory application). Our comprehensive analysis using an experimentally calibrated compact model for the doped-HfO2 ferroelectric capacitor integrated with 14-nm-FinFET technology indicates that the final polarization state of the Fe-FinFET corresponds to the maximum input feature irrespective of the order of application of inputs with a mean relative error of 4.95% while consuming 22.8 fJ of energy. Furthermore, extensive network simulations show that such a small deviation of the max-pooling layer output from the ideal value does not lead to a significant degradation in the classification accuracy ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$&lt; 2$ </tex-math></inline-formula> % drop in network fidelity).

Nonuniform Compressive Sensing via Ohmic Voltage Attenuation: A Memristive Crossbar Design Approach Leveraging Intrinsic Computation

Compressive sensing (CS) is a promising technique for transmitting signals in power-critical applications such as Internet of Things (IoT) devices. Nonuniform CS optimizes this process by adjusting sampling frequency based on the relative importance levels characterized by the signal of interest. Recent advances have yielded energy-efficient hardware implementations of CS sampling, leveraging spin-based crossbar architectures for in-memory vector-matrix multiplication and through the use of probabilistic bit (p-bit) devices to generate tunable random outputs for writing the array. Thus, a region of interest is generated via column-ordered density of on-state devices. Herein, we propose a simple design for supplying inputs to the p-bit devices, based on Ohmic voltage attenuation occurring along the word lines of the crossbar array. The technique embeds some required computations to be conducted intrinsically by the cross-points of the memristive array, thus bypassing overheads of conventional instruction execution and eliminating the need for costly hardware components, such as lookup tables (LUTs) and data converters. The design is shown to be robust for various array sizes and parasitics while generating the appropriate tuning signals within a single clock cycle duration of 1.6 ns, and at an energy overhead of 333 fJ. Compared with a standard approach using LUTs and digital to-analog converters, the design herein achieves a 583-fold reduction in energy and 23-fold reduction in transistor count.

Online Fault Detection in ReRAM-Based Computing Systems for Inferencing

A resistive switching random access memory (ReRAM)-based computing system (RCS) provides an energy-efficient hardware implementation of vector–matrix multiplication for machine-learning hardware. However, it is susceptible to faults due to the immature resistive switching random access memory (ReRAM) fabrication process. We propose an efficient online fault-detection method for RCS. This method monitors the dynamic power consumption of each ReRAM crossbar and determines the occurrence of faults when a changepoint is detected in the monitored power-consumption time series. To estimate the percentage of faulty cells in a faulty ReRAM crossbar, we compute statistical features immediately before and after the changepoint and use them as independent variables; we use the percentage of faulty cells as dependent variables to train a predictive model using machine learning. In this way, the computationally expensive fault localization and error-recovery steps are carried out only when a high fault rate is estimated. Simulation results show that, with the fault-detection method and the predictive model, the test time is significantly reduced, the hardware overhead is negligible, and high classification accuracy for the MNIST and CIFAR-10 datasets using RCS can still be ensured.

ExPAN(N)D: Exploring Posits for Efficient Artificial Neural Network Design in FPGA-Based Systems

The high computational complexity, memory footprints, and energy requirements of machine learning models, such as Artificial Neural Networks (ANNs), hinder their deployment on resource-constrained embedded systems. Most state-of-the-art works have considered this problem by proposing various low bit-width data representation schemes and optimized arithmetic operators’ implementations. To further elevate the implementation gains offered by these individual techniques, there is a need to cross-examine and combine these techniques’ unique features. This paper presents ExPAN(N)D, a framework to analyze and ingather the efficacy of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Posit</i> number representation scheme and the efficiency of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fixed-point</i> arithmetic implementations for ANNs. The Posit scheme offers a better dynamic range and higher precision for various applications than IEEE 754 single-precision floating-point format. However, due to the dynamic nature of the various fields of the Posit scheme, the corresponding arithmetic circuits have higher critical path delay and resource requirements than the single-precision-based arithmetic units. Towards this end, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Posit to fixed-point</i> converter for enabling high-performance and energy-efficient hardware implementations for ANNs with minimal drop in the output accuracy. We also propose a modified Posit-based representation to store the trained parameters of a network. With the proposed Posit to fixed-point converter-based designs, we provide multiple design points with varying accuracy-performance trade-offs for an ANN. For instance, compared to the lowest power dissipating Posit-only accelerator design, one of our proposed designs results in 80% and 48% reduction in power dissipation and LUT utilization respectively, with marginal increase in classification error for Imagenet dataset classification using VGG-16.

Towards Efficient Neuromorphic Hardware: Unsupervised Adaptive Neuron Pruning

To solve real-time challenges, neuromorphic systems generally require deep and complex network structures. Thus, it is crucial to search for effective solutions that can reduce network complexity, improve energy efficiency, and maintain high accuracy. To this end, we propose unsupervised pruning strategies that are focused on pruning neurons while training in spiking neural networks (SNNs) by utilizing network dynamics. The importance of neurons is determined by the fact that neurons that fire more spikes contribute more to network performance. Based on these criteria, we demonstrate that pruning with an adaptive spike count threshold provides a simple and effective approach that can reduce network size significantly and maintain high classification accuracy. The online adaptive pruning shows potential for developing energy-efficient training techniques due to less memory access and less weight-update computation. Furthermore, a parallel digital implementation scheme is proposed to implement spiking neural networks (SNNs) on field programmable gate array (FPGA). Notably, our proposed pruning strategies preserve the dense format of weight matrices, so the implementation architecture remains the same after network compression. The adaptive pruning strategy enables 2.3× reduction in memory size and 2.8× improvement on energy efficiency when 400 neurons are pruned from an 800-neuron network, while the loss of classification accuracy is 1.69%. And the best choice of pruning percentage depends on the trade-off among accuracy, memory, and energy. Therefore, this work offers a promising solution for effective network compression and energy-efficient hardware implementation of neuromorphic systems in real-time applications.

Algorithmic Fault Detection for RRAM-based Matrix Operations

An RRAM-based computing system (RCS) provides an energy-efficient hardware implementation of vector-matrix multiplication for machine-learning hardware. However, it is vulnerable to faults due to the immature RRAM fabrication process. We propose an efficient fault tolerance method for RCS; the proposed method, referred to as extended-ABFT (X-ABFT), is inspired by algorithm-based fault tolerance (ABFT). We utilize row checksums and test-input vectors to extract signatures for fault detection and error correction. We present a solution to alleviate the overflow problem caused by the limited number of voltage levels for the test-input signals. Simulation results show that for a Hopfield classifier with faults in 5% of its RRAM cells, X-ABFT allows us to achieve nearly the same classification accuracy as in the fault-free case.

An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications

Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms for SNNs, however, lags behind hardware implementations: most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding.

Energy-Efficient Hardware Implementation of Road-Lane Detection Based on Hough Transform with Parallelized Voting Procedure and Local Maximum Algorithm

Energy-Efficient Hardware Implementation of Road-Lane Detection Based on Hough Transform with Parallelized Voting Procedure and Local Maximum Algorithm

Hypercolumn Sparsification for Low-Power Convolutional Neural Networks

We provide here a novel method, called hypercolumn sparsification, to achieve high recognition performance for convolutional neural networks (CNNs) despite low-precision weights and activities during both training and test phases. This method is applicable to any CNN architecture that operates on signal patterns (e.g., audio, image, video) to extract information such as class membership. It operates on the stack of feature maps in each of the cascading feature matching and pooling layers through the processing hierarchy of the CNN by an explicit competitive process ( k -WTA, winner take all) that generates a sparse feature vector at each spatial location. This principle is inspired by local brain circuits, where neurons tuned to respond to different patterns in the incoming signals from an upstream region inhibit each other using interneurons, such that only the ones that are maximally activated survive the quenching threshold. We show this process of sparsification is critical for probabilistic learning of low-precision weights and bias terms, thereby making pattern recognition amenable for energy-efficient hardware implementations. Further, we show that hypercolumn sparsification could lead to more data-efficient learning as well as having an emergent property of significantly pruning down the number of connections in the network. A theoretical account and empirical analysis are provided to understand these effects better.

Photonic In-Memory Computing Primitive for Spiking Neural Networks Using Phase-Change Materials

Spiking Neural Networks (SNNs) offer an event-driven and more biologically realistic alternative to standard Artificial Neural Networks based on analog information processing. This can potentially enable energy-efficient hardware implementations of neuromorphic systems which emulate the functional units of the brain, namely, neurons and synapses. Recent demonstrations of ultra-fast photonic computing devices based on phase-change materials (PCMs) show promise of addressing limitations of electrically driven neuromorphic systems. However, scaling these standalone computing devices to a parallel in-memory computing primitive is a challenge. In this work, we utilize the optical properties of the PCM, Ge\textsubscript{2}Sb\textsubscript{2}Te\textsubscript{5} (GST), to propose a Photonic Spiking Neural Network computing primitive, comprising of a non-volatile synaptic array integrated seamlessly with previously explored `integrate-and-fire' neurons. The proposed design realizes an `in-memory' computing platform that leverages the inherent parallelism of wavelength-division-multiplexing (WDM). We show that the proposed computing platform can be used to emulate a SNN inferencing engine for image classification tasks. The proposed design not only bridges the gap between isolated computing devices and parallel large-scale implementation, but also paves the way for ultra-fast computing and localized on-chip learning.

A Comparison of Machine Learning Classifiers for Energy-Efficient Implementation of Seizure Detection

The closed-loop application of electrical stimulation via chronically implanted electrodes is a novel approach to stop seizures in patients with focal-onset epilepsy. To this end, an energy efficient seizure detector that can be implemented in an implantable device is of crucial importance. In this study, we first evaluated the performance of two machine learning algorithms (Random Forest classifier and support vector machine (SVM)) by using selected time and frequency domain features with a limited need of computational resources. Performance of the algorithms was further compared to a detection strategy implemented in an existing closed loop neurostimulation device for the treatment of epilepsy. The results show a superior performance of the Random Forest classifier compared to the SVM classifier and the reference approach. Next, we implemented the feature extraction and classification process of the Random Forest classifier on a microcontroller to evaluate the energy efficiency of this seizure detector. In conclusion, the feature set in combination with Random Forest classifier is an energy efficient hardware implementation that shows an improvement of detection sensitivity and specificity compared to the presently available closed-loop intervention in epilepsy while preserving a low detection delay.

A Novel Approach for Using TSVs As Membrane Capacitance in Neuromorphic 3-D IC

An advanced neurophysiological computing system can incorporate a 3-D integration system composed of emerging nano-scale devices to provide massive parallelism having high speed, low cost, and energy efficient hardware implementation. Due to process technology constraints, a certain amount of redundant through silicon vias (TSVs) and dummy TSVs are always required in a 3-D integrated system. In this paper, we propose to use these redundant and dummy TSVs to supply the neuronal membrane capacitance that maps the membrane electrical activity in a hybrid 3-D neuromorphic system. This proposition could also serve the need of neuronal ion transportation dynamics. We also investigate two new methodologies that could significantly enhance the TSV capacitance in a 3-D neuromorphic system. The capacitance of these enhanced TSVs is studied with analytical models; the accuracy of the models are evaluated against 3-D field extracted values. The advantage of using the TSVs to mimic membrane capacitance in a 3-D neuromorphic chip is demonstrated through comparisons of both silicon area and energy consumption against their 2-D counterpart designs.

Online Adaptation and Energy Minimization for Hardware Recurrent Spiking Neural Networks

The Liquid State Machine (LSM) is a promising model of recurrent spiking neural networks that provides an appealing brain-inspired computing paradigm for machine-learning applications such as pattern recognition. Moreover, processing information directly on spiking events makes the LSM well suited for cost- and energy-efficient hardware implementation. In this article, we systematically present three techniques for optimizing energy efficiency while maintaining good performance of the proposed LSM neural processors from both an algorithmic and hardware implementation point of view. First, to realize adaptive LSM neural processors, thus boost learning performance, we propose a hardware-friendly Spike-Timing Dependent Plastic (STDP) mechanism for on-chip tuning. Then, the LSM processor incorporates a novel runtime correlation-based neuron gating scheme to minimize the power dissipated by reservoir neurons. Furthermore, an activity-dependent clock gating approach is presented to address the energy inefficiency due to the memory-intensive nature of the proposed neural processors. Using two different real-world tasks of speech and image recognition to benchmark, we demonstrate that the proposed architecture boosts the average learning performance by up to 2.0% while reducing energy dissipation by up to 29% compared to a baseline LSM with little extra hardware overhead on a Xilinx Virtex-6 FPGA.

An Energy-Efficient Hardware Implementation of HOG-Based Object Detection at 1080HD 60 fps with Multi-Scale Support

A real-time and energy-efficient multi-scale object detector hardware implementation is presented in this paper. Detection is done using Histogram of Oriented Gradients (HOG) features and Support Vector Machine (SVM) classification. Multi-scale detection is essential for robust and practical applications to detect objects of different sizes. Parallel detectors with balanced workload are used to increase the throughput, enabling voltage scaling and energy consumption reduction. Image pre-processing is also introduced to further reduce power and area costs of the image scales generation. This design can operate on high definition 1080HD video at 60 fps in real-time with a clock rate of 270 MHz, and consumes 45.3 mW (0.36 nJ/pixel) based on post-layout simulations. The ASIC has an area of 490 kgates and 0.538 Mbit on-chip memory in a 45 nm SOI CMOS process.

Real-Time Classification of Bladder Events for Effective Diagnosis and Treatment of Urinary Incontinence.

Diagnosis of lower urinary tract dysfunction with urodynamics has historically relied on data acquired from multiple sensors using nonphysiologically fast cystometric filling. In addition, state-of-the-art neuromodulation approaches to restore bladder function could benefit from a bladder sensor for closed-loop control, but a practical sensor and automated data analysis are not available. We have developed an algorithm for real-time bladder event detection based on a single in situ sensor, making it attractive for both extended ambulatory bladder monitoring and closed-loop control of stimulation systems for diagnosis and treatment of bladder overactivity. Using bladder pressure data acquired from 14 human subjects with neurogenic bladder, we developed context-aware thresholding, a novel, parameterized, user-tunable algorithmic framework capable of real-time classification of bladder events, such as detrusor contractions, from single-sensor bladder pressure data. We compare six event detection algorithms with both single-sensor and two-sensor systems using a metric termed Conditional Stimulation Score, which ranks algorithms based on projected stimulation efficacy and efficiency. We demonstrate that adaptive methods are more robust against day-to-day variations than static thresholding, improving sensitivity and specificity without parameter modifications. Relative to other methods, context-aware thresholding is fast, robust, highly accurate, noise-tolerant, and amenable to energy-efficient hardware implementation, which is important for mapping to an implant device.