Energy-efficient Deep Learning Research Articles

Hybrid energy efficient deep learning and redactable consortium blockchain-based secure framework for smart parking in smart cities

Hybrid energy efficient deep learning and redactable consortium blockchain-based secure framework for smart parking in smart cities

Leveraging Bit-Serial Architectures for Hardware-Oriented Deep Learning Accelerators with Column-Buffering Dataflow

Bit-serial neural network accelerators address the growing need for compact and energy-efficient deep learning tools. Traditional neural network accelerators, while effective, often grapple with issues of size, power consumption, and versatility in handling a variety of computational tasks. To counter these challenges, this paper introduces an approach that hinges on the integration of bit-serial processing with advanced dataflow techniques and architectural optimizations. Central to this approach is a column-buffering (CB) dataflow, which significantly reduces access and movement requirements for the input feature map (IFM), thereby enhancing efficiency. Moreover, a simplified quantization process effectively eliminates biases, streamlining the overall computation process. Furthermore, this paper presents a meticulously designed LeNet-5 accelerator leveraging a convolutional layer processing element array (CL PEA) architecture incorporating an improved bit-serial multiply–accumulate unit (MAC). Empirically, our work demonstrates superior performance in terms of frequency, chip area, and power consumption compared to current state-of-the-art ASIC designs. Specifically, our design utilizes fewer hardware resources to implement a complete accelerator, achieving a high performance of 7.87 GOPS on a Xilinx Kintex-7 FPGA with a brief processing time of 284.13 μs. The results affirm that our design is exceptionally suited for applications requiring compact, low-power, and real-time solutions.

Analog monolayer SWCNTs-based memristive 2D structure for energy-efficient deep learning in spiking neural networks

Advances in materials science and memory devices work in tandem for the evolution of Artificial Intelligence systems. Energy-efficient computation is the ultimate goal of emerging memristor technology, in which the storage and computation can be done in the same memory crossbar. In this work, an analog memristor device is fabricated utilizing the unique characteristics of single-wall carbon nanotubes (SWCNTs) to act as the switching medium of the device. Via the planar structure, the memristor device exhibits analog switching ability with high state stability. The device’s conductance and capacitance can be tuned simultaneously, increasing the device's potential and broadening its applications' horizons. The multi-state storage capability and long-term memory are the key factors that make the device a promising candidate for bio-inspired computing applications. As a demonstrator, the fabricated memristor is deployed in spiking neural networks (SNN) to exploit its analog switching feature for energy-efficient classification operation. Results reveal that the computation-in-memory implementation performs Vector Matrix Multiplication with 95% inference accuracy and few femtojoules per spike energy efficiency. The memristor device presented in this work opens new insights towards utilizing the outstanding features of SWCNTs for efficient analog computation in deep learning systems.

Compact optimized deep learning model for edge: a review

<p>Most real-time computer vision applications, such as pedestrian detection, augmented reality, and virtual reality, heavily rely on convolutional neural networks (CNN) for real-time decision support. In addition, edge intelligence is becoming necessary for low-latency real-time applications to process the data at the source device. Therefore, processing massive amounts of data impact memory footprint, prediction time, and energy consumption, essential performance metrics in machine learning based internet of things (IoT) edge clusters. However, deploying deeper, dense, and hefty weighted CNN models on resource-constraint embedded systems and limited edge computing resources, such as memory, and battery constraints, poses significant challenges in developing the compact optimized model. Reducing the energy consumption in edge IoT networks is possible by reducing the computation and data transmission between IoT devices and gateway devices. Hence there is a high demand for making energy-efficient deep learning models for deploying on edge devices. Furthermore, recent studies show that smaller compressed models achieve significant performance compared to larger deep-learning models. This review article focuses on state-of-the-art techniques of edge intelligence, and we propose a new research framework for designing a compact optimized deep learning (DL) model deployment on edge devices.</p>

Optimized scheme for cluster head selection and EAMPR routing protocol design in WSN

Wireless sensor networks (WSN) is a popularly emerging technology with several opportunities to sustain in various field that require multipurpose sensor nodes, less energy and non-expensive system. But in the WSN, the radio transmission needs high amount of energy and this creates the critical problem. Hence consumption of energy has to be decreased to extend the network durability. Even though there are so many techniques existing for clustering approach of WSN, they have limitations like increased energy consumption, less delivery rate of data, redundancy and unbalanced network load. Hence, these problems are solved by introducing the energy efficient deep learning techniques for clustering and finding the optimal route. Initially the initialization process of system model is performed with the implementation of energy model. In WSN, energy consumption should be reduced to enhance the QoS and balance the network traffic. Hence clustering method is used to group up the sensor nodes and the optimal cluster head is selected with the proposed technique of hybrid cuckoo search and particle swarm optimization (CSO-PSO). As the CH is chosen, the optimal path of routing data should be found in addition with the procedure of optimization and it is done through the proposed model of Optimization based routing protocol that incorporates the Energy Aware Multi Point Routing (EAMPR) protocol along with the Improved Tuna Search Optimization (ITSO) algorithm. Finally, by the use of ITSO-EAMPR technique the energy consumption will get reduced with the decrease in relative mobility and high stability of nodes would be achieved. The simulations are proceeded and the outcomes are validated. The result obtained is compared with the traditional methods to show the effectiveness of proposed technique. As per the results obtained the proposed ITSO-EAMPR attains maximized PDR and Throughput, higher energy efficiency with extension in lifetime of WSN along with decrease in BER, end-to-end latency as compared to the existing techniques.

Energy-efficient deep learning model for fruit freshness detection

Fruits are usually used as complementary foods because they contain good nutrients such as protein and vitamins. In addition to having good content, it turns out that there are potentially harmful microorganisms contained in fruits caused by decay. Currently, many artificial intelligence (AI) techniques have been proposed in research related to fruit freshness. Deep learning is one of its most prominent types in similar studies. As deep learning typically requires a lot of computation power, it usually consumes a lot of electricity. This is an important concern, especially for agribusiness companies that require AI implementations. Based on these problems, we propose to build a convolutional neural network (CNN) model consisting of six layers to detect fruit freshness and save energy. The CNN model we built uses electrical power ranging from 55 to 73 Watts during the training process and 20 to 27 Watts during the testing process. For accuracy, the result is 98.64%. However, compared to previous studies with the MobileNetV2 model, our model only excels in several aspects, such as recall in fresh banana and fresh oranges, recall and F1-score in Rotten Banana.

Achieving Green AI with Energy-Efficient Deep Learning Using Neuromorphic Computing

research-article Free Access Share on Achieving Green AI with Energy-Efficient Deep Learning Using Neuromorphic Computing Authors: Tao Luo Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (ASTAR), in Singapore Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (ASTAR), in SingaporeSearch about this author , Weng-Fai Wong National University of Singapore (NUS), Singapore National University of Singapore (NUS), SingaporeSearch about this author , Rick Siow Mong Goh Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (ASTAR), in Singapore Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (ASTAR), in SingaporeSearch about this author , Anh Tuan Do Institute of Microelectronics (IME), Agency for Science, Technology and Research (ASTAR), in Singapore Institute of Microelectronics (IME), Agency for Science, Technology and Research (ASTAR), in SingaporeSearch about this author , Zhixian Chen Institute of Microelectronics (IME), Agency for Science, Technology and Research (ASTAR), in Singapore Institute of Microelectronics (IME), Agency for Science, Technology and Research (ASTAR), in SingaporeSearch about this author , Haizhou Li National University of Singapore (NUS), Singapore National University of Singapore (NUS), SingaporeSearch about this author , Wenyu Jiang Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (ASTAR), in Singapore Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (ASTAR), in SingaporeSearch about this author , Weiyun Yau Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (ASTAR), in Singapore Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (ASTAR), in SingaporeSearch about this author Authors Info & Claims Communications of the ACMVolume 66Issue 7July 2023pp 52–57https://doi.org/10.1145/3588591Published:22 June 2023Publication History 0citation0DownloadsMetricsTotal Citations0Total Downloads0Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

DLA-E: a deep learning accelerator for endoscopic images classification

The super power of deep learning in image classification problems have become very popular and applicable in many areas like medical sciences. Some of the medical applications are real-time and may be implemented in embedded devices. In these cases, achieving the highest level of accuracy is not the only concern. Computation runtime and power consumption are also considered as the most important performance indicators. These parameters are mainly evaluated in hardware design phase. In this research, an energy efficient deep learning accelerator for endoscopic images classification (DLA-E) is proposed. This accelerator can be implemented in the future endoscopic imaging equipments for helping medical specialists during endoscopy or colonoscopy in order of making faster and more accurate decisions. The proposed DLA-E consists of 256 processing elements with 1000 bps network on chip bandwidth. Based on the simulation results of this research, the best dataflow for this accelerator based on MobileNet v2 is kcp_ws from the weight stationary (WS) family. Total energy consumption and total runtime of this accelerator on the investigated dataset is 4.56 × 109 MAC (multiplier–accumulator) energy and 1.73 × 107 cycles respectively, which is the best result in comparison to other combinations of CNNs and dataflows.

X-NVDLA: Runtime Accuracy Configurable NVDLA Based on Applying Voltage Overscaling to Computing and Memory Units

This paper investigates a runtime accuracy reconfigurable implementation of an energy efficient deep learning accelerator. It is based on voltage overscaling (VOS) technique which provides dynamic adjustment of approximation level as well as improving lifetime/reliability of the accelerator. The technique is applied to both computing and memory units where based on the minimum required accuracy, the applied voltage is adjusted during the runtime. The implementation of the network is performed using NVDLA which is an open-source CNN (Convolutional Neural Network) accelerator. The approximation is applied to both the accelerator MAC array employed for the required network computations and to the accelerator SRAM memory utilized for storing the inputs (images), weights, and activation data. To control the accuracy degradation of the approximate accelerator (called X-NVDLA), the reduced voltage is applied only to LSB bits of the MAC array and SRAM unit. To assess the efficacy of the proposed energy efficient accelerator, the energy-accuracy characteristics of X-NVDLA when running LeNet-5 and ResNet-50 networks with 8-bit (integer) precision are investigated. In addition, the characteristic of bias temperature instability (BTI), as one of the lifetime deteriorating phenomena is determined. The study includes energy improvement versus accuracy degradation as a function of overscaled voltages and number of approximate the least significant bits using a 15nm FinFET technology.

An energy efficient deep learning model for intrusion detection in smart healthcare with optimal feature selection mechanism

In healthcare (HC), Internet of Things (IoT) integrated cloud computing provides various features and real-time applications. However, owing to the nature of IoT architecture, their types, various modes of communication and the density of data transformed in the network, security is currently a critical issue in the IoT healthcare (IoT-HC) field. This paper proposes a deep learning (DL) model, namely Adaptive Swish-based Deep Multi-Layer Perceptron (ASDMLP) that identifies the intrusions or attacks in the IoT healthcare (IoT-HC) platform. The proposed model starts by clustering the patients’ sensor devices in the network using the Probability-based Fuzzy C-Means (PFCM) model. After clustering the devices, the cluster heads (CHs) among the cluster members are selected based on the energy, distance and degree of the sensor devices for aggregating the data sensed by the medical sensor devices. The base station (BS) sends the patient’s data collected by the CHs to the cloud server (CS). At the cloud end, the proposed model implements an IDS by applying training of the DL model in publicly available databases. The DL approach first performs preprocessing of the data and then selects optimal features from the dataset using the Opposition and Greedy Levy mutation-based Coyotes Optimization Algorithm (OGCOA). The ASDMLP trains these optimal features for the detection of HC data intrusions. The outcomes confirm that the proposed approach works well on real-time IoT datasets for intrusion detection (ID) without compromising the energy consumption (EC) and lifespan of the network.

Lightweight and Energy-Efficient Deep Learning Accelerator for Real-Time Object Detection on Edge Devices.

Tiny machine learning (TinyML) has become an emerging field according to the rapid growth in the area of the internet of things (IoT). However, most deep learning algorithms are too complex, require a lot of memory to store data, and consume an enormous amount of energy for calculation/data movement; therefore, the algorithms are not suitable for IoT devices such as various sensors and imaging systems. Furthermore, typical hardware accelerators cannot be embedded in these resource-constrained edge devices, and they are difficult to drive real-time inference processing as well. To perform the real-time processing on these battery-operated devices, deep learning models should be compact and hardware-optimized, and hardware accelerator designs also have to be lightweight and consume extremely low energy. Therefore, we present an optimized network model through model simplification and compression for the hardware to be implemented, and propose a hardware architecture for a lightweight and energy-efficient deep learning accelerator. The experimental results demonstrate that our optimized model successfully performs object detection, and the proposed hardware design achieves 1.25× and 4.27× smaller logic and BRAM size, respectively, and its energy consumption is approximately 10.37× lower than previous similar works with 43.95 fps as a real-time process under an operating frequency of 100 MHz on a Xilinx ZC702 FPGA.

Noise-resilient and high-speed deep learning with coherent silicon photonics

The explosive growth of deep learning applications has triggered a new era in computing hardware, targeting the efficient deployment of multiply-and-accumulate operations. In this realm, integrated photonics have come to the foreground as a promising energy efficient deep learning technology platform for enabling ultra-high compute rates. However, despite integrated photonic neural network layouts have already penetrated successfully the deep learning era, their compute rate and noise-related characteristics are still far beyond their promise for high-speed photonic engines. Herein, we demonstrate experimentally a noise-resilient deep learning coherent photonic neural network layout that operates at 10GMAC/sec/axon compute rates and follows a noise-resilient training model. The coherent photonic neural network has been fabricated as a silicon photonic chip and its MNIST classification performance was experimentally evaluated to support accuracy values of >99% and >98% at 5 and 10GMAC/sec/axon, respectively, offering 6× higher on-chip compute rates and >7% accuracy improvement over state-of-the-art coherent implementations.

Variable-Precision Approximate Floating-Point Multiplier for Efficient Deep Learning Computation

In this brief, a variable-precision approximate floating-point multiplier is proposed for energy efficient deep learning computation. The proposed architecture supports approximate multiplication with BFloat16 format. As the input and output activations of deep learning models usually follow normal distribution, inspired by the posit format, for numbers with different values, different precisions can be applied to represent them. In the proposed architecture, posit encoding is used to change the level of approximation, and the precision of the computation is controlled by the value of product exponent. For large exponent, smaller precision multiplication is applied to mantissa and for small exponent, higher precision computation is applied. Truncation is used as approximate method in the proposed design while the number of bit positions to be truncated is controlled by the values of the product exponent. The proposed design can achieve 19% area reduction and 42% power reduction compared to the normal BFloat16 multiplier. When applying the proposed multiplier in deep learning computation, almost the same accuracy as that of normal BFloat16 multiplier can be achieved.

Correction to: Energy Efficient Deep Learning Inference Embedded on FPGA for Sleep Apnea Detection

Correction to: Energy Efficient Deep Learning Inference Embedded on FPGA for Sleep Apnea Detection

Optical coherent dot-product chip for sophisticated deep learning regression

Optical implementations of neural networks (ONNs) herald the next-generation high-speed and energy-efficient deep learning computing by harnessing the technical advantages of large bandwidth and high parallelism of optics. However, due to the problems of the incomplete numerical domain, limited hardware scale, or inadequate numerical accuracy, the majority of existing ONNs were studied for basic classification tasks. Given that regression is a fundamental form of deep learning and accounts for a large part of current artificial intelligence applications, it is necessary to master deep learning regression for further development and deployment of ONNs. Here, we demonstrate a silicon-based optical coherent dot-product chip (OCDC) capable of completing deep learning regression tasks. The OCDC adopts optical fields to carry out operations in the complete real-value domain instead of in only the positive domain. Via reusing, a single chip conducts matrix multiplications and convolutions in neural networks of any complexity. Also, hardware deviations are compensated via in-situ backpropagation control provided the simplicity of chip architecture. Therefore, the OCDC meets the requirements for sophisticated regression tasks and we successfully demonstrate a representative neural network, the AUTOMAP (a cutting-edge neural network model for image reconstruction). The quality of reconstructed images by the OCDC and a 32-bit digital computer is comparable. To the best of our knowledge, there is no precedent of performing such state-of-the-art regression tasks on ONN chips. It is anticipated that the OCDC can promote the novel accomplishment of ONNs in modern AI applications including autonomous driving, natural language processing, and scientific study.

A Lightweight Framework for Human Activity Recognition on Wearable Devices

Human Activity Recognition (HAR) is the automatic detection and understanding of human motion behavior based on data extracted from video camera, ambient sensors or wearable sensors which particularly has recently attracted increased attention from both researchers and industry. However, for running practical HAR systems on wearable devices, there are some requirements such as design and development of small, lightweight, powerful, and low-cost smart sensors. In this context, data must be continuously collected, and edge computing is a viable solution, which is an energy-efficient technique, offering real-time response and privacy requirements for HAR applications. Rather than sending data to the cloud, edge computing is a local process that minimizes the data transmission time and responds with low latency. Recently, HAR system designers have adopted deep learning techniques inspired by their outstanding performance in many application areas and achieved relevant gain in activity recognition performance, however, these techniques were not demonstrated suitable for running on resource constrained devices. Thus, designing energy-efficient deep learning models is critical for realizing efficient HAR for mobile applications. In this work, we present a lightweight framework for the deployment of low-power but accurate HAR systems for these devices. We also implement and run the system in a microcontroller and analyze computational cost and energy consumption, and how different system configurations and deep learning model complexity influence on that.

Optical Convolutional Neural Network With WDM-Based Optical Patching and Microring Weighting Banks

We propose an optical convolutional neural network (OCNN) architecture for high-speed and energy-efficient deep learning accelerators. The WDM-based optical patching scheme (WDM-OPS) is adopted as the data-feeding structure for its superior energy efficiency and the microring banks are used for the large-scale weighting and summing (the computing core). We thoroughly investigate the performance (including prediction accuracy, speed, and energy efficiency) of this architecture in different system defects. The results indicate that, the prediction accuracy of OCNN can reach 97% in the MNIST dataset with a computing speed of over 100 TMAC/s on condition of achievable low insertion loss. It is also observed that the WDM-OPS notably reduces the energy consumption of the electro-optic modulation and thus the OCNN becomes an exceptionally energy-efficient architecture among several well-known optical architectures. In the evaluations, instead of merely considering the computing core, we take the holistic optical system including lasers, electro-optic modulators, data preprocessing, photodetection and transimpedance amplification into consideration. Therefore, this work provides a potential guide for the systematic implementation of the OCNN architecture.

Initializing photonic feed-forward neural networks using auxiliary tasks

Photonics is among the most promising emerging technologies for providing fast and energy-efficient Deep Learning (DL) implementations. Despite their advantages, these photonic DL accelerators also come with certain important limitations. For example, the majority of existing photonic accelerators do not currently support many of the activation functions that are commonly used in DL, such as the ReLU activation function. Instead, sinusoidal and sigmoidal nonlinearities are usually employed, rendering the training process unstable and difficult to tune, mainly due to vanishing gradient phenomena. Thus, photonic DL models usually require carefully fine-tuning all their training hyper-parameters in order to ensure that the training process will proceed smoothly. Despite the recent advances in initialization schemes, as well as in optimization algorithms, training photonic DL models is still especially challenging. To overcome these limitations, we propose a novel adaptive initialization method that employs auxiliary tasks to estimate the optimal initialization variance for each layer of a network. The effectiveness of the proposed approach is demonstrated using two different datasets, as well as two recently proposed photonic activation functions and three different initialization methods. Apart from significantly increasing the stability of the training process, the proposed method can be directly used with any photonic activation function, without further requiring any other kind of fine-tuning, as also demonstrated through the conducted experiments.

Accurate deep neural network inference using computational phase-change memory

In-memory computing using resistive memory devices is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to phase-change memory (PCM) devices. We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on CIFAR-10 and a top-1 accuracy of 71.6% on ImageNet benchmarks after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one-day period, where each of the 361,722 synaptic weights is programmed on just two PCM devices organized in a differential configuration.

A Deep Learning Approach for Energy Efficient Computational Offloading in Mobile Edge Computing

Mobile edge computing (MEC) has shown tremendous potential as a means for computationally intensive mobile applications by partially or entirely offloading computations to a nearby server to minimize the energy consumption of user equipment (UE). However, the task of selecting an optimal set of components to offload considering the amount of data transfer as well as the latency in communication is a complex problem. In this paper, we propose a novel energy-efficient deep learning based offloading scheme (EEDOS) to train a deep learning based smart decision-making algorithm that selects an optimal set of application components based on remaining energy of UEs, energy consumption by application components, network conditions, computational load, amount of data transfer, and delays in communication. We formulate the cost function involving all aforementioned factors, obtain the cost for all possible combinations of component offloading policies, select the optimal policies over an exhaustive dataset, and train a deep learning network as an alternative for the extensive computations involved. Simulation results show that our proposed model is promising in terms of accuracy and energy consumption of UEs.