Compact Deep Neural Network Research Articles

Energy-Efficient Approximate Edge Inference Systems

The rapid proliferation of the Internet of Things and the dramatic resurgence of artificial intelligence based application workloads have led to immense interest in performing inference on energy-constrained edge devices. Approximate computing (a design paradigm that trades off a small degradation in application quality for disproportionate energy savings) is a promising technique to enable energy-efficient inference at the edge. This article introduces the concept of an approximate edge inference system ( AxIS ) and proposes a systematic methodology to perform joint approximations between different subsystems in a deep neural network (DNN)-based edge inference system, leading to significant energy benefits compared to approximating individual subsystems in isolation. We use a smart camera system that executes various DNN-based image classification and object detection applications to illustrate how the sensor, memory, compute, and communication subsystems can all be approximated synergistically. We demonstrate our proposed methodology using two variants of a smart camera system: (a) Cam Edge , where the DNN is executed locally on the edge device, and (b) Cam Cloud , where the edge device sends the captured image to a remote cloud server that executes the DNN. We have prototyped such an approximate inference system using an Intel Stratix IV GX-based Terasic TR4-230 FPGA development board. Experimental results obtained using six large DNNs and four compact DNNs running image classification applications demonstrate significant energy savings (≈ 1.6× -4.7× for large DNNs and ≈ 1.5× -3.6× for small DNNs), for minimal (<1%) loss in application-level quality. Furthermore, results using four object detection DNNs exhibit energy savings of ≈ 1.5× -5.2× for similar quality loss. Compared to approximating a single subsystem in isolation, AxIS achieves 1.05× -3.25× gains in energy savings for image classification and 1.35× -4.2× gains for object detection on average, for minimal (<1%) application-level quality loss.

High-Throughput, High-Performance Deep Learning-Driven Light Guide Plate Surface Visual Quality Inspection Tailored for Real-World Manufacturing Environments

Light guide plates are essential optical components widely used in a diverse range of applications ranging from medical lighting fixtures to back-lit TV displays. An essential step in the manufacturing of light guide plates is the quality inspection of defects such as scratches, bright/dark spots, and impurities. This is mainly done in industry through manual visual inspection for plate pattern irregularities, which is time-consuming and prone to human error and thus act as a significant barrier to high-throughput production. Advances in deep learning-driven computer vision has led to the exploration of automated visual quality inspection of light guide plates to improve inspection consistency, accuracy, and efficiency. However, given the computational constraints and high-throughput nature of real-world manufacturing environments, the widespread adoption of deep learning-driven visual inspection systems for inspecting light guide plates in real-world manufacturing environments has been greatly limited due to high computational requirements and integration challenges of existing deep learning approaches in research literature. In this work, we introduce a fully-integrated, high-throughput, high-performance deep learning-driven workflow for light guide plate surface visual quality inspection (VQI) tailored for real-world manufacturing environments. To enable automated VQI on the edge computing within the fully-integrated VQI system, a highly compact deep anti-aliased attention condenser neural network (which we name Light-DefectNet) tailored specifically for light guide plate surface defect detection in resource-constrained scenarios was created via machine-driven design exploration with computational and “best-practices” constraints as well as L1 paired classification discrepancy loss. Experiments show that Light-DetectNet achieves a detection accuracy of ∼98.2% on the LGPSDD benchmark while having just 770K parameters (∼33× and ∼6.9× lower than ResNet-50 and EfficientNet-B0, respectively) and ∼93M FLOPs (∼88× and ∼8.4× lower than ResNet-50 and EfficientNet-B0, respectively) and ∼8.8× faster inference speed than EfficientNet-B0 on an embedded ARM processor. As such, the proposed deep learning-driven workflow, integrated with the aforementioned LightDefectNet neural network, is highly suited for high-throughput, high-performance light plate surface VQI within real-world manufacturing environments.

DyNNamic: Dynamically Reshaping, High Data-Reuse Accelerator for Compact DNNs

Convolutional layers dominate the computation and energy costs of Deep Neural Network (DNN) inference. Recent algorithmic works attempt to reduce these bottlenecks via compact DNN structures and model compression. Likewise, state-of-the-art accelerator designs leverage spatiotemporal characteristics of convolutional layers to reduce data movement overhead and improve throughput. Although both are independently effective at reducing latency and energy costs, combining these approaches does not guarantee cumulative improvements due to inefficient mapping. This inefficiency can be attributed to (1) inflexibility of underlying hardware and (2) inherent reduction of data-reuse opportunities of compact DNN structures. To address these issues, we propose a dynamically reshaping, high data-reuse PE array accelerator, namely <i>DyNNamic</i> . DyNNamic leverages kernel-wise filter decomposition to partition the convolution operation into two compact stages: Shared Kernels Convolution (SKC) and Weighted Accumulation (WA). Because both stages have vastly different dimensions, DyNNamic reshapes its PE array to effectively map the algorithm to the architecture. The architecture then exploits data-reuse opportunities created by the SKC stage, further reducing data movement with negligible overhead. We evaluate our approach on various representative networks and compare against state-of-the-art accelerators. On average, DyNNamic outperforms DianNao by <inline-formula><tex-math notation="LaTeX">$8.4\times$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$12.3\times$</tex-math></inline-formula> in terms of inference energy and latency, respectively.

Training Compact DNNs with [formula omitted] Regularization

Deep neural network(DNN) has achieved unprecedented success in many fields. However, its large model parameters which bring a great burden on storage and calculation hinder the development and application of DNNs. It is worthy of compressing the model to reduce the complexity of the DNN. Sparsity-inducing regularizer is one of the most common tools for compression. In this paper, we propose utilizing the ℓ1/2 quasi-norm to zero out weights of neural networks and compressing the networks automatically during the learning process. To our knowledge, it is the first work applying the non-Lipschitz continuous regularizer for the compression of DNNs. The resulting sparse optimization problem is solved by stochastic proximal gradient algorithm. For further convenience of calculation, an approximation of the threshold-form solution to the proximal operator with ℓ1/2 is given at the same time. Extensive experiments with various datasets and baselines demonstrate the advantages of our new method.

Plug and play augmented HQS: Convergence analysis and its application in MRI reconstruction

Sparse recovery in the context of the inverse problem has become an enormously popular technique in reconstructing various degraded images in various applications. One of the well-known techniques in modularizing these particular inverse problems is the Plug and Play Half-Quadratic-Splitting (PnP-HQS). This method has been demonstrated to achieve good results in the literature; however, it is still too plain to be fully exploited for reconstruction. In this regard, we introduce an augmented version of this technique dubbed “PnP-AugHQS” to efficiently utilize its capabilities in image reconstruction. We provide a comprehensive convergence analysis of the proposed algorithm to ensure its effectiveness in image reconstruction. We then exploit the new parameters to further modify the procedure of the conventional PnP in order to account for the noise in the measurement. The PnP-AugHQS is equipped with a compact deep Convolutional Neural Network denoising regularization to maximize its power in image recovery. As a special case, we further modified the algorithm to be used in the application of MRI reconstruction. Various experiments evaluated on the proposed algorithm showed the superiority of the PnP-AugHQS compared to the PnP-HQS and other state-of-the-art techniques in MRI reconstruction.

An efficient EEGNet processor design for portable EEG-Based BCIs

Brain-computer interfaces (BCIs) could manipulate devices through non-invasive electroencephalogram (EEG) signal. A compact deep convolutional neural network model, EEGNet, and its hardware implementation, has been developed for obtaining generalization towards different BCI paradigms. However, the exist design consumes huge amount of area and power resources. BCI devices are used dynamically on portable devices in most scenario, which has restricted limitations on resource consumption. Hence, in this paper, we present an efficient EEGNet processor design to accomplish accurate and real-time EEG classification. The proposed design reuses the processing elements in a convolutional channel, and simplifies the exponential functions in activation layer and classifier layer to improve efficiency. The experimental results of FPGA implementation show that the proposed design consumes 2.54% in terms of area, and 3.66% in terms of power consumption comparing to the previous work, which is more suitable for portable BCI devices.

A Fast and Compact Deep Gabor Network for Micro-Doppler Signal Processing and Human Motion Classification

With the advances in radar technology, there is an increasing interest in radar-based applications for automated surveillance, tracking, and recognition. Radars capture the Doppler-induced frequency shifts, which can be utilized to recognize the small motions by vibrating or rotating parts of the target. This paper introduces a fast and compact deep Gabor neural network for human motion classification using backscattered signals from a continuous-wave Doppler radar. The joint time-frequency representation produced by the S-method is employed to depict the micro-Doppler signature of a walking person. We introduce a new Gabor layer as a generic feature extractor for designing compact neural architectures. Each Gabor layer consists of several steerable Gabor filters that can be trained to extract the salient micro-Doppler features and improve computation efficiency. For human motion classification from a distance, the Deep Gabor Network (DGN) is trained in an end-to-end manner to process the local patches in the time-frequency representation of the radar signal. A Bayesian optimization technique is employed to select the optimal network hyperparameters. The experimental results on a real radar dataset show that the proposed method achieves competitive classification rates compared to several existing approaches while having a significantly smaller model size and shorter prediction time.

LEARNING COMPACT DNN MODELS FOR BEHAVIOR PREDICTION FROM NEURAL ACTIVITY OF CALCIUM IMAGING.

In this paper, we develop methods for efficient and accurate information extraction from calcium-imaging-based neural signals. The particular form of information extraction we investigate involves predicting behavior variables linked to animals from which the calcium imaging signals are acquired. More specifically, we develop algorithms to systematically generate compact deep neural network (DNN) models for accurate and efficient calcium-imaging-based predictive modeling. We also develop a software tool, called NeuroGRS, to apply the proposed methods for compact DNN derivation with a high degree of automation. GRS stands for Greedy inter-layer order with Random Selection of intra-layer units, which describes the central algorithm developed in this work for deriving compact DNN structures. Through extensive experiments using NeuroGRS and calcium imaging data, we demonstrate that our methods enable highly streamlined information extraction from calcium images of the brain with minimal loss in accuracy compared to much more computationally expensive approaches.

Deep sparse transfer learning for remote smart tongue diagnosis.

People are exploring new ideas based on artificial intelligent infrastructures for immediate processing, in which the main obstacles of widely-deploying deep methods are the huge volume of neural network and the lack of training data. To meet the high computing and low latency requirements in modeling remote smart tongue diagnosis with edge computing, an efficient and compact deep neural network design is necessary, while overcoming the vast challenge on modeling its intrinsic diagnosis patterns with the lack of clinical data. To address this challenge, a deep transfer learning model is proposed for the effective tongue diagnosis, based on the proposed similar-sparse domain adaptation (SSDA) scheme. Concretely, a transfer strategy of similar data is introduced to efficiently transfer necessary knowledge, overcoming the insufficiency of clinical tongue images. Then, to generate simplified structure, the network is pruned with transferability remained in domain adaptation. Finally, a compact model combined with two sparse networks is designed to match limited edge device. Extensive experiments are conducted on the real clinical dataset. The proposed model can use fewer training data samples and parameters to produce competitive results with less power and memory consumptions, making it possible to widely run smart tongue diagnosis on low-performance infrastructures.

Predicting brain age with complex networks: From adolescence to adulthood

In recent years, several studies have demonstrated that machine learning and deep learning systems can be very useful to accurately predict brain age. In this work, we propose a novel approach based on complex networks using 1016 T1-weighted MRI brain scans (in the age range 7−64years). We introduce a structural connectivity model of the human brain: MRI scans are divided in rectangular boxes and Pearson’s correlation is measured among them in order to obtain a complex network model. Brain connectivity is then characterized through few and easy-to-interpret centrality measures; finally, brain age is predicted by feeding a compact deep neural network. The proposed approach is accurate, robust and computationally efficient, despite the large and heterogeneous dataset used. Age prediction accuracy, in terms of correlation between predicted and actual age r=0.89and Mean Absolute Error MAE =2.19years, compares favorably with results from state-of-the-art approaches. On an independent test set including 262 subjects, whose scans were acquired with different scanners and protocols we found MAE =2.52. The only imaging analysis steps required in the proposed framework are brain extraction and linear registration, hence robust results are obtained with a low computational cost. In addition, the network model provides a novel insight on aging patterns within the brain and specific information about anatomical districts displaying relevant changes with aging.

CDeepArch: A Compact Deep Neural Network Architecture for Mobile Sensing

Mobile sensing is a promising sensing paradigm in the era of Internet of Things (IoT) that utilizes mobile device sensors to collect sensory data about sensing targets and further applies learning techniques to recognize the sensed targets to correct classes or categories. Due to the recent great success of deep learning, an emerging trend is to adopt deep learning in this recognition process, while we find an overlooked yet crucial issue to be solved in this paper — The size of deep learning models should be sufficiently large for reliably classifying various types of recognition targets, while the achieved processing delay may fail to satisfy the stringent latency requirement from applications. If we blindly shrink the deep learning model for acceleration, the performance cannot be guaranteed. To cope with this challenge, this paper presents a compact deep neural network architecture, namely cDeepArch . The key idea of the cDeepArch design is to decompose the entire recognition task into two lightweight sub-problems: context recognition and the context-oriented target recognitions. This decomposition essentially utilizes the adequate storage to trade for the CPU and memory resource consumptions during execution. In addition, we further formulate the execution latency for decomposed deep learning models and propose a set of enhancement techniques, so that system performance and resource consumption can be quantitatively balanced. We implement a cDeepArch prototype system and conduct extensive experiments. The result shows that cDeepArch achieves excellent recognition performance and the execution latency is also lightweight.

A Multiobjective Learning and Ensembling Approach to High-Performance Speech Enhancement With Compact Neural Network Architectures

In this study, we propose a novel deep neural network (DNN) architecture for speech enhancement (SE) via a multiobjective learning and ensembling (MOLE) framework to achieve a compact and lowlatency design, while maintaining good performance in quality evaluations. MOLE follows the boosting concept when combining weak models into a strong classifier and consists of two compact DNNs. The first, called the multiobjective learning DNN (MOL-DNN), takes multiple features, such as log-power spectra (LPS), mel-frequency cepstral coefficients (MFCCs) and Gammatone frequency cepstral coefficients (GFCCs) to predict a multiobjective set that includes clean speech feature, dynamic noise feature, and ideal ratio mask (IRM). The second, called the multiobjective ensembling DNN (MOE-DNN), takes the learned features from MOL-DNN as inputs and separately predicts clean LPS and IRM, clean MFCC and IRM, and clean GFCC and IRM using three sets of weak regression functions. Finally, a postprocessing operation can be applied to the estimated clean features by leveraging the multiple targets learned from both the MOL-DNN and the MOE-DNN. On speech corrupted by 15 noise types not seen in model training the SE results show that the MOLE approach, which features a small model size and low run-time latency, can achieve consistent improvements over both DNN- and long short-term memory (LSTM)-based techniques in terms of all the objective metrics evaluated in this study for all three cases (the input contexts contain 1-frame, 4-frame and 7-frame instances). The 1-frame MOLE-based SE system outperforms the DNN-based SE system with a 7-frame input expansion at a 3-frame delay and also achieves better performance than the LSTM-based SE system with 4-frame, no delay expansion by including only 3 previous frames, and with 170 times less processing latency.

MicronNet: A Highly Compact Deep Convolutional Neural Network Architecture for Real-Time Embedded Traffic Sign Classification

Traffic sign recognition is a very important computer vision task for a number of real-world applications such as intelligent transportation surveillance and analysis. While deep neural networks have been demonstrated in recent years to provide the state-of-the-art performance traffic sign recognition, a key challenge for enabling the widespread deployment of deep neural networks for embedded traffic sign recognition is the high computational and memory requirements of such networks. As a consequence, there are significant benefits in investigating compact deep neural network architectures for traffic sign recognition that are better suited for embedded devices. In this paper, we introduce MicronNet, a highly compact deep convolutional neural network for real-time embedded traffic sign recognition designed based on macroarchitecture design principles (e.g., spectral macroarchitecture augmentation, parameter precision optimization, etc.) as well as numerical microarchitecture optimization strategies. The resulting overall architecture of MicronNet is thus designed with as few parameters and computations as possible while maintaining recognition performance, leading to optimized information density of the proposed network. The resulting MicronNet possesses a model size of just ~1 MB and ~5 10 000 parameters ( $\sim 27\times $ fewer parameters than state-of-the-art) while still achieving a human performance level top-1 accuracy of 98.9% on the German traffic sign recognition benchmark. Furthermore, the MicronNet requires just ~10 million multiply-accumulate operations to perform inference, and has a time-to-compute of just 32.19 ms on a Cortex-A53 high efficiency processor. These experimental results show that the highly compact, optimized deep neural network architectures can be designed for real-time traffic sign recognition that are well-suited for embedded scenarios.