Lightweight CNN Model Research Articles

Automatic Fingerprint Classification Using Deep Learning Technology (DeepFKTNet)

Fingerprints are gaining in popularity, and fingerprint datasets are becoming increasingly large. They are often captured utilizing a variety of sensors embedded in smart devices such as mobile phones and personal computers. One of the primary issues with fingerprint recognition systems is their high processing complexity, which is exacerbated when they are gathered using several sensors. One way to address this issue is to categorize fingerprints in a database to condense the search space. Deep learning is effective in designing robust fingerprint classification methods. However, designing the architecture of a CNN model is a laborious and time-consuming task. We proposed a technique for automatically determining the architecture of a CNN model adaptive to fingerprint classification; it automatically determines the number of filters and the layers using Fukunaga–Koontz transform and the ratio of the between-class scatter to within-class scatter. It helps to design lightweight CNN models, which are efficient and speed up the fingerprint recognition process. The method was evaluated two public-domain benchmark datasets FingerPass and FVC2004 benchmark datasets, which contain noisy, low-quality fingerprints obtained using live scan devices and cross-sensor fingerprints. The designed models outperform the well-known pre-trained models and the state-of-the-art fingerprint classification techniques.

Performance Analysis of CNN Models for Mobile Device Eye Tracking with Edge Computing

Eye-tracking is a technique used for determining where users are looking and how long they keep their gaze fixed on a particular location. Developments in mobile technology have made mobile applications pervasive; however, eye tracking on mobile devices is still uncommon. This paper proposes a mobile edge computing architecture for eye tracking. We evaluate four lightweight CNN models (LeNet-5, AlexNet, MobileNet, and ShuffleNet) for gaze estimation on mobile devices using a publicly available dataset called GazeCapture. In order to analyse the feasibility of different inference modes such as on-device, edge-based and cloud-based, we conduct an empirical measurement study to quantify inference time, communication time, and resource consumption in these inference modes. Our analysis indicates that while cloud-based inference provides faster predictions, the communication time between the mobile device and the cloud introduces significant latency into the application. This effectively eliminates the ability to perform real-time eye tracking via cloud inference. Furthermore, our findings show that on-device inference performance is limited by energy and memory consumption, making it unsuitable to provide a high-quality user experience. Additionally, we demonstrated that edge-based inference results in a reasonable response time, memory usage, and energy consumption for eye-tracking applications on mobile devices.

An Efficient and Lightweight CNN Model With Soft Quantification for Ship Detection in SAR Images

Convolutional neural networks have been widely used for synthetic aperture radar (SAR) target detection. Typical methods based on convolutional neural network have obtained favorable detection accuracy at the cost of high model complexity, and thus are difficult to be directly applied to real-time satellites on board as well as maritime rescue. To deal with this problem, this paper proposes an efficient and lightweight target detection network incorporating soft quantization. Firstly, to compensate for the lack of accuracy caused by lightweight networks, a feature fusion module called split bidirectional feature pyramid network is proposed to alleviate the interference of complex background on SAR images. Meanwhile, to adapt the lightweight network and the feature fusion module, a linear transformation module is presented to enhance the linear representation of the model via learnable parameters. Eventually, to make the model size smaller, a soft quantization algorithm is proposed to reduce the accuracy degradation caused by quantization errors. We validate the robustness of the model in several publicly available datasets. Experimental results show that our model achieves 97.0% detection accuracy on SAR ship detection dataset, with a 0.9% accuracy improvement compared to mainstream methods using less than 15x the number of parameters and less than 6x the number of flops.

Learning Lightweight Low-Light Enhancement Network Using Pseudo Well-Exposed Images

Recently, there has been growing attention on deep learning-based low-light image enhancement algorithms. With this interest, various synthetic low-light image datasets have been released publicly. However, real-world low-light and well-exposed image pair datasets are still lacking. In this paper, we propose a real-world low-light image dataset and a practical lightweight low-light image enhancement network. In order to construct a large-scale real-world low-light dataset, we have not only captured under-exposed images by ourselves but also collected under-exposed images from the Internet. Then, we produce pseudo well-exposed images for each low-light image. Using pairs of a real-world low-light image and a pseudo well-exposed image, we present a lightweight deep CNN model through knowledge distillation. Experimental results demonstrate the effectiveness and practicality of the proposed method on various datasets.

Transfer Learning-Based Lightweight Cnn Model for Recognition of Pest in Citrus

Transfer Learning-Based Lightweight Cnn Model for Recognition of Pest in Citrus

Abnormal Activity Recognition from Surveillance Videos Using Convolutional Neural Network.

Background and motivation: Every year, millions of Muslims worldwide come to Mecca to perform the Hajj. In order to maintain the security of the pilgrims, the Saudi government has installed about 5000 closed circuit television (CCTV) cameras to monitor crowd activity efficiently. Problem: As a result, these cameras generate an enormous amount of visual data through manual or offline monitoring, requiring numerous human resources for efficient tracking. Therefore, there is an urgent need to develop an intelligent and automatic system in order to efficiently monitor crowds and identify abnormal activity. Method: The existing method is incapable of extracting discriminative features from surveillance videos as pre-trained weights of different architectures were used. This paper develops a lightweight approach for accurately identifying violent activity in surveillance environments. As the first step of the proposed framework, a lightweight CNN model is trained on our own pilgrim’s dataset to detect pilgrims from the surveillance cameras. These preprocessed salient frames are passed to a lightweight CNN model for spatial features extraction in the second step. In the third step, a Long Short Term Memory network (LSTM) is developed to extract temporal features. Finally, in the last step, in the case of violent activity or accidents, the proposed system will generate an alarm in real time to inform law enforcement agencies to take appropriate action, thus helping to avoid accidents and stampedes. Results: We have conducted multiple experiments on two publicly available violent activity datasets, such as Surveillance Fight and Hockey Fight datasets; our proposed model achieved accuracies of 81.05 and 98.00, respectively.

MEAN-SSD: A novel real-time detector for apple leaf diseases using improved light-weight convolutional neural networks

Alternaria blotch, Brown spot, Mosaic, Grey spot, and Rust are 5 common apple leaf diseases that severely impact apple production and quality. At present, although many CNN methods have been proposed for apple leaf diseases, there are still lack of apple leaf disease detection models that can be applied on mobile devices, which limits their application in practical production. This paper proposes a light-weight CNN model that can be deployed on mobile devices to detect apple leaf diseases in real time. First, a dataset of apple leaf diseases composed of simple background images and complex background images, which is called AppleDisease5, is constructed via data augmentation technology and data annotation technology. Then a basic module called MEAN block(Mobile End AppleNet block) is proposed to increase the detection speed and reduce model’s size by reconstructing the common 3×3 convolution. Meanwhile, the Apple-Inception module is built by introducing GoogLeNet’s Inception module and replacing all 3×3 convolution kernels with MEAN block in Inception. Finally, a new apple leaf disease detection model, MEAN-SSD(Mobile End AppleNet based SSD algorithm), is built by using the MEAN block and Apple-Inception module. The experiment results show that MEAN-SSD can achieve the detection performance of 83.12% mAP and a speed of 12.53 FPS, which illustrates that the novel MEAN-SSD model can efficiently and accurately detect 5 common apple leaf diseases on mobile devices.

SCNN: Scalogram-based convolutional neural network to detect obstructive sleep apnea using single-lead electrocardiogram signals

Sleep apnea is a common symptomatic disease affecting nearly 1 billion people around the world. The gold standard approach for determining the severity of sleep apnea is full-night polysomnography conducted in the laboratory, which is very costly and cumbersome. In this work, we propose a novel scalogram-based convolutional neural network (SCNN) to detect obstructive sleep apnea (OSA) using single-lead electrocardiogram (ECG) signals. Firstly, we use continuous wavelet transform (CWT) to convert ECG signals into conventional scalograms. In parallel, we also apply empirical mode decomposition (EMD) to the signals to find correlated intrinsic mode functions (IMFs) and then apply CWT on the IMFs to obtain hybrid scalograms. Finally, we train a lightweight CNN model on these scalograms to extract deep features for OSA detection. Experiments on the benchmark Apnea-ECG dataset demonstrate that our proposed model results in an accuracy of 94.30%, sensitivity 94.30%, specificity 94.51%, and F1-score 95.85% in per-segment classification. Our model also achieves an accuracy of 81.86%, sensitivity 71.62%, specificity 86.05%, and F1-score 69.63% for UCDDB dataset. Furthermore, our model achieves an accuracy of 100.00% in per-recording classification for Apnea-ECG dataset. The experimental results outperform the existing OSA detection approaches using ECG signals.

Progressive Mimic Learning: A new perspective to train lightweight CNN models

Knowledge distillation (KD) builds a lightweight Student Model (SM) and trains it to approximate a large Teacher Model (TM) by exploring knowledge learned by the TM, which shows effectiveness to train lightweight CNN models. However, training a small SM to achieve better performance remains a challenging problem. Recent researches on human learning behaviors show that both the knowledge from teachers and the knowledge learning processes of teachers are significant for students. Inspired by this characteristic, in this paper, we propose a new perspective, called Progressive Mimic Learning (PML), to train lightweight CNN models by mimicking the learning trajectory of the TM. In order to obtain a more powerful SM, the useful hints in the learning process of the TM are explored. To start with, the TM learning process is divided into multiple stages, and the last state of the TM in each stage is recorded as a landmark. The learning trajectory of the TM is composed of these landmarks. Then, a landmark loss is defined to constrain the SM to progressively mimic the learning process of the TM, by employing landmarks in the learning trajectory as a training hint of the SM. Several experiments are conducted on four benchmark data sets, CIFAR-10, CIFAR-100, Fashion-MNIST, and ImageNet-10, to investigate the performance of the PML. The results show that the PML can make SMs generate more accurate predictions than SMs trained by its counterparts.

A Lightweight Motional Object Behavior Prediction System Harnessing Deep Learning Technology for Embedded ADAS Applications

This paper proposes a lightweight moving object prediction system to detect and recognize pedestrian crossings, vehicles cutting-in, and vehicles ahead applying emergency brakes based on a 3D Convolution network for behavior prediction. The proposed design significantly improves the performance of the conventional 3D convolution network (C3D) adapted to predict the behaviors employing behavior recognition network capable of performing object localization, which is pivotal in detecting the numerous moving objects’ behaviors, combining and verifying the detected objects with the results of the YOLO v3 detection model with that of the proposed C3D model. Since the proposed system is a lightweight CNN model requiring far lesser parameters, it can be efficiently realized on an embedded system for real-time applications. The proposed lightweight C3D model achieves 10 frames per second (FPS) on a NVIDIA Jetson AGX Xavier and yields over 92.8% accuracy in recognizing pedestrian crossing, over 94.3% accuracy in detecting vehicle cutting-in behavior, and over 95% accuracy for vehicles applying emergency brakes.

DcaNAS: efficient convolutional network Design for Desktop CPU platforms

The hardware platform is a significant consideration in efficient CNN model design. Most lightweight networks are based on GPUs and mobile devices. However, they are usually not efficient nor fast enough for desktop CPU platforms. In this paper, we aim to explore the design of highly-efficient convolutional architectures for desktop CPU platforms. To achieve our goal, we first derive a series of CNN model design guidelines for CPU-based devices by comparing different computing platforms. Based on these proposed guidelines, we further present a Desktop CPU-Aware network architecture search (DcaNAS) to search for the optimal network structure with lower CPU latency. By combining automatic search and manual design, our DcaNAS achieves better flexibility and efficiency. On the ImageNet benchmark, we employ DcaNAS to produce two CPU-based lightweight CNN models: DcaNAS-L for higher accuracy and DcaNAS-S for faster speed. On a single CPU core, DcaNAS-L achieves 78.8% Top-1 (94.6% Top-5) accuracy at 13.6 FPS (73.5 ms), and our DcaNAS-S achieves extremely low CPU latency (43.1 ms). The results show that our DcaNAS method can obtain new state-of-the-art CPU-based networks.

CNN Ensemble Approach to Detect COVID-19 from Computed Tomography Chest Images

In January 2020, the World Health Organization declared a global health emergency concerning the spread of a new coronavirus disease, which was later named COVID-19. Early and fast diagnosis and isolation of COVID-19 patients have proven to be instrumental in limiting the spread of the disease. Computed tomography (CT) is a promising imaging method for fast diagnosis of COVID-19. In this study, we develop a unique preprocessing step to resize CT chest images to a fixed size (256 × 256 pixels) that preserves the aspect ratio and reduces image loss. Then, we present a deep learning (DL) method to classify CT chest images based on the light-weight pre-trained EfficientNet-B3 CNN model and ensemble techniques. The proposed method, which we refer to as EfficientNet-B3-GAP-Ensemble, comprises an ensemble of a modified version of the EfficientNet-B3. We build the ensemble using multiple runs and multiple training epochs. We test the EfficientNet-B3-GAP-Ensemble on two common benchmark datasets, i.e., the COVID19-CT and SARS-CoV-2-CT datasets. The proposed method has outperformed state-of-the-art methods for both datasets. For the COVID19-CT dataset, it achieved 88.18% sensitivity, 88.29% precision, 88.18% accuracy, an F1-score of 88.15%, and AUC of 92.10%. With the SARS-CoV-2-CT dataset, we tested the proposed method under different train-test splits, i.e., 20%–80%, 50%–50%, and 80%–20%. For the latter split, the proposed method achieved 99.72% accuracy, 99.80% sensitivity, precision, and F1-scores, and an AUC score of 99.99%.

Distilling the Knowledge of Multiscale Densely Connected Deep Networks in Mechanical Intelligent Diagnosis

At present, deep neural network (DNN) technology is often used in intelligent diagnosis research. However, the huge amount of calculation of DNN makes it difficult to apply in industrial practice. In this paper, an advanced multiscale dense connection deep network MSDC‐NET is designed. A well‐designed multiscale parallel branch module is used in the network. This module can greatly improve the acceptance domain of MSDC‐NET, so as to learn useful information from input samples more effectively. Based on the inspiration of Densely Connected Convolutional Networks, MSDC‐NET designed a similar dense connection technology, so that the model will not have the problem of gradient vanishing because of the deep network. The experimental data of MSDC‐NET on MFPT, SEU, and Pu datasets show that our method has higher performance than other latest technologies. At the same time, we carried out knowledge distillation based on the high‐precision classification level of MSDC‐NET, which makes the diagnosis ability and robustness of the lightweight CNN model improve significantly.

Convolutional Neural Network With Attention Mechanism for SAR Automatic Target Recognition

Synthetic aperture radar automatic target recognition (SAR ATR) is a key technique of remote-sensing image recognition, which has many potential applications in the fields of military surveillance, national defense, civil application, and so on. With the development of science and technology, deep convolutional neural network (DCNN) has been widely applied for SAR ATR. However, it is difficult to use deep learning to train models with limited ray SAR images. To resolve this problem, we proposed an effectively lightweight attention mechanism CNN (AM-CNN) model for SAR ATR. Extensive experimental results on the Moving and Stationary Target Acquisition and Recognition (MSTAR) data set illustrate that the AM-CNN model can achieve a superior recognition performance, and the average recognition accuracy can reach 99.35% on the classification of 10 class targets. Compared with the traditional CNN and the state-of-the-art method, our model is significantly superior to improve performance and efficiency.

Collaborative Image Relevance Learning for Visual Re-Ranking

In content-based image retrieval, the initial retrieval result may be unsatisfactory, which can be refined with visual re-ranking techniques, such as query expansion, geometric verification, etc . In this work, we approach visual re-ranking from a novel perspective. Observing that the contextual similarity of images from a retrieval result list exhibits strong visual relevance, we propose to collaboratively learn the semantic relevance among images for visual re-ranking. In our approach, we represent the image set of a fixed-length retrieval list into a correlation matrix, and learn the relevance of all image pairs simultaneously with a lightweight CNN model. To optimize the CNN model, a weighted MSE loss is defined, which takes into account the sparsity of labels. To find the optimal length of retrieval result list for different queries, we present a query sensitive selection method. We conduct comprehensive experiments on five benchmark datasets, and demonstrate the generality, and effectiveness of the proposed visual re-ranking method.

Extracting road maps from high-resolution satellite imagery using refined DSE-LinkNet

Road detection and extraction have gained momentum in recent past years with crucial applications such as urban planning, autonomous driving, automated map update, providing aid to rescue missions, etc. The current methodologies generate the disconnected road segments, cause boundary loss, and also they are incapable of handling the imbalanced class distribution problems. In this paper, we propose a fully convolutional architecture, named as refined DSE-LinkNet, to extract the connected and precise road maps. We use a pre-trained encoder by combining the layers of the two very efficient and light-weight CNN models: DenseNet and SE-Net that makes the proposed model more expressive with faster convergence. We introduce a new module, Fusion block, in our architecture that enhances its precise localisation as well as classification ability by capturing multilevel as well as multiscale features. To address the imbalanced class distribution problem, a new aggregate loss function is proposed by integrating binary cross-entropy, Jaccard coefficient, and Lovasz sigmoid loss functions. The experiments are performed on a publicly available dataset, DeepGlobe Road Extraction Challenge 2018, to show its efficacy over the D-LinkNet, winner of DeepGlobe Challenge 2018, by achieving IoU of 0.69 with lesser number of parameters and better computational complexity.

MBBNet: An edge IoT computing-based traffic light detection solution for autonomous bus

Abstract Traffic light detection is a key module in the autonomous driving system to enhance the interactions between drivers and unmanned vehicles. In recent studies, deep neural networks are widely used for traffic light detection and resource/power consumption is a major concern for model deployment in vehicular edge devices. This paper proposes a novel light-weight deep CNN model that integrates the multi-backbone of state-of-the-art architectures for the self-driving traffic light detection. The MBBNet (Multi-BackBone Network) consists of three common convolutional backbones, i.e., the normal, residual and highway (DenseNet) convolutional modules. Simple ensemble of those backbones may incur high computational load. Therefore, channel compression is adopted to control the model parameters, while guaranteeing the accuracy for mobile and embedded hardware. Evaluation of a dataset collected from real road conditions demonstrate the robustness of our detection system, and it achieves higher accuracy (accuracy > 0.94 and A v e r a g e _ I O U > 74.05 % ) for self-driving buses. In terms of resource consumption, the trained model size is 1.35 MB, and can process high-resolution images (1280 × 960) at 14 FPS (frames per second) on low-power edge devices.

A lightweight CNN model and its application in intelligent practical teaching evaluation

In this paper, we propose a lightweight CNN model. Firstly, we standardize the existing CNN model structure based on the minimum computing unit, and second we apply a parameter control solution to solve the problem of parameter redundancy in the model. At last we build a lightweight nonaligned CNN model. The experimental results show that the model parameters can be reduced by more than 50% when the test error is almost the same. Through deep learning, the proposed model is applied to the practical teaching system to achieve the intelligent evaluation effect of the practical teaching process, while improve the quality and efficiency of teaching.

A Lightweight CNN Model for Refining Moving Vehicle Detection From Satellite Videos

With the recent development of earth observation technology, the satellites have obtained the ability to capture city-scale videos, which enable potential applications in intelligent traffic management. Because of the broad field-of-view, the moving vehicles in satellite videos are usually composed of only tens of pixels, making it difficult to differentiate true objects from noise and other distractors. In addition, the edges of tall building tops are often mistakenly detected as moving vehicles because of the effects of motion parallax. This article proposed a terse framework that can effectively suppress false targets, achieving high precision and recall. The study involves three parts: 1) An adaptive filtering method is proposed to reduce noise, thus making the detection algorithm more reliable; 2) Several background subtraction models are tested, and the best one is chosen to produce the preliminary detection results at high recall but low accuracy; 3) A lightweight convolutional neural network (LCNN) is designed and trained on a small collection of samples, and then used to eliminate false targets. The experiments and evaluations demonstrate that our method can largely improve the precision at the expense of a slight reduction of recall.

Class-specific discriminant regularization in real-time deep CNN models for binary classification problems

In this paper, we first propose lightweight deep CNN models, capable of effectively operating in real-time on-drone for high-resolution video input, addressing various binary classification problems, e.g. crowd, face, football player, and bicycle detection, in the context of media coverage of specific sport events by drones with increased decisional autonomy. Furthermore, we propose a novel class-specific discriminant regularizer in order to improve the generalization ability of the proposed real-time models, exploiting the nature of the considered two-class problems. The experimental evaluation on four datasets validates the effectiveness of the proposed regularizer in enhancing the generalization ability of the proposed models.