Lightweight Network Architecture Research Articles

Lightweight network learning with Zero-Shot Neural Architecture Search for UAV images

Lightweight Network Architecture is essential for autonomous and intelligent monitoring of Unmanned Aerial Vehicles (UAVs), such as in object detection, image segmentation, and crowd counting applications. The state-of-the-art lightweight network learning based on Neural Architecture Search (NAS) usually costs enormous computation resources. Alternatively, low-performance embedded platforms and high-resolution drone images pose a challenge for lightweight network learning. To alleviate this problem, this paper proposes a new lightweight object detection model, called GhostShuffleNet (GSNet), for UAV images, which is built based on Zero-Shot Neural Architecture Search. This paper also introduces the new components which compose GSNet, namely GhostShuffle units (loosely based on ShuffleNetV2) and the backbone GSmodel-L. Firstly, a lightweight search space is constructed with the GhostShuffle (GS) units to reduce the parameters and floating-point operations (FLOPs). Secondly, the parameters, FLOPs, layers, and memory access cost (MAC) as constraints add to search strategy on a Zero-Shot Neural structure search algorithm, which then searches for an optimal network GSmodel-L. Finally, the optimal GSmodel-L is used as the backbone network and a Ghost-PAN feature fusion module and detection heads are added to complete the design of the lightweight object detection network (GSNet). Extensive experiments are conducted on the VisDrone2019 (14.92%mAP) dataset and the our UAV-OUC-DET (8.38%mAP) dataset demonstrating the efficiency and effectiveness of GSNet. The completed code is available at: https://github.com/yfq-yy/GSNet.

Construction of Edge Computing Platform Using 3D LiDAR and Camera Heterogeneous Sensing Fusion for Front Obstacle Recognition and Distance Measurement System

This research aims to utilise heterogeneous sensor fusion using 3D Light Detection and Ranging (LiDAR) and cameras, combined with an object recognition system and a ranging system, to construct an edge computing platform such that a vehicle equipped with the platform can perform computations offline in real time. This work comprises two main sections: the first is heterogeneous fusion, and the second is obstacle recognition and ranging detection. To achieve heterogeneous sensor fusion, 3D–3D point matching was used to find rigid body transformation between two sensors and finally project the LiDAR 3D point cloud image onto the 2D image. For object recognition, YOLOv4-Tiny was used as the detection network. A lightweight network architecture and high computational speed could be effectively used on edge computing hardware with limited performance. Further, by drawing the bounding box, we could detect the point cloud within the bounding box to estimate the distance to the obstacle. For detecting distance, we conducted experiments in two ways: ‘minimum point in box’ and ‘median point in box’ and compared the results. With heterogeneous sensor fusion, object recognition and the ranging system, detecting the category and distance of obstacles ahead of the vehicle was possible in real time. Furthermore, integrating the edge computing platform architecture enabled moving the entire system offline, making it an independent system that returns results in real time. Finally, a dynamic test was conducted on a road. The experiment showed that the detection speed of YOLOv4-Tiny in the dynamic test was higher than 60 FPS, and the accuracy rate surpassed 70%. Furthermore, the distance detection error of the 3D LiDAR was less than 3 cm, which is sufficiently accurate to be applied to complex environments on roads.

Lightweight Inception Networks for the Recognition and Detection of Rice Plant Diseases

Deep learning is a hot research topic in image processing and computer vision. It has been employed in many fields owing to the excellent performance. However, so far, deep learning methods have been rarely applied in the task of plant disease recognition, except that some existing work focuses on it from a public dataset of images zoomed on plant leaves. The main reasons behind the limited usage of deep learning models in plant disease recognition include: the large volume of deep learning models, which is difficult to deploy on embedded systems, the great computational complexity, which requires large memory overhead, the complex backgrounds of experimental materials, which are hard to train an efficient model, and others. To address these challenges, the paper proposes a valid lightweight network architecture, namely MobInc-Net, to perform the crop disease recognition and detection. In this study, the Inception module was enhanced by replacing the original convolutions with depth-wise and point-wise convolutions, and then the modified Inception (M-Inception) module paired with the pre-trained MobileNet was chosen as the backbone extractor to extract high-quality image features. After that, the completely linked Softmax layer with the actual number of categories and the SSD block were separately added behind the foundation network for classifying and detecting crop disease types. To train an efficient model, the two-stage transfer learning was applied in the model training. Experimental findings show that the proposed method can attain the desired performance with an average recognition accuracy of 99.21% on the public dataset and 97.89% on the local dataset.

MS-DNet: A mobile neural network for plant disease identification

Plant disease identification has recently attracted immense attention from the perspective of food security. Owing to the complexity and diversity of plant diseases, plant disease recognition using image processing techniques is a challenging task. Although the widely applied deep neural networks are promising for recognizing diverse plant diseases, they have certain drawbacks such as their requirement for a large number of parameters, which necessitates a large amount of annotation data for training models. To overcome this challenge, this study proposes a novel lightweight network architecture called MS-DNet for the recognition of crop diseases; the network has a small model size and high computation speed. The proposed method has attained a satisfactory performance in comparative experiments, with the highest average accuracy of 98.32% in recognizing different crop disease types. The experimental results further show that the proposed method outperforms other state-of-the-art methods and also demonstrate its efficiency and extensibility. Our code is available at https://github.com/xtu502/Automatic-crop-disease-identification-under-field-conditions.

Deep learning-based hyperspectral image reconstruction from emulated and real computed tomography imaging spectrometer data

The computed tomography imaging spectrometer (CTIS) is a hyperspectral imaging (HSI) approach where spectral and spatial information of a scene is mixed during the imaging process onto a monochromatic sensor. This mixing is due to a diffractive optical element integrated into the underlying optics and creates a set of diffraction orders. To reconstruct a three-dimensional hyperspectral cube from the CTIS sensor image, iterative algorithms were applied. Unfortunately, such methods are highly sensitive to noise and require high computational time for reconstruction thus hindering their applicability in real-time and high frame-rate applications. To overcome such limitations, we propose a lightweight and efficient deep convolutional neural network for hyperspectral image reconstruction from CTIS sensor images. Compared with classical approaches our model delivers considerably better reconstruction results on synthetic as well as real CTIS images in under 0.17 s, which is over 60 times faster compared with the standard iterative approach. In addition, the reshaping method we have developed enables a lightweight network architecture with over 100 times fewer parameters than previously reported.

Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture

Automated real-time detection of surface defects in manufacturing processes of aluminum alloy strip using a lightweight network architecture

Melanoma image classification based on MobileNetV2 network

Melanoma is one of the most common types of cancer that can lead to high mortality rates if not detected early. Recent studies about deep learning methods show promising results in the development of computer-aided diagnosis for accurate disease detection. Therefore, in this research, we propose a method for classifying melanoma images into benign and malignant classes by using deep learning model and transfer learning. MobileNetV2 network is used as the base model because it has lightweight network architecture. Therefore, the proposed system is promising to be implemented further on mobile devices. Moreover, experimental results on several melanoma datasets show that the proposed method can give high accuracy, up to 85%, compared with other networks. Furthermore, the proposed architecture of the head model, which uses a global average pooling layer followed by two fully-connected layers, gives high accuracy while maintaining the network’s efficiency.

An improved lightweight network architecture for identifying tobacco leaf maturity based on Deep learning

The classification of fresh tobacco leaves during the picking process plays an important role in the subsequent roasting. In this paper, a lightweight convolutional neural network is used to detect the maturity of tobacco leaves quickly. Fresh tobacco leaves in the datasets are divided into 3 categories by the picking position, and each category is divided into 4 maturity levels and finally gets 12 types of tobacco leaves with different maturity. To ensure the lightweight of the model, the new network is based on the MobileNetV2 to establish. By utilizing shortcut operation, the shallow network information is preserved, and network degradation is suppressed. In the tobacco leaf datasets we obtained, the improved network has superior performance and compared with other classic networks, the model size and the number of operations have been reduced.

Lightweight network architecture using difference saliency maps for facial action unit detection

Lightweight network architecture using difference saliency maps for facial action unit detection

A Helmholtz equation solver using unsupervised learning: Application to transcranial ultrasound

Transcranial ultrasound therapy is increasingly used for the non-invasive treatment of brain disorders. However, conventional numerical wave solvers are currently too computationally expensive to be used online during treatments to predict the acoustic field passing through the skull (e.g., to account for subject-specific dose and targeting variations). As a step towards real-time predictions, in the current work, a fast iterative solver for the heterogeneous Helmholtz equation in 2D is developed using a fully-learned optimizer. The lightweight network architecture is based on a modified UNet that includes a learned hidden state. The network is trained using a physics-based loss function and a set of idealized sound speed distributions with fully unsupervised training (no knowledge of the true solution is required). The learned optimizer shows excellent performance on the test set, and is capable of generalization well outside the training examples, including to much larger computational domains, and more complex source and sound speed distributions, for example, those derived from x-ray computed tomography images of the skull.

Training Binary Neural Network without Batch Normalization for Image Super-Resolution

Recently, binary neural network (BNN) based super-resolution (SR) methods have enjoyed initial success in the SR field. However, there is a noticeable performance gap between the binarized model and the full-precision one. Furthermore, the batch normalization (BN) in binary SR networks introduces floating-point calculations, which is unfriendly to low-precision hardwares. Therefore, there is still room for improvement in terms of model performance and efficiency. Focusing on this issue, in this paper, we first explore a novel binary training mechanism based on the feature distribution, allowing us to replace all BN layers with a simple training method. Then, we construct a strong baseline by combining the highlights of recent binarization methods, which already surpasses the state-of-the-arts. Next, to train highly accurate binarized SR model, we also develop a lightweight network architecture and a multi-stage knowledge distillation strategy to enhance the model representation ability. Extensive experiments demonstrate that the proposed method not only presents advantages of lower computation as compared to conventional floating-point networks but outperforms the state-of-the-art binary methods on the standard SR networks.

LiteSCANet: An Efficient Lightweight Network Based on Spectral and Channel-Wise Attention for Hyperspectral Image Classification

Deep learning (DL) algorithms have been demonstrated to have great potential in hyperspectral image (HSI) classification. However, most DL methods mainly focus on improving classification performance, neglecting the computational cost. In order to broaden the application scenarios of DL-based HSI classification methods, it is necessary to develop lightweight and fast models to fit the deployment needs of computationalresource-limited platforms. Taking into account the efficency and accuracy, this paper designs a lightweight network architecture based on spectral and channel-wise attention modules, namely LiteSCANet, for HSI classification. It contains a residual double branch structure, which makes the model effectively extract spectral-spatial features and achieve good performance with fast inference speed and low computational consumption (i.e., floating-point operations, number of parameters and graphics processing unit memory usage). The experiment results on four benchmark data sets show that our proposed model achieves an excellent trade-off between efficiency and accuracy compared with the other six existing networks.

Automatic pancreas segmentation based on lightweight DCNN modules and spatial prior propagation

Nowadays, pancreas segmentation in CT scans has gained more and more attention for computer-assisted diagnosis of inflammation (pancreatitis) or cancer. Despite the thrilling success of deep convolutional neural networks (DCNNs) in automatic pancreas segmentation, the heavy computational complexity of such networks impedes the deployment in clinical applications. To alleviate this issue, this paper establishes a novel end-to-end DCNN model for pursuing high-accurate automatic pancreas segmentation but with low computational cost. Specifically, built upon a simplified FCN architecture, we propose two novel network modules, named as the scale-transferrable feature fusion module (STFFM) and prior propagation module (PPM), respectively, for pancreas segmentation. Equipped with the scale-transferrable operation, STFFM can learn rich fusion features but with very lightweight network architecture. By dynamically adapting the spatial prior to the input slice data as well as the deep feature maps, PPM enables the network model to explore informative spatial priors for pancreas segmentation. Comprehensive experiments on the NIH dataset and the MSD dataset are conducted to evaluate the proposed approach. The obtained experimental results demonstrate that our approach can effectively reduce the computational cost and simultaneously archive the outperforming performance when compared to the state-of-the-art methods.

Lightweight Semantic Segmentation Network for Real-Time Weed Mapping Using Unmanned Aerial Vehicles

The timely and efficient generation of weed maps is essential for weed control tasks and precise spraying applications. Based on the general concept of site-specific weed management (SSWM), many researchers have used unmanned aerial vehicle (UAV) remote sensing technology to monitor weed distributions, which can provide decision support information for precision spraying. However, image processing is mainly conducted offline, as the time gap between image collection and spraying significantly limits the applications of SSWM. In this study, we conducted real-time image processing onboard a UAV to reduce the time gap between image collection and herbicide treatment. First, we established a hardware environment for real-time image processing that integrates map visualization, flight control, image collection, and real-time image processing onboard a UAV based on secondary development. Second, we exploited the proposed model design to develop a lightweight network architecture for weed mapping tasks. The proposed network architecture was evaluated and compared with mainstream semantic segmentation models. Results demonstrate that the proposed network outperform contemporary networks in terms of efficiency with competitive accuracy. We also conducted optimization during the inference process. Precision calibration was applied to both the desktop and embedded devices and the precision was reduced from FP32 to FP16. Experimental results demonstrate that this precision calibration further improves inference speed while maintaining reasonable accuracy. Our modified network architecture achieved an accuracy of 80.9% on the testing samples and its inference speed was 4.5 fps on a Jetson TX2 module (Nvidia Corporation, Santa Clara, CA, USA), which demonstrates its potential for practical agricultural monitoring and precise spraying applications.

Real-time Visual Tracking Based on Convolutional Neural Networks

Traditional target tracking is based on target detection. When the target changes significantly, such as occlusion, scale change, the update of the tracking model will waste a lot of space and time resources, resulting in a very slow tracking speed, which cannot meet the actual engineering needs. In view of the above situation, an end-to-end tracking strategy is proposed, which is simpler and faster than the existing technology. The proposed tracker only needs to detect the first frame image and use it as the input of the model, and set the multi-task loss function to predict the position of the next frame of the target and the size of the bounding box. This paper constructs a lightweight network architecture with an additional selection mechanism to avoid wasting resources for global search and matching. Through experiments, good results can be achieved on the standard data set, and tracking speeds close to one hundred frames per second are achieved, which is very competitive with existing advanced trackers.

Underwater image enhancement with global–local networks and compressed-histogram equalization

Due to the light absorption and scattering, captured underwater images usually contain severe color distortion and contrast reduction. To address the above problems, we combine the merits of deep learning and conventional image enhancement technology to improve the underwater image quality. We first propose a two-branch network to compensate the global distorted color and local reduced contrast, respectively. Adopting this global–local network can greatly ease the learning problem, so that it can be handled by using a lightweight network architecture. To cope with the complex and changeable underwater environment, we then design a compressed-histogram equalization to complement the data-driven deep learning, in which the parameters are fixed after training. The proposed compression strategy is able to generate vivid results without introducing over-enhancement and extra computing burden. Experiments demonstrate that our method significantly outperforms several state-of-the-arts in both qualitative and quantitative qualities.

TAI-SARNET: Deep Transferred Atrous-Inception CNN for Small Samples SAR ATR.

Since Synthetic Aperture Radar (SAR) targets are full of coherent speckle noise, the traditional deep learning models are difficult to effectively extract key features of the targets and share high computational complexity. To solve the problem, an effective lightweight Convolutional Neural Network (CNN) model incorporating transfer learning is proposed for better handling SAR targets recognition tasks. In this work, firstly we propose the Atrous-Inception module, which combines both atrous convolution and inception module to obtain rich global receptive fields, while strictly controlling the parameter amount and realizing lightweight network architecture. Secondly, the transfer learning strategy is used to effectively transfer the prior knowledge of the optical, non-optical, hybrid optical and non-optical domains to the SAR target recognition tasks, thereby improving the model’s recognition performance on small sample SAR target datasets. Finally, the model constructed in this paper is verified to be 97.97% on ten types of MSTAR datasets under standard operating conditions, reaching a mainstream target recognition rate. Meanwhile, the method presented in this paper shows strong robustness and generalization performance on a small number of randomly sampled SAR target datasets.

HDRANet: Hybrid Dilated Residual Attention Network for SAR Image Despeckling

In order to remove speckle noise from original synthetic aperture radar (SAR) images effectively and efficiently, this paper proposes a hybrid dilated residual attention network (HDRANet) with residual learning for SAR despeckling. Firstly, HDRANet employs the hybrid dilated convolution (HDC) in lightweight network architecture to enlarge the receptive field and aggregate global information. Then, a simple yet effective attention module, convolutional block attention module (CBAM), is integrated into the proposed model to constitute a residual HDC attention block through skip connection, which further enhances representation power and performance of the model. Extensive experimental results on the synthetic and real SAR images demonstrate the superior performance of HDRANet over the state-of-the-art methods in terms of quantitative metrics and visual quality.

Adaptive Importance Learning for Improving Lightweight Image Super-Resolution Network

Deep neural networks have achieved remarkable success in single image super-resolution (SISR). The computing and memory requirements of these methods have hindered their application to broad classes of real devices with limited computing power, however. One approach to this problem has been lightweight network architectures that balance the super-resolution performance and the computation burden. In this study, we revisit this problem from an orthogonal view, and propose a novel learning strategy to maximize the pixel-wise fitting ability of a given lightweight network architecture. Considering that the initial performance of the lightweight network is very limited, we present an adaptive importance learning scheme for SISR that trains the network with an easy-to-complex paradigm by dynamically updating the importance of image pixels on the basis of the training loss. Specifically, we formulate the network training and the importance learning into a joint optimization problem. With a carefully designed importance penalty function, the importance of individual pixels can be gradually increased through solving a convex optimization problem. The training process thus begins with pixels that are easy to reconstruct, and gradually proceeds to more complex pixels as fitting improves. Furthermore, the proposed learning scheme is able to seamlessly assimilate knowledge from a more powerful teacher network in the form of importance initialization, thus obtaining better initial performance for the network. Through learning the network parameters, and updating pixel importance, the proposed learning scheme enables smaller, lightweight, networks to achieve better performance than has previously been possible. Extensive experiments on four benchmark datasets demonstrate the potential benefits of the proposed learning strategy in lightweight SISR network enhancement. In some cases, our learned network with only $$25\%$$ of the parameters and computational complexity can produce comparable or even better results than the corresponding full-parameter network.

Efficient Face Alignment with Fast Normalization and Contour Fitting Loss

Face alignment is a key component of numerous face analysis tasks. In recent years, most existing methods have focused on designing high-performance face alignment systems and paid less attention to efficiency. However more face alignment systems are now applied on low-cost devices, such as mobile phones. In this article, we design a common efficient framework that can team with any face alignment regression network and improve the overall performance with nearly no extra computational cost. First, we discover that the maximum regression error exists in the face contour, where landmarks do not have distinct semantic positions, and thus are randomly labeled along the face contours in training data. To address this problem, we propose a novel contour fitting loss that dynamically adjusts the regression target during training so the network can learn more accurate semantic meanings of the contour landmarks and achieve better localization performance. Second, we decouple the complex sample variations in face alignment task and propose a Fast Normalization Module (FNM) to efficiently normalize considerable variations that can be described by geometric transformation. Finally, a new lightweight network architecture named Lightweight Alignment Module (LAM) is also proposed to achieve fast and precise face alignment on mobile devices. Our method achieves competitive performance with state-of-the-arts on 300W and AFLW2000-3D benchmarks. Meanwhile, the speed of our framework is significantly faster than other CNN-based approaches.