Lightweight Module Research Articles

Research on cassava disease classification using the multi-scale fusion model based on EfficientNet and attention mechanism.

Cassava disease is one of the leading causes to the serious decline of cassava yield. Because it is difficult to identify the characteristics of cassava disease, if not professional cassava growers, it will be prone to misjudgment. In order to strengthen the judgment of cassava diseases, the identification characteristics of cassava diseases such as different color of cassava leaf disease spots, abnormal leaf shape and disease spot area were studied. In this paper, deep convolutional neural network was used to classify cassava leaf diseases, and image classification technology was used to recognize and classify cassava leaf diseases. A lightweight module Multi-scale fusion model (MSFM) based on attention mechanism was proposed to extract disease features of cassava leaves to enhance the classification of disease features. The resulting feature map contained key disease identification information. The study used 22,000 cassava disease leaf images as a data set, including four different cassava leaf disease categories and healthy cassava leaves. The experimental results show that the cassava leaf disease classification model based on multi-scale fusion Convolutional Neural Network (CNN) improves EfficientNet compared with the original model, with the average recognition rate increased by nearly 4% and the average recognition rate up to 88.1%. It provides theoretical support and practical tools for the recognition and early diagnosis of plant disease leaves.

Yolo-Based Improvements in Remote Sensing Image Applications

The identification of some specific targets in remote sensing images is still quite challenging despite the adequate accuracy of deep learning-based target detection models. This work proposes a variant of YOLOv3 based on the residual structure as the backbone and the attention mechanism module, which improves the ability of YOLOv3 to extract features. SGE is a lightweight module that can fully extract features from images without bringing an increase in computation. Furthermore, the dilated encoder module used in YOLOF was introduced as a neck to enrich the perceptual field of the C5 feature layer by concatenating four layers of dilated convolution with different expansion coefficients. The C5 feature layer and the residual structure were further processed to contain sufficient scale information for further detection. In terms of the mean average precision (mAP), experimental results demonstrate that the proposed model outperforms the other models: YOLOv3, faster-RCNN-r50+GACL Net, and YOLOv4.

Joint alignment of the distribution in input and feature space for cross-domain aerial image semantic segmentation

Benefiting from the development of deep learning, researchers have made significant progress and achieved superior performance in the semantic segmentation of remote sensing (RS) data. However, when encountering an unseen scenario, the performance of a trained model deteriorates dramatically because of the domain shift. Unsupervised domain adaptation (UDA) provides an alternative to address the issue. Aligning the high-level representations via adversarial learning is a popular way, but it is difficult when there is a large gap in input space. With this consideration, we design a framework to jointly align the distribution in input and feature space. For input space alignment, we unify the resolution for the consistency of content and propose a lightweight module named Digital Number Transformer (DNT) to reduce the visual differences. For feature space alignment, we design a Multi-Scale Feature aggregation (MSF) module and introduce the Fine-Grained Discriminator (FGD) to conduct a category-level alignment at multiple layers so that features can be fully aligned and negative transfer can be reduced. We carried out experiments in diverse cross-domain scenarios, including the discrepancy in geographic position and the discrepancy in both geographic position and imaging mode. Comprehensive experiments demonstrate that our method outperforms other state-of-the-art methods in all scenarios.

Research on Insulator Defect Detection Based on an Improved MobilenetV1-YOLOv4.

Insulator devices are important for transmission lines, and defects such as insulator bursting and string loss affect the safety of transmission lines. In this study, we aim to investigate the problems of slow detection speed and low efficiency of traditional insulator defect detection algorithms, and to improve the accuracy of insulator fault identification and the convenience of daily work; therefore, we propose an insulator defect detection algorithm based on an improved MobilenetV1-YOLOv4. First, the backbone feature extraction network of YOLOv4 'Backbone' is replaced with the lightweight module Mobilenet-V1. Second, the scSE attention mechanism is introduced in stages of preliminary feature extraction and enhanced feature extraction, sequentially. Finally, the depthwise separable convolution substitutes the 3 × 3 convolution of the enhanced feature extraction network to reduce the overall number of network parameters. The experimental results show that the weight of the improved algorithm is 57.9 MB, which is 62.6% less than that obtained by the MobilenetV1-YOLOv4 model; the average accuracy of insulator defect detection is improved by 0.26% and reaches 98.81%; and the detection speed reaches 190 frames per second with an increase of 37 frames per second.

Fabricating lightweight ceramics by spraying clay on knitted structures

This research was carried out to develop a novel composite material consisting of a thread reinforcement and a clay matrix, as well as to develop a method of shaping this material into hollow spatial structures. Ceramic elements in the building industry are currently created by applying extruding, pressing and casting methods. The approach of spraying clay onto predefined knitted meshes increases the usability of digitally fabricated lightweight ceramic elements, while eliminating the need for scaffolding. In this approach, multiple layers of a fluid clay mass are sprayed onto the tensioned mesh using an industrial, six-axis robotic arm. This allows the precise application of the material and results in varying material thicknesses. Due to the complementary qualities of clay which absorbs compressive forces and threads which absorb tensile forces, lightweight structures can be created. The research involved experimenting with clay mixtures, several thread types, knitting methods and spraying techniques, as well as fabricating a 1:1 lightweight module as an architectural prototype.

FASTNN: A Deep Learning Approach for Traffic Flow Prediction Considering Spatiotemporal Features.

Traffic flow forecasting is a critical input to intelligent transportation systems. Accurate traffic flow forecasting can provide an effective reference for implementing traffic management strategies, developing travel route planning, and public transportation risk assessment. Recent deep learning approaches of spatiotemporal neural networks to predict traffic flow show promise, but could be difficult to separately model the spatiotemporal aggregation in traffic data and intrinsic correlation or redundancy of spatiotemporal features extracted by the filter of the convolutional network. This can introduce biases in the predictions that interfere with subsequent planning decisions in transportation. To solve the mentioned problem, the filter attention-based spatiotemporal neural network (FASTNN) was proposed in this paper. First, the model used 3-dimensional convolutional neural networks to extract universal spatiotemporal dependencies from three types of historical traffic flow, the residual units were employed to prevent network degradation. Then, the filter spatial attention module was constructed to quantify the spatiotemporal aggregation of the features, thus enabling dynamic adjustment of the spatial weights. To model the intrinsic correlation and redundancy of features, this paper also constructed a lightweight module, named matrix factorization based resample module, which automatically learned the intrinsic correlation of the same features to enhance the concentration of the model on information-rich features, and used matrix factorization to reduce the redundant information between different features. The FASTNN has experimented on two large-scale real datasets (TaxiBJ and BikeNYC), and the experimental results show that the FASTNN has better prediction performance than various baselines and variant models.

ARM: a lightweight module to amend facial expression representation

ARM: a lightweight module to amend facial expression representation

Sparse‐Dyn: Sparse dynamic graph multirepresentation learning via event‐based sparse temporal attention network

Dynamic graph neural networks (DGNNs) have been widely used in modeling and representation learning of graph structure data. Current dynamic representation learning focuses on either discrete learning which results in temporal information loss, or continuous learning which involves heavy computation. In this study, we proposed a novel DGNN, sparse dynamic (Sparse-Dyn). It adaptively encodes temporal information into a sequence of patches with an equal amount of temporal-topological structure. Therefore, while avoiding using snapshots which cause information loss, it also achieves a finer time granularity, which is close to what continuous networks could provide. In addition, we also designed a lightweight module, Sparse Temporal Transformer, to compute node representations through structural neighborhoods and temporal dynamics. Since the fully connected attention conjunction is simplified, the computation cost is far lower than the current state-of-the-art. Link prediction experiments are conducted on both continuous and discrete graph data sets. By comparing several state-of-the-art graph embedding baselines, the experimental results demonstrate that Sparse-Dyn has a faster inference speed while having competitive performance.

Deep parameter-free attention hashing for image retrieval

Deep hashing method is widely applied in the field of image retrieval because of its advantages of low storage consumption and fast retrieval speed. There is a defect of insufficiency feature extraction when existing deep hashing method uses the convolutional neural network (CNN) to extract images semantic features. Some studies propose to add channel-based or spatial-based attention modules. However, embedding these modules into the network can increase the complexity of model and lead to over fitting in the training process. In this study, a novel deep parameter-free attention hashing (DPFAH) is proposed to solve these problems, that designs a parameter-free attention (PFA) module in ResNet18 network. PFA is a lightweight module that defines an energy function to measure the importance of each neuron and infers 3-D attention weights for feature map in a layer. A fast closed-form solution for this energy function proves that the PFA module does not add any parameters to the network. Otherwise, this paper designs a novel hashing framework that includes the hash codes learning branch and the classification branch to explore more label information. The like-binary codes are constrained by a regulation term to reduce the quantization error in the continuous relaxation. Experiments on CIFAR-10, NUS-WIDE and Imagenet-100 show that DPFAH method achieves better performance.

Recursive Multi-Scale Channel-Spatial Attention for Fine-Grained Image Classification

Fine-grained image classification is a difficult problem, and previous studies mainly overcome this problem by locating multiple discriminative regions in different scales and then aggregating complementary information explored from the located regions. However, locating discriminative regions introduces heavy overhead and is not suitable for real-world application. In this paper, we propose the recursive multi-scale channel-spatial attention module (RMCSAM) for addressing this problem. Following the experience of previous research on fine-grained image classification, RMCSAM explores multi-scale attentional information. However, the attentional information is explored by recursively refining the deep feature maps of a convolutional neural network (CNN) to better correspond to multi-scale channel-wise and spatial-wise attention, instead of localizing attention regions. In this way, RMCSAM provides a lightweight module that can be inserted into standard CNNs. Experimental results show that RMCSAM can improve the classification accuracy and attention capturing ability over baselines. Also, RMCSAM performs better than other state-of-the-art attention modules in fine-grained image classification, and is complementary to some state-of-the-art approaches for fine-grained image classification. Code is available at https://github.com/Dichao-Liu/Recursive-Multi-Scale-Channel-Spatial-Attention-Module.

Convolutional neural network optimization via channel reassessment attention module

The performance of convolutional neural networks (CNNs) can be improved by adjusting the interrelationship between channels with attention mechanism. However, attention mechanism in recent advance ignores the effects of different spatial locations in feature maps on attention weights, which makes a great difference to the results of generated channel attentions. In this paper, we propose a novel network optimization module called Channel Reassessment Attention (CRA) which uses channel attentions with spatial information of feature maps to enhance representational power of networks. We employ CRA module to assess channel attentions in different channels, then adaptively refine the final features by channel multiplication between channel attentions and feature maps. CRA is a computational lightweight module and it can be embedded into any architectures of CNNs. The experiments on ImageNet, CIFAR and MS COCO datasets demonstrate that the embedding of CRA module on various networks effectively improves the performance under different evaluation standards.

Surface defect detection of vehicle light guide plates based on an improved RetinaNet

Aiming at the vehicle navigation light guide plate (LGP) image characteristics, such as complex and gradient textures, uneven brightness, and small defects, this paper proposes a visual inspection method for LGP defects based on an improved RetinaNet. First, we use ResNeXt50 with higher accuracy under the same parameters as the backbone network, and propose the lightweight module Ghost_module to replace the 1 × 1 convolution in the lower half of the ResNeXt_block. This can reduce the resource parameters and consumption, and speed up training and inference. Second, we propose and use an improved feature pyramid network module to improve the feature fusion network in RetinaNet. It can more effectively fuse the shallow semantic information and high-level semantic information in the backbone feature extraction network, and further improve the detection ability of small target defects. Finally, the defect detection dataset constructed based on the vehicle LGP images collected at a industrial site, and experiments are performed on the vehicle LGP dataset and Aluminum Profile Defect Identification dataset (Aluminum Profile DID). The experimental results show that the proposed method is both efficient and effective. It achieves a better average detection rate of 98.6% on the vehicle LGP dataset. The accuracy and real-time performance can meet the requirements of industrial detection.

An Improved Normed-Deformable Convolution for Crowd Counting

In recent years, crowd counting has become an important issue in computer vision. In most methods, the density maps are generated by convolving with a Gaussian kernel from the ground-truth dot maps which are marked around the center of human heads. Due to the fixed geometric structures in CNNs and indistinct head-scale information, the head features are obtained incompletely. Deformable convolution is proposed to exploit the scale-adaptive capabilities for CNN features in the heads. By learning the coordinate offsets of the sampling points, it is tractable to improve the ability to adjust the receptive field. However, the heads are not uniformly covered by the sampling points in the deformable convolution, resulting in loss of head information. To handle the non-uniformed sampling, an improved Normed-Deformable Convolution (\textit{i.e.,}NDConv) implemented by Normed-Deformable loss (\textit{i.e.,}NDloss) is proposed in this paper. The offsets of the sampling points which are constrained by NDloss tend to be more even. Then, the features in the heads are obtained more completely, leading to better performance. Especially, the proposed NDConv is a light-weight module which shares similar computation burden with Deformable Convolution. In the extensive experiments, our method outperforms state-of-the-art methods on ShanghaiTech A, ShanghaiTech B, UCF\_QNRF, and UCF\_CC\_50 dataset, achieving 61.4, 7.8, 91.2, and 167.2 MAE, respectively. The code is available at https://github.com/bingshuangzhuzi/NDConv

Ionizing Radiation Sensor for Nanosatellites, Microdrones and Small Unmanned Ground Vehicles

The purpose of this paper is to present an ionizing radiation sensor suitable for nanosatellites, small drones and other types of unmanned vehicles. When implemented on an airborne or ground-based unmanned vehicle the sensor is beneficial in disaster management scenarios such as inspection of buildings and facilities for ionizing radiation contamination and measurements of the ionizing radiation dose. Herein, a design and laboratory tests of the sensor are presented. The instrument is a lightweight module having low power consumption suitable for nanosatellite platforms and for mobile use by having it installed in an unmanned system (airborne or ground-based). For test purposes a 36-rotor drone was developed along with a mobile ground based unmanned vehicle. With all implementations the radiological sensor is directly connected to the main processor unit of the platform. In the case of the drone and ground-based vehicle systems the processor unit is the microcontroller of the autopilot. Experimental results of laboratory measurements of different radiation sources are shown and discussed. The experimental setup demonstrates a few advances related to specific problems encountered in the existing ionizing radiation measurement systems.

Attentive WaveBlock: Complementarity-Enhanced Mutual Networks for Unsupervised Domain Adaptation in Person Re-Identification and Beyond.

Unsupervised domain adaptation (UDA) for person re-identification is challenging because of the huge gap between the source and target domain. A typical self-training method is to use pseudo-labels generated by clustering algorithms to iteratively optimize the model on the target domain. However, a drawback to this is that noisy pseudo-labels generally cause trouble in learning. To address this problem, a mutual learning method by dual networks has been developed to produce reliable soft labels. However, as the two neural networks gradually converge, their complementarity is weakened and they likely become biased towards the same kind of noise. This paper proposes a novel light-weight module, the Attentive WaveBlock (AWB), which can be integrated into the dual networks of mutual learning to enhance the complementarity and further depress noise in the pseudo-labels. Specifically, we first introduce a parameter-free module, the WaveBlock, which creates a difference between features learned by two networks by waving blocks of feature maps differently. Then, an attention mechanism is leveraged to enlarge the difference created and discover more complementary features. Furthermore, two kinds of combination strategies, i.e. pre-attention and post-attention, are explored. Experiments demonstrate that the proposed method achieves state-of-the-art performance with significant improvements on multiple UDA person re-identification tasks. We also prove the generality of the proposed method by applying it to vehicle re-identification and image classification tasks. Our codes and models are available at: AWB.

Spatial–Spectral Involution MLP Network for Hyperspectral Image Classification

Recently, more and more multi-layer perceptron (MLP) -like models have been proposed. Among them, CycleMLP is good at dense feature prediction tasks, which is potentially useful for hyperspectral image (HSI) classification. However, the receptive field of CycleMLP tends to be cross-shaped, which will lead to insufficient spatial information extraction. Additionally, most of the HSI classification methods only use information from single HSI data. Lack of diversity in the features of a single modality limits classification performance. To address these issues, a novel spatial-spectral involution MLP network (SSIN) is proposed for HSI classification. SSIN contains two paths for extracting different kinds of information, namely the image path and the coordinate path. In the image path, we combine the MLP structure with the involution operation and propose involution MLP (InvoMLP). It obtains the spatial kernel weights corresponding to each pixel individually, thus improving the spatial interaction capability. At the same time, InvoMLP has the same receptive field range as conventional convolution, i.e., a rectangular receptive field. In the coordinate path, we build a lightweight module for extracting information. Unlike the information of images, the coordinates are intuitive information about the location distribution. Considering that the coordinate information contains the global spatial distribution of HSI, fusing it with the image information could improve long-distance dependencies of feature maps. Experimental results on four HSI datasets illustrate that SSIN can outperform some state-of-the-art methods.

Compact Band Weighting Module Based on Attention-Driven for Hyperspectral Image Classification

Hyperspectral image (HSI) data have large numbers of bands that probably not all bands are equally informative and predictive for an effective HSI classification. Effective algorithms are highly desired in many real-world HSI applications, especially in cases requiring rapid learning with limited computing power. To address the abovementioned case, we present in this article a novel plug-and-play compact band weighting (CBW) module based on the attention-driven mechanism that evaluates different spectral bands according to their contributions to a given classification task. Compared to existing band weighting (BW) modules with tens of thousands of network parameters by deep learning, the proposed CBW is a lightweight module with only 20 parameters. Both model complexity and time cost are significantly reduced. The CBW module implements BW by making full use of the correlation among the adjacent spectral bands and spectral statistic information and, thereby, leads to the effect of recalibrated HSI. The experimental study has been conducted on three widely used HSI data sets, and results show the superiority of the proposed algorithm over current state-of-the-art methods of BW. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/JarvenYi/CBW</uri> .

Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding

The prevailing framework for solving referring expression grounding is based on a two-stage process: 1) detecting proposals with an object detector and 2) grounding the referent to one of the proposals. Existing two-stage solutions mostly focus on the grounding step, which aims to align the expressions with the proposals. In this paper, we argue that these methods overlook an obvious mismatch between the roles of proposals in the two stages: they generate proposals solely based on the detection confidence (i.e., expression-agnostic), hoping that the proposals contain all right instances in the expression (i.e., expression-aware). Due to this mismatch, current two-stage methods suffer from a severe performance drop between detected and ground-truth proposals. To this end, we propose Ref-NMS, which is the first method to yield expression-aware proposals at the first stage. Ref-NMS regards all nouns in the expression as critical objects, and introduces a lightweight module to predict a score for aligning each box with a critical object. These scores can guide the NMS operation to filter out the boxes irrelevant to the expression, increasing the recall of critical objects, resulting in a significantly improved grounding performance. Since Ref- NMS is agnostic to the grounding step, it can be easily integrated into any state-of-the-art two-stage method. Extensive ablation studies on several backbones, benchmarks, and tasks consistently demonstrate the superiority of Ref-NMS. Codes are available at: https://github.com/ChopinSharp/ref-nms.

LCC-Net: A Lightweight Cross-Consistency Network for Semisupervised Cardiac MR Image Segmentation.

Semantic segmentation plays a crucial role in cardiac magnetic resonance (MR) image analysis. Although supervised deep learning methods have made significant performance improvements, they highly rely on a large amount of pixel-wise annotated data, which are often unavailable in clinical practices. Besides, top-performing methods usually have a vast number of parameters, which result in high computation complexity for model training and testing. This study addresses cardiac image segmentation in scenarios where few labeled data are available with a lightweight cross-consistency network named LCC-Net. Specifically, to reduce the risk of overfitting on small labeled datasets, we substitute computationally intensive standard convolutions with a lightweight module. To leverage plenty of unlabeled data, we introduce extreme consistency learning, which enforces equivariant constraints on the predictions of different perturbed versions of the input image. Cutting and mixing different training images, as an extreme perturbation on both the labeled and unlabeled data, are utilized to enhance the robust representation learning. Extensive comparisons demonstrate that the proposed model shows promising performance with high annotation- and computation-efficiency. With only two annotated subjects for model training, the LCC-Net obtains a performance gain of 14.4% in the mean Dice over the baseline U-Net trained from scratch.

Insulator Fault Detection in Aerial Images based on the Mixed-grouped Fire Single-shot Multibox Detector

Abstract In a complex background, insulator fault is the main factor behind transmission accidents. With the wide application of unmanned aerial vehicle (UAV) photography, digital image recognition technology has been further developed to detect the position and fault of insulators. There are two mainstream methods based on deep learning: the first is the “two-stage” example for a region convolutional neural network and the second is the “one-stage” example such as a single-shot multibox detector (SSD), both of which pose many difficulties and challenges. However, due to the complex background and various types of insulators, few researchers apply the “two-stage” method for the detection of insulator faults in aerial images. Moreover, the detection performance of “one-stage” methods is poor for small targets because of the smaller scope of vision and lower accuracy in target detection. In this article, the authors propose an accurate and real-time method for small object detection, an example for insulator location, and its fault inspection based on a mixed-grouped fire single-shot multibox detector (MGFSSD). Based on SSD and deconvolutional single-shot detector (DSSD) networks, the MGFSSD algorithm solves the problems of inaccurate recognition in small objects of the SSD and complex structure and long running time of the DSSD. To resolve the problems of some target repeated detection and small-target missing detection of the original SSD, the authors describe how to design an effective and lightweight feature fusion module to improve the performance of traditional SSDs so that the classifier network can take full advantage of the relationship between the pyramid layer features without changing the base network closest to the input data. The data processing results show that the method can effectively detect insulator faults. The average detection accuracy of insulator faults is 92.4% and the average recall rate is 91.2%.