Lightweight Feature Extraction Research Articles

A Lightweight Hand Attitude Estimation Method Based on GCN Feature Enhancement

In this study, a hand pose estimation method based on GCN feature enhancement is proposed to address the problem of the time-consuming nature and neglection of the internal relationships between hand joint points, which results in the low accuracy of hand pose estimation. Firstly, a lightweight feature extraction network RexNet is used, and deep separable convolutions are used instead of ordinary convolutions to reduce the model parameters and computational complexity. Secondly, deconvolution is added to the backend of the network to obtain preliminary estimation results of joint points. Finally, the GCN feature enhancement module is used to modify the preliminary estimation results to improve the accuracy of hand pose estimation. The proposed method is tested for accuracy on the CMU-Hand and RHD datasets. The results show that the proposed method achieves an AUC metric of 80.1% on the CMU-Hand dataset and 97.0% on the RHD dataset, and the accuracy of hand pose estimation is high.

Lightweight Multi-Domain Fusion Model for Through-Wall Human Activity Recognition Using IR-UWB Radar

Impulse radio ultra-wideband (IR-UWB) radar, operating in the low-frequency band, can penetrate walls and utilize its high range resolution to recognize different human activities. Complex deep neural networks have demonstrated significant performance advantages in classifying radar spectrograms of various actions, but at the cost of a substantial computational overhead. In response, this paper proposes a lightweight model named TG2-CAFNet. First, clutter suppression and time–frequency analysis are used to obtain range–time and micro-Doppler feature maps of human activities. Then, leveraging GhostV2 convolution, a lightweight feature extraction module, TG2, suitable for radar spectrograms is constructed. Using a parallel structure, the features of the two spectrograms are extracted separately. Finally, to further explore the correlation between the two spectrograms and enhance the feature representation capabilities, an improved nonlinear fusion method called coordinate attention fusion (CAF) is proposed based on attention feature fusion (AFF). This method extends the adaptive weighting fusion of AFF to a spatial distribution, effectively capturing the subtle spatial relationships between the two radar spectrograms. Experiments showed that the proposed method achieved a high degree of model lightweightness, while also achieving a recognition accuracy of 99.1%.

LRSDet: lightweight remote sensing target detection network with local-global information awareness and rapid sample assignment

Abstract With the rapid development of aerospace and unmanned aerial vehicles, using neural networks for object detection in optical remote sensing images (O-RSI) has encountered heightened challenges. The optical remote sensing images have the characteristics of complex geometric scenes, dense groups of objects, and significant multi-scale variations of objects; researchers need to use more complex models to achieve higher accuracy. However, this complexity also brings challenges to the application of lightweight scenes. Therefore, to cope with the trade-off challenge between model complexity and detection accuracy, we propose a lightweight network model LRSDet in this study. The model integrates local and global information processing mechanisms and introduces a fast positive sample assignment strategy to adapt to resource-constrained embedded and mobile platforms. By constructing a lightweight feature extraction network and a lightweight path aggregation network and incorporating the ESM-Attention module, the feature extraction capability of the model in complex remote sensing scenarios is significantly improved. In addition, the application of a dynamic soft threshold strategy further optimizes the positive sample selection process and improves the detection efficiency of the model. Experimental on the O-RSI datasets DIOR, NWPU VHR-10, and RSOD, while analyzing model real-time performance on aerial video and embedded devices, outperforming other state-of-the-art methods.

A Lightweight Model Enhancing Facial Expression Recognition with Spatial Bias and Cosine-Harmony Loss

This paper proposes a facial expression recognition network called the Lightweight Facial Network with Spatial Bias (LFNSB). The LFNSB model effectively balances model complexity and recognition accuracy. It has two key components: a lightweight feature extraction network (LFN) and a Spatial Bias (SB) module for aggregating global information. The LFN introduces combined channel operations and depth-wise convolution techniques, effectively reducing the number of parameters while enhancing feature representation capability. The Spatial Bias module enables the model to focus on local facial features while capturing the dependencies between different facial regions. Additionally, a new loss function called Cosine-Harmony Loss is designed. This function optimizes the relative positions of feature vectors in high-dimensional space, resulting in better feature separation and clustering. Experimental results on the AffectNet and RAF-DB datasets demonstrate that the LFNSB model achieves competitive recognition accuracy, with 63.12% accuracy on AffectNet-8, 66.57% accuracy on AffectNet-7, and 91.07% accuracy on RAF-DB, while significantly reducing the model complexity.

A lightweight finger multimodal recognition model based on detail optimization and perceptual compensation embedding

Multimodal biometric recognition technology has attracted the attention of many scholars due to its higher security and stability than single-modal recognition, but its additional parameter quantity and computational cost have brought challenges to the lightweight deployment of the model. In order to meet the needs of a wider range of application scenarios, this paper proposes a lightweight model DPNet using fingerprint and finger vein images for multimodal recognition, which adopts a double-branch lightweight feature extraction structure combining detail optimization and perception compensation. Among them, the detail extraction optimization branch uses multi-scale dimensionality reduction filtering to obtain low-redundant detail information, and combines the depth extension operation to enhance the generalization ability of detail features. The perception compensation branch expands and compensates the model's perceptual field of view through lightweight spatial location query and global information attention. In addition, this paper designs a perceptual feature embedding method to embed perceptual compensation information in the way of importance adjustment to improve the consistency of embedded features. The ABFM fusion module is proposed to carry out multi-level lightweight and deep interactive fusion of the extracted finger modal features from the global to the spatial region, so as to improve the degree and utilization rate of feature fusion. In this paper, the model recognition performance and lightweight advantages are verified on three multimodal datasets. Experimental results show that the proposed model achieves the most advanced lightweight effect and recognition performance in the experimental comparison of all datasets.

An in-vehicle real-time infrared object detection system based on deep learning with resource-constrained hardware

Advanced driver assistance systems primarily rely on visible images for information. However, in low-visibility weather conditions, such as heavy rain or fog, visible images struggle to capture road conditions accurately. In contrast, infrared (IR) images can overcome this limitation, providing reliable information regardless of external lighting. Addressing this problem, we propose an in-vehicle IR object detection system. We optimize the you only look once (YOLO) v4 object detection algorithm by replacing its original backbone with MobileNetV3, a lightweight feature extraction network, resulting in the MobileNetV3-YOLOv4 model. Furthermore, we replace traditional pre-processing methods with an Image Enhancement Conditional Generative Adversarial Network inversion algorithm to enhance the pre-processing of the input IR images. Finally, we deploy the model on the Jetson Nano, an edge device with constrained hardware resources. Our proposed method achieves an 82.7% mean Average Precision and a frame rate of 55.9 frames per second on the FLIR dataset, surpassing state-of-the-art methods. The experimental results confirm that our approach provides outstanding real-time detection performance while maintaining high precision.

FedCPG: A class prototype guided personalized lightweight federated learning framework for cross-factory fault detection

Industrial equipment condition monitoring and fault detection are crucial to ensure the reliability of industrial production. Recently, data-driven fault detection methods have achieved significant success, but they all face challenges due to data fragmentation and limited fault detection capabilities. Although centralized data collection can improve detection accuracy, the conflicting interests brought by data privacy issues make data sharing between different devices impractical, thus forming the problem of industrial data silos. To address these challenges, this paper proposes a class prototype guided personalized lightweight federated learning framework(FedCPG). This framework decouples the local network, only uploading the backbone model to the server for model aggregation, while employing the head model for local personalized updates, thereby achieving efficient model aggregation. Furthermore, the framework incorporates prototype constraints to steer the local personalized update process, mitigating the effects of data heterogeneity. Finally, a lightweight feature extraction network is designed to reduce communication overhead. Multiple complex industrial data distribution scenarios were simulated on two benchmark industrial datasets. Extensive experiments have demonstrated that FedCPG can achieve an average detection accuracy of 95% in complex industrial scenarios, while simultaneously reducing memory usage and the number of parameters by 82%, surpassing existing methods in most average metrics. These findings offer novel perspectives on the application of personalized federated learning in industrial fault detection.

Fault diagnosis method for bearings under variable working conditions based on transfer relation network

Abstract In order to achieve robust fault diagnosis under varying conditions with limited labeled data, this study combines metric-based meta-learning with feature-based domain adaptation. It introduces a new approach for variable-condition bearing fault diagnosis using transfer relation networks. To enhance the network’s ability to generalize across different domains, the paper integrates local maximum mean discrepancy (LMMD) into the relation network architecture. LMMD aligns the data distributions of various classes between the source and target domains, effectively addressing distributional differences and improving model generalization. To accurately and swiftly extract meaningful fault features, the study proposes a lightweight feature extraction module based on Shuffle Attention (SA). This module employs depth-wise separable convolutions for efficiency and integrates SA after each convolutional layer to bolster feature representation. Finally, experiments on two bearing datasets under varying conditions validate the efficacy and superiority of the proposed model over alternative methods.

Insulator Defect Detection Based on YOLOv5s-KE

To tackle the issue of low detection accuracy in insulator images caused by intricate backgrounds and small defect sizes, as well as the requirement for real-time detection on embedded and mobile devices, this research introduces the YOLOv5s-KE model. Integrating multiple strategies, YOLOv5s-KE aims to boost detection accuracy significantly. Initially, an enhanced anchor generation method utilizing the K-means++ algorithm is proposed to generate more appropriate anchor boxes for insulator defects. Moreover, an attention mechanism is integrated into both the backbone and neck networks to enhance the model’s capacity to focus on defect features and resist interference. To improve the detection of small defects, the EIoU loss function is implemented in place of the original CIoU loss function. In order to meet the real-time detection needs on embedded and mobile devices, the model is further refined through the integration of Ghost convolution for lightweight feature extraction and a linear transformation to reduce the computational burden of standard convolution. A channel pruning strategy is deployed to optimize the sparsely trained network, diminishing redundancy, and improving model generalization. Additionally, the CARAFE operator replaces the original upsampling operator to minimize model parameters and elevate detection speed. Experimental outcomes demonstrate that YOLOv5s-KE achieves a detection accuracy of 92.3% on the Chinese transmission line insulator dataset, marking a 5.2% enhancement over the original YOLOv5s. The streamlined version of YOLOv5s-KE achieves a detection speed of 94.3 frames per second, indicating an improvement of 30.1 frames per second compared to the original model. Model parameters are condensed to 9.6 M, resulting in a detection accuracy of 91.1%. This study underscores the precision and efficiency of the proposed approach, suggesting that the advanced strategies explored introduce novel possibilities for insulator defect detection.

Detection of Road Risk Sources Based on Multi-Scale Lightweight Networks.

Timely discovery and disposal of road risk sources constitute the cornerstone of road operation safety. Presently, the detection of road risk sources frequently relies on manual inspections via inspection vehicles, a process that is both inefficient and time-consuming. To tackle this challenge, this paper introduces a novel automated approach for detecting road risk sources, termed the multi-scale lightweight network (MSLN). This method primarily focuses on identifying road surfaces, potholes, and scattered objects. To mitigate the influence of real-world factors such as noise and uneven brightness on test results, pavement images were carefully collected. Initially, the collected images underwent grayscale processing. Subsequently, the median filtering algorithm was employed to filter out noise interference. Furthermore, adaptive histogram equalization techniques were utilized to enhance the visibility of cracks and the road background. Following these preprocessing steps, the MSLN model was deployed for the detection of road risk sources. Addressing the challenges associated with two-stage network models, such as prolonged training and testing times, as well as deployment difficulties, this study adopted the lightweight feature extraction network MobileNetV2. Additionally, transfer learning was incorporated to elevate the model's training efficiency. Moreover, this paper established a mapping relationship model that transitions from the world coordinate system to the pixel coordinate system. This model enables the calculation of risk source dimensions based on detection outcomes. Experimental results reveal that the MSLN model exhibits a notably faster convergence rate. This enhanced convergence not only boosts training speed but also elevates the precision of risk source detection. Furthermore, the proposed mapping relationship coordinate transformation model proves highly effective in determining the scale of risk sources.

A lightweight feature extraction backbone for gangue detection with improved Resnet50

ABSTRACT In response to the sizable parameters and floating point operations (FLOPs) of feature extraction backbone of gangue detection algorithm, a lightweight backbone was proposed by improving Resnet50. Initially, the partial 3 × 3 convolution was used to replace regular 3 × 3 convolution in original Resnet50 for reducing parameters and FLOPs in convolution operation. Subsequently, layers of batch normalization and rectified linear unit after 3 × 3 convolution were removed for their further reduction. Finally, bottleneck attention module was integrated at the bottleneck of Resnet50 to improve the mean average precision (mAP) in gangue detection. Based on three proposed methods, the ablation and comparative experiments were carried out. The reduction in parameters and FLOPs and increase in mAP were analyzed and discussed according to experimental results. It could be concluded that the parameters and FLOPs of the obtained lightweight backbone were no more than 56.36% and 57.42% of those of the popular backbones, while the decrease amplitude of its mAP value was no more than 0.2%.

Continuous Wavelet Transform Peak-Seeking Attention Mechanism Conventional Neural Network: A Lightweight Feature Extraction Network with Attention Mechanism Based on the Continuous Wave Transform Peak-Seeking Method for Aero-Engine Hot Jet Fourier Transform Infrared Classification

Continuous Wavelet Transform Peak-Seeking Attention Mechanism Conventional Neural Network: A Lightweight Feature Extraction Network with Attention Mechanism Based on the Continuous Wave Transform Peak-Seeking Method for Aero-Engine Hot Jet Fourier Transform Infrared Classification

An effective object detection algorithm for UAV-based urban regulation

Target detection from the perspective of UAV has great potential in the field of urban regulation, limited by the dense small targets, severe environmental obstructions, camera shake, and changes in lighting conditions in the aerial view of drones, the existing object detection algorithms cannot effectively undertake this task. This paper introduces two lightweight feature extraction modules based on YOLOv5, which are C3-Faster with PConv and COT3 with transformer structure. Meanwhile, an extra small detection head is added to the output layer. These approaches enhance accuracy while maintaining the advantages of being lightweight and easy to deploy. The ablation experiments and comparative experiments are designed to verify the effectiveness of these modules. The algorithm presented in this paper can be deployed into embedded systems of small UAVs to assist UAVs in completing various regulatory tasks in complex urban scenarios.

A Efficient and Accurate UAV Detection Method Based on YOLOv5s

Due to the limited computational resources of portable devices, target detection models for drone detection face challenges in real-time deployment. To enhance the detection efficiency of low, slow, and small unmanned aerial vehicles (UAVs), this study introduces an efficient drone detection model based on YOLOv5s (EDU-YOLO), incorporating lightweight feature extraction and balanced feature fusion modules. The model employs the ShuffleNetV2 network and coordinate attention mechanisms to construct a lightweight backbone network, significantly reducing the number of model parameters. It also utilizes a bidirectional feature pyramid network and ghost convolutions to build a balanced neck network, enriching the model’s representational capacity. Additionally, a new loss function, EloU, replaces CIoU to improve the model’s positioning accuracy and accelerate network convergence. Experimental results indicate that, compared to the YOLOv5s algorithm, our model only experiences a minimal decrease in mAP by 1.1%, while reducing GFLOPs from 16.0 to 2.2 and increasing FPS from 153 to 188. This provides a substantial foundation for networked optoelectronic detection of UAVs and similar slow-moving aerial targets, expanding the defensive perimeter and enabling earlier warnings.

A real-time and high-accuracy railway obstacle detection method using lightweight CNN and improved transformer

With the sustained development of railway transportation, the urgent necessity to improve train operation safety makes obstacle detection become the research focus. However, existing railway obstacle detectors still face challenges in balancing detection accuracy and speed during the shunting process. In addition, they are not robust enough in real-world railway environments, especially in complex scenes involving small obstacles. To address these problems, this paper presents a real-time and high-accuracy railway obstacle detection model using lightweight CNN and improved transformer (RH-Net) for detecting railway obstacles efficiently to guarantee traffic safety. First, the Lightweight Feature Extraction Module (LEM) is designed to minimize the model’s computational load while maintaining its feature extraction ability. Then, the Improved Transformer Module (IFM) is developed to boost the model’s ability about stably extract global contextual information. Finally, the Enhanced Multi-Scale Feature Fusion Module (EFM) is proposed to optimize the detection of obstacles with different sizes, especially small objects. In the experiments on railway dataset, RH-Net achieves optimal detection performance on GeForce GTX 1080Ti (96.99% mAP and 135 FPS) and Jetson Xavier NX (97.02% mAP and 43 FPS), which is significantly superior to the existing detection models. Experimental results show that RH-Net has excellent detection ability, which can accurately and efficiently detect obstacles in complicated railway environments. Moreover, the experiments on MS COCO indicate that RH-Net can achieve more satisfactory detection performance than existing state-of-the-art methods. Therefore, the proposed model can be well-applied to more complex real-world scenes for multiple object detection.

A two-stream-network based steganalysis network: TSNet

The three-stage framework consisting of preprocessing, feature extraction and classification is the most commonly used structure in traditional deep learning based steganalysis networks (SNs). Preprocessing, as the core part of the three-stage framework, typically adopts fixed high-pass filters to generate filter residuals, and exploits a feature enhancement module following high-pass filters to improve signal-to-noise ratio of the stego signal, thus strengthening the expression of steganographic features. However, fixed high-pass filters can only provide filter residuals through the linear relationships, rather than complex dependencies automatically learned, among local pixels, which suggests that fixed high-pass filters lack powerful ability of extracting comprehensive steganographic features from the beginning of SNs. Inspired by the motivation, a two-stream-network based SN called TSNet is proposed to extract comprehensive steganographic features from two different perspectives, namely automatic learning and fixed filtering. The discrete cosine transform attention mechanism (DCTAM)-based fusion module (DCTAM-FM) with considering the steganalysis properties is proposed to fuse two streams and enhance channel representation using DCTAM, which is conductive to improving the detection accuracy. Two streams aim to explore different types of residuals from two perspectives, but inevitably lead to a considerable number of parameters and high computational complexity. To this end, the feature extraction stage contains lightweight depthwise separable convolution blocks and the linearly increasing style of the number of channels. Combining the two-stream network, DCTAM-FM as well as the lightweight feature extraction stage, TSNet obtains deep composite features, which can fully express the comprehensive steganographic features, and thus brings a high detection accuracy. Experimental results from three benchmark datasets demonstrate that TSNet outperforms current state-of-the-art methods in detection accuracy and has an acceptable computational complexity.

Enhanced Real-Time Target Detection for Picking Robots Using Lightweight CenterNet in Complex Orchard Environments

The rapid development of artificial intelligence and remote sensing technologies is indispensable for modern agriculture. In orchard environments, challenges such as varying light conditions and shading complicate the tasks of intelligent picking robots. To enhance the recognition accuracy and efficiency of apple-picking robots, this study aimed to achieve high detection accuracy in complex orchard environments while reducing model computation and time consumption. This study utilized the CenterNet neural network as the detection framework, introducing gray-centered RGB color space vertical decomposition maps and employing grouped convolutions and depth-separable convolutions to design a lightweight feature extraction network, Light-Weight Net, comprising eight bottleneck structures. Based on the recognition results, the 3D coordinates of the picking point were determined within the camera coordinate system by using the transformation relationship between the image’s physical coordinate system and the camera coordinate system, along with depth map distance information of the depth map. Experimental results obtained using a testbed with an orchard-picking robot indicated that the proposed model achieved an average precision (AP) of 96.80% on the test set, with real-time performance of 18.91 frames per second (FPS) and a model size of only 17.56 MB. In addition, the root-mean-square error of positioning accuracy in the orchard test was 4.405 mm, satisfying the high-precision positioning requirements of the picking robot vision system in complex orchard environments.

Deep Compressed Communication and Application in Multi-Robot 2D-Lidar SLAM: An Intelligent Huffman Algorithm.

Multi-robot Simultaneous Localization and Mapping (SLAM) systems employing 2D lidar scans are effective for exploration and navigation within GNSS-limited environments. However, scalability concerns arise with larger environments and increased robot numbers, as 2D mapping necessitates substantial processor memory and inter-robot communication bandwidth. Thus, data compression prior to transmission becomes imperative. This study investigates the problem of communication-efficient multi-robot SLAM based on 2D maps and introduces an architecture that enables compressed communication, facilitating the transmission of full maps with significantly reduced bandwidth. We propose a framework employing a lightweight feature extraction Convolutional Neural Network (CNN) for a full map, followed by an encoder combining Huffman and Run-Length Encoding (RLE) algorithms to further compress a full map. Subsequently, a lightweight recovery CNN was designed to restore map features. Experimental validation involves applying our compressed communication framework to a two-robot SLAM system. The results demonstrate that our approach reduces communication overhead by 99% while maintaining map quality. This compressed communication strategy effectively addresses bandwidth constraints in multi-robot SLAM scenarios, offering a practical solution for collaborative SLAM applications.

Automatic arrival-time picking of P- and S-waves of micro-seismic events based on relative standard generative adversarial network and GHRA

Rapid, high-precision pickup of microseismic P- and S-waves is an important basis for microseismic monitoring and early warning. However, it is difficult to provide fast and highly accurate pickup of micro-seismic P- and S-waves arrival-time. To address this, the study proposes a lightweight and high-precision micro-seismic P- and S-waves arrival times picking model, lightweight adversarial U-shaped network (LAU-Net), based on the framework of the generative adversarial network, and successfully deployed in low-power devices. The pickup network constructs a lightweight feature extraction layer (GHRA) that focuses on extracting pertinent feature information, reducing model complexity and computation, and speeding up pickup. We propose a new adversarial learning strategy called application-aware loss function. By introducing the distribution difference between the predicted results and the artificial labels during the training process, we improve the training stability and further improve the pickup accuracy while ensuring the pickup speed. Finally, 8986 and 473 sets of micro-seismic events are used as training and testing sets to train and test the LAU-Net model, and compared with the STA/LTA algorithm, CNNDET+CGANet algorithm, and UNet++ algorithm, the speed of each pickup is faster than that of the other algorithms by 11.59ms, 15.19ms, and 7.79ms, respectively. The accuracy of the P-wave pickup is improved by 0.221, 0.01, and 0.029, respectively, and the S-wave pickup accuracy is improved by 0.233, 0.135, and 0.102, respectively. It is further applied in the actual project of the Shengli oilfield in Sichuan. The LAU-Net model can meet the needs of practical micro-seismic monitoring and early warning and provides a new way of thinking for accurate and fast on-time picking of micro-seismic P- and S-waves.

MFMENet: multi-scale features mutual enhancement network for change detection in remote sensing images

ABSTRACT Change detection, an important task in remote sensing image analysis, has been extensively studied in recent years. However, change detection still faces problems such as difficulty in detecting small targets and incomplete edge detection. Furthermore, pseudo changes such as seasonal changes can also lead to many false detections. In response to these challenges, we propose the Multi-scale Features Mutual Enhancement Network (MFMENet), a simple yet efficient network. MFMENet maximizes feature utilization through mutual guidance and supplementation, enhancing the detection capabilities for small targets and edges. First, we use a lightweight feature extraction network to extract features, which mitigates the information loss caused by continuous downsampling of an overly deep network structure. Then, we design a Context Adaptive Interaction Module (CAIM) to realize the complementarity of feature information at different levels. This facilitates shallow features in gaining more semantic information and deep features in acquiring more texture information, thereby enhancing the model’s capability to capture more comprehensive edge features while effectively mitigating interference from pseudo changes. Finally, we introduce a Feature Aggregation Comparison Module (FACM), which uses a combination of aggregation and comparison methods to refine and enhance features. FACM can not only highlight the changed features but also retain more details, improving the model’s detection ability of small targets and edge details. The full utilization of features and effective mutual enhancement of information ensure the improvement of MFMENet’s performance in small target and edge detection. Extensive experiments on three publicly available datasets (LEVIR, DSIFN, and CDD) demonstrate that our approach achieves superior performance with fewer parameters compared to state-of-the-art methods in recent years. In comparison to these baseline methods, our proposed approach achieves improvements of 0.98%, 12.24%, and 2.03% in the IOU metric on the LEVIR, DSIFN, and CDD datasets, respectively, while utilizing only 1.1 M parameters.