Multi-scale Features Research Articles

DLCH-YOLO: An Object Detection Algorithm for Monitoring the Operation Status of Circuit Breakers in Power Scenarios

In the scenario of power system monitoring, detecting the operating status of circuit breakers is often inaccurate due to variable object scales and background interference. This paper introduces DLCH-YOLO, an object detection algorithm aimed at identifying the operating status of circuit breakers. Firstly, we propose a novel C2f_DLKA module based on Deformable Large Kernel Attention. This module adapts to objects of varying scales within a large receptive field, thereby more effectively extracting multi-scale features. Secondly, we propose a Semantic Screening Feature Pyramid Network designed to fuse multi-scale features. By filtering low-level semantic information, it effectively suppresses background interference to enhance localization accuracy. Finally, the feature extraction network incorporates Generalized-Sparse Convolution, which combines depth-wise separable convolution and channel mixing operations, reducing computational load. The DLCH-YOLO algorithm achieved a 91.8% mAP on our self-built power equipment dataset, representing a 4.7% improvement over the baseline network Yolov8. With its superior detection accuracy and real-time performance, DLCH-YOLO outperforms mainstream detection algorithms. This algorithm provides an efficient and viable solution for circuit breaker status detection.

Contrastive Learning based Speech Spoofing Detection for Multimedia Security in Edge Intelligence

Artificial intelligence (AI) empowered edge computing has given rise to a new paradigm and effectively facilitated the promotion and development of multimedia applications. The speech assistant is one of the significant services provided by multimedia applications, which aims to offer intelligent interactive experiences between humans and machines. However, malicious attackers may exploit spoofed speeches to deceive speech assistants, posing great challenges to the security of multimedia applications. The limited resources of multimedia terminal devices hinder their ability to effectively load speech spoofing detection models. Furthermore, processing and analyzing speech in the cloud can result in poor real-time performance and potential privacy risks. Existing speech spoofing detection methods rely heavily on annotated data and exhibit poor generalization capabilities for unseen spoofed speeches. To address these challenges, this paper first proposes the Coordinate Attention Network (CA2Net) that consists of coordinate attention blocks and Res2Net blocks. CA2Net can simultaneously extract temporal and spectral speech feature information and represent multi-scale speech features at a granularity level. Besides, a contrastive learning-based speech spoofing detection framework named GEMINI is proposed. GEMINI can be effectively deployed on edge nodes and autonomously learn speech features with strong generalization capabilities. GEMINI first performs data augmentation on speech signals and extracts conventional acoustic features to enhance the feature robustness. Subsequently, GEMINI utilizes the proposed CA2Net to further explore the discriminative speech features. Then, a tensor-based multi-attention comparison model is employed to maximize the consistency between speech contexts. GEMINI continuously updates CA2Net with contrastive learning, which enables CA2Net to effectively represent speech signals and accurately detect spoofed speeches. Extensive experiments on the ASVspoof2019 dataset show that GEMINI reduces the Equal Error Rate and tandem Detection Cost Function by up to 96.75% and 96.35% in the physical access scenario, and by up to 86.62% and 87.71% in the logical access scenario compared to peer methods.

Biomimetic Hierarchies for Universal Surface Enhancement and Applications in Water Treatment.

Hierarchical superstructures, ubiquitously found in nature, offer enhanced efficiency in both substance reaction and mass transport owing to their unique multiscale features. Inspired by these natural systems, this research reports a general and scalable electrochemical scheme for creating highly branched, multilevel porous superstructures on various electrically conductive substrates. These structures exhibit cascading features from centimeters, submillimeters, micrometers, down to sub-100 nm, significantly increasing the surface area of substrates, such as foams, foils, and carbon cloth by 2 orders of magnitude─among the highest reported enhancements. This versatile and low-cost method, applicable to a range of electrically conductive substrates, enables innovative flow-assisted water purification with enhanced energy efficiency. The performance, successfully removing 99% of mercury within 0.5 h at 540 rpm and meeting the U.S. Environmental Protection Agency (EPA) safety standards for drinking water, further validates the advantages of these unique structures. Overall, the reported general, economical, and versatile scheme could broadly impact energy and environmental remediation.

Improved Multi-Size, Multi-Target and 3D Position Detection Network for Flowering Chinese Cabbage Based on YOLOv8.

Accurately detecting the maturity and 3D position of flowering Chinese cabbage (Brassica rapa var. chinensis) in natural environments is vital for autonomous robot harvesting in unstructured farms. The challenge lies in dense planting, small flower buds, similar colors and occlusions. This study proposes a YOLOv8-Improved network integrated with the ByteTrack tracking algorithm to achieve multi-object detection and 3D positioning of flowering Chinese cabbage plants in fields. In this study, C2F-MLCA is created by adding a lightweight Mixed Local Channel Attention (MLCA) with spatial awareness capability to the C2F module of YOLOv8, which improves the extraction of spatial feature information in the backbone network. In addition, a P2 detection layer is added to the neck network, and BiFPN is used instead of PAN to enhance multi-scale feature fusion and small target detection. Wise-IoU in combination with Inner-IoU is adopted as a new loss function to optimize the network for different quality samples and different size bounding boxes. Lastly, ByteTrack is integrated for video tracking, and RGB-D camera depth data are used to estimate cabbage positions. The experimental results show that YOLOv8-Improve achieves a precision (P) of 86.5% and a recall (R) of 86.0% in detecting the maturity of flowering Chinese cabbage. Among them, mAP50 and mAP75 reach 91.8% and 61.6%, respectively, representing an improvement of 2.9% and 4.7% over the original network. Additionally, the number of parameters is reduced by 25.43%. In summary, the improved YOLOv8 algorithm demonstrates high robustness and real-time detection performance, thereby providing strong technical support for automated harvesting management.

WTCL-Dehaze: Rethinking Real-World Image Dehazing via Wavelet Transform and Contrastive Learning

Images captured in hazy outdoor conditions often suffer from colour distortion, low contrast, and loss of detail, which impair high-level vision tasks. Single image dehazing is essential for applications such as autonomous driving and surveillance, with the aim of restoring image clarity. In this work, we propose WTCL-Dehaze an enhanced semi-supervised dehazing network that integrates Contrastive Loss and Discrete Wavelet Transform (DWT). We incorporate contrastive regularization to enhance feature representation by contrasting hazy and clear image pairs. Additionally, we utilize DWT for multi-scale feature extraction, effectively capturing high-frequency details and global structures. Our approach leverages both labelled and unlabelled data to mitigate the domain gap and improve generalization. The model is trained on a combination of synthetic and real-world datasets, ensuring robust performance across different scenarios. Extensive experiments demonstrate that our proposed algorithm achieves superior performance and improved robustness compared to state-of-the-art single image dehazing methods on both benchmark datasets and real-world images.

Wind Turbine Bearing Failure Diagnosis Using Multi-Scale Feature Extraction and Residual Neural Networks with Block Attention

Wind turbine rolling bearings are crucial components for ensuring the reliability and stability of wind power systems. Their failure can lead to significant economic losses and equipment downtime. Therefore, the accurate diagnosis of bearing faults is of great importance. Although existing deep learning fault diagnosis methods have achieved certain results, they still face limitations such as inadequate feature extraction capabilities, insufficient generalization to complex working conditions, and ineffective multi-scale feature capture. To address these issues, this paper proposes an advanced fault diagnosis method named the two-stream feature fusion convolutional neural network (TSFFResNet-Net). Firstly, the proposed method combines the advantages of one-dimensional convolutional neural networks (1D-ResNet) and two-dimensional convolutional neural networks (2D-ResNet). It transforms one-dimensional vibration signals into two-dimensional images through the empirical wavelet transform (EWT) method. Then, parallel convolutional kernels in 1D-ResNet and 2D-ResNet are used to extract multi-scale features, respectively. Next, the Convolutional Block Attention Module (CBAM) is introduced to enhance the network’s ability to capture key features by focusing on important features in specific channels or spatial areas. After feature fusion, CBAM is introduced again to further enhance the effect of feature fusion, ensuring that the features extracted by different network branches can be effectively integrated, ultimately providing more accurate input features for the classification task of the fully connected layer. The experimental results demonstrate that the proposed method outperforms other traditional methods and advanced convolutional neural network models on different datasets. Compared with convolutional neural network models such as LeNet-5, AlexNet, and ResNet, the proposed method achieves a significantly higher accuracy on the test set, with a stable accuracy of over 99%. Compared with other models, it shows better generalization and stability, effectively improving the overall performance of rolling bearing vibration signal fault diagnosis. The method provides an effective solution for the intelligent fault diagnosis of wind turbine rolling bearings.

A novel multi-scale network intrusion detection model with transformer

Network is an essential tool today, and the Intrusion Detection System (IDS) can ensure the safe operation. However, with the explosive growth of data, current methods are increasingly struggling as they often detect based on a single scale, leading to the oversight of potential features in the extensive traffic data, which may result in degraded performance. In this work, we propose a novel detection model utilizing multi-scale transformer namely IDS-MTran. In essence, the collaboration of multi-scale traffic features broads the pattern coverage of intrusion detection. Firstly, we employ convolution operators with various kernels to generate multi-scale features. Secondly, to enhance the representation of features and the interaction between branches, we propose Patching with Pooling (PwP) to serve as a bridge. Next, we design multi-scale transformer-based backbone to model the features at diverse scales, extracting potential intrusion trails. Finally, to fully capitalize these multi-scale branches, we propose the Cross Feature Enrichment (CFE) to integrate and enrich features, and then output the results. Sufficient experiments show that compared with other models, the proposed method can distinguish different attack types more effectively. Specifically, the accuracy on three common datasets NSL-KDD, CIC-DDoS 2019 and UNSW-NB15 has all exceeded 99%, which is more accurate and stable.

EFMF-pillars: 3D object detection based on enhanced features and multi-scale fusion

As unmanned vehicle technology advances rapidly, obstacle recognition and target detection are crucial links, which directly affect the driving safety and efficiency of unmanned vehicles. In response to the inaccurate localization of small targets such as pedestrians in current object detection tasks and the problem of losing local features in the PointPillars, this paper proposes a three-dimensional object detection method based on improved PointPillars. Firstly, addressing the issue of lost spatial and local information in the PointPillars, the feature encoding part of the PointPillars is improved, and a new pillar feature enhancement extraction module, CSM-Module, is proposed. Channel encoding and spatial encoding are introduced in the new pillar feature enhancement extraction module, fully considering the spatial information and local detailed geometric information of each pillar, thereby enhancing the feature representation capability of each pillar. Secondly, based on the fusion of CSPDarknet and SENet, a new backbone network CSE-Net is designed in this paper, enabling the extraction of rich contextual semantic information and multi-scale global features, thereby enhancing the feature extraction capability. Our method achieves higher detection accuracy when validated on the KITTI dataset. Compared to the original network, the improved algorithm’s average detection accuracy is increased by 3.42%, it shows that the method is reasonable and valuable.

Mutually-Guided Hierarchical Multi-Modal Feature Learning for Referring Image Segmentation

Referring image segmentation aims to locate and segment the target region based on a given textual expression query. The primary challenge is to understand semantics from visual and textual modalities and achieve alignment and matching. Prior works have attempted to address this challenge by leveraging separately pretrained unimodal models to extract global visual and textual features, and perform straightforwardly fusion to establish cross-modal semantic associations. However, these methods often concentrate solely on the global semantics, disregarding the hierarchical semantics of expression and image, and struggling with complex and open real scenarios, thus failing to capture critical cross-modal information. To address these limitations, this paper introduces an innovative mutually-guided hierarchical multi-modal feature learning scheme. By leveraging the guidance of global visual features, the model mines hierarchical text features from different stages of the text encoder. Simultaneously, the guidance of global textual features is leveraged to aggregate multi-scale visual features. This mutually guided hierarchical feature learning effectively addresses the semantically inaccurate cause by free-form text and naturally occurring scale variations. Furthermore, a Segment Detail Refinement (SDR) module is designed to enhance the model’s spatial detail awareness through attention mapping of low-level visual features and cross-modal features. To evaluate the effectiveness of the proposed approach, extensive experiments are conducted on three widely used referring image object segmentation datasets. The results demonstrate the superiority of the presented method in accurately locating and segmenting objects in images.

ADT: Person re-identification based on efficient attention mechanism and single-channel dual-channel fusion with transformer features aggregation

In person re-identification(Re-ID) tasks, using only convolutional neural network(CNNs) makes it challenging to fully exploit the global key information in person images, while using only the Transformer structure fails to adequately capture the multi-scale local detail information in person images. Therefore, to address the more challenging person Re-ID tasks, effectively integrating CNN and Transformer into an algorithmic framework capable of fusing multi-dimensional features has become one of the current focal points of research. Due to the ability of the attention mechanism to suppress the extraction of non-salient features by convolutional neural networks, and the capability of the dual-channel multi-scale mechanism to facilitate the extraction of finer-grained deep information by the Transformer, we propose a network Based on Efficient Attention Mechanism and Single-channel Dual-channel Fusion with Transformer Features Aggregation (ADT). First, we introduce the Convolutional Block Attention Module(CBAM) and Spatial Reconstruction Unit (SRU) into the CNN network to construct a Multi-dimensional Attention Features Re-fusion Network(MCF). By weighting the local features extracted in the convolutional network, attention to person detail features in the model is enhanced. Then, we propose the Single and Dual-channel Integrated Transformer Features Aggregator(SDT) and overlap it in MCF. This integration is designed to more effectively aggregate the multi-scale high-dimensional global features from each layer of the CNN network. Experimental results on the Market1501, DukeMTMC, MSMT17, and CUHK03 datasets demonstrate that our model significantly enriches features representation compared to most methods.

MG-Net: A fetal brain tissue segmentation method based on multiscale feature fusion and graph convolution attention mechanisms

Background and Objective:Fetal brain tissue segmentation provides foundational support for comprehensively understanding the neurodevelopment of normal and congenital disease-affected fetuses. Manual labeling is very time-consuming, and automated segmentation methods can greatly improve the efficiency of doctors. At the same time, fetal brain tissue undergoes various changes throughout the pregnancy, leading to a continuous change in tissue contrast, which greatly increases the difficulty of training segmentation methods. This study aims to develop an automated segmentation model that can efficiently and accurately segment fetal brain tissue, improving the workflow for medical professionals. Methods:We propose a novel deep learning-based segmentation model that incorporates three innovative components: Firstly, a new Dual Dilated Attention Block (DDAB) is proposed in the encoder part to enhance the feature extraction of local spatial and structural contextual information. Secondly, a Multi-scale Deformable Transformer (MSDT) is integrated into the bottleneck to improve the feature extraction of global information on local spatial and structural contextual information. Thirdly, we use a novel block based on Graph Convolution Attention (GCAB) in the decoder, which effectively enhances the features at the decoder.The code is available at https://github.com/unicoco7/MG-Net/. Results:We trained and tested on the FeTA 2021 and FeTA 2022 datasets, and evaluated using seven popular metrics, including Dice, IoU, MAE, BoundaryF, PRE, SEN, and SPE. Compared to the current state-of-the-art 3D segmentation models such as nnFormer, SwinUNETR, and 3DUX-net, our proposed method has surpassed all of them in metrics like Dice, IoU, and MAE. Specifically, on the FeTA 2021 dataset, our model achieved a Dice of 0.8666, an IoU of 0.7646, and an MAE of 0.0027; on the FeTA 2022 dataset, it achieved a Dice of 0.8552, an IoU of 0.7470, and an MAE of 0.0005. Conclusion:In this paper, we propose a model for three-dimensional fetal brain tissue segmentation based on multi-scale feature fusion and graph convolution attention mechanism, and conduct experimental evaluation on the FeTA 2021 and FeTA 2022 datasets. Understanding the boundaries of fetal brain tissue is crucial for doctors’ diagnosis, so the proposed model is expected to improve the speed and accuracy of doctors’ diagnoses.

Faceify- Human Face Sketch to Real

The Human Face Sketch to Real Image Application using deep learning presents the development of an innovative application designed to convert human face sketches into realistic image using advanced deep learning techniques. The primary objective of this research is to bridge the gap between manual sketching and digital realism, offering a powerful tool for artists, law enforcement, and digital content creators. The proposed system leverages generative adversarial networks (GANs), particularly a variant known as the Sketch-to-Image GAN (SI-GAN), which is optimizes for translating sparse and abstract sketch lines into high-fidelity facial representations. The architecture consists of a generator network that refines sketches into realistic images, and a discriminator network that distinguishes between generated images and real photographs, continuously enhancing the model’s output. To ensure high accuracy and detail preservation, the model is trained on a diverse dataset comprising thousands of paired sketch and real face images. Advanced techniques such as attention mechanisms and multi-scale feature extraction are incorporated to maintain the integrity of facial features and textures. Additionally, a user friendly interface has been developed, allowing users to input sketches and receive high-quality realistic images in real-time. Evaluation metrics, including structural similarity index (SSIM) and Frechet Inception Distance (FID), Indicate that the proposed model outperforms existing state-of-the-art methods in both quality and realism. Keywords: Image Translation, Conditional Generative Adversarial Networks (CGANs), Deep Learning, Mobile Application Development, Backend Integration, Face Sketch, Photorealistic Image.

Enhancing Small Target Detection in Aerial Imagery with Bi-PAN-FPN and EDMOA-Optimized YOLOv8-s

Across the globe, people are working to build "smart cities" that will employ technology to make people's lives better and safer. Installing cameras at strategic spots across the city to monitor public spaces besides provide real-time footage to law enforcement besides other local authorities is a crucial part of smart city infrastructure, which includes video surveillance. A more effective answer is provided by deep learning algorithms, however research in this area still faces significant problems from changes in target size, form change, occlusion, and illumination circumstances as seen from the drone's perspective. In light of the aforementioned issues, this study presents a highly effective and resilient approach for aerial picture identification. To begin, the concept of Bi-PAN-FPN is presented to enhance the neck component of YOLOv8-s, taking into consideration the prevalent issue of small targets being easily misdetected or ignored in aerial photos. We achieve a more advanced and thorough feature fusion procedure much as feasible by completely considering and reusing multiscale features. To further reduce the amount of parameters in the model and prevent info loss during long-distance feature transfer, the benchmark model's backbone incorporates the GhostblockV2 structure in lieu of a portion of the C2f module. With the help of the Enhanced Dwarf Mongoose Optimization Algorithm (EDMOA), the suggested model's hyper-parameters are optimised. Lastly, a dynamic nonmonotonic focusing mechanism is employed in conjunction with WiseIoU loss as bounding box regression loss. The detector accounts for varying anchor box quality by utilizing "outlier" evaluations, thus improving the complete presentation of the detection task.

SFINet: A semantic feature interactive learning network for full-time infrared and visible image fusion

Infrared and visible image fusion aims to combine data from various source images to generate a high-quality image. Nevertheless, numerous fusion methods often prioritize visual quality above semantic information. To address this problem, we present a Semantic Feature Interactive Learning Network (SFINet) for full-time infrared and visible images. The SFINet encompasses an image fusion network and an image segmentation network through a Semantic Feature Interaction (SFI) module. The image fusion network employs Multi-scale Feature Extraction (MFE) modules to capture global and local information at multiple scales. Meanwhile, it performs an adaptive fusion of complementary information using a Dual Attention Feature Fusion (DAFF) module. The image segmentation network guides the image fusion network using the SFI module for semantic feature interaction. Comparative results prove that the proposed method is superior to state-of-the-art (SOTA) models in image fusion and semantic segmentation tasks. The code is available at https://github.com/songwenhao123/SFINet.

Weak Edge Target Segmentation Network Based on Dual Attention Mechanism

Segmentation of weak edge targets such as glass and plastic poses a challenge in the field of target segmentation. The detection process is susceptible to background interference and various external factors due to the transparent nature of these materials. To address this issue, this paper introduces a segmentation network for weak edge target objects (WETS-Net). To effectively extract edge information of such objects and eliminate redundant information during feature extraction, a dual-attention mechanism is employed, including the Edge Attention Extraction Module (EAEM) and the Multi-Scale Information Fusion Module (MIFM). Specifically, the EAEM combines improved edge feature extraction kernels to selectively enhance the importance of edge features, aiding in more precise target region extraction. The MIFM utilizes spatial attention mechanisms to fuse multi-scale features, reducing background and external interference. These innovations enhance the performance of WETS-Net, offering a new direction for weak edge target segmentation research. Finally, through ablation experiments, the effectiveness of each module is effectively validated. Moreover, the proposed algorithm achieves an average detection accuracy of 95.83% and 96.13% on the dataset and a self-made dataset, respectively, outperforming similar U-Net-improved networks.

Polycrystalline silicon photovoltaic cell defects detection based on global context information and multi-scale feature fusion in electroluminescence images

In photovoltaic (PV) cell inspection, electroluminescence (EL) imaging provides high spatial resolution for detecting various types of defects. The recent integration of EL imaging with deep learning models has enhanced the recognition of defects in PV cells. However, the high surface impurity content in polycrystalline silicon PV cells presents a challenge. Defect features can be similar to complex background textures, making highlighting features for small target defects difficult and leading to potential misclassification as background or other classes. To address these challenges, we propose a novel deep convolutional neural network (CNN) model for effectively identifying small target defects in polycrystalline PV cells. We first utilize a global context information (GCI) block to improve CNN’s modeling of global information, aiding in distinguishing PV cell defects with similar local details. Moreover, we design a channel weight feature pyramid (CWFP) that dynamically adjusts the channel weights of extracted features. We further achieve multi-scale feature fusion through the pyramid structure to enhance the resolution capability for small targets. The proposed model was evaluated on a publicly available dataset of 8 defect classes in polycrystalline PV cells, achieving an accuracy of 96.36 %. The experimental results show that the proposed model outperforms existing deep learning models in detecting small target defects with higher precision.

Segmentation-assisted classification model with convolutional neural network for weld defect detection

Detecting weld defects in battery trays is crucial for the safety of new energy vehicles. Existing methods for weld surface defect detection, relying on traditional computer vision algorithms and convolutional neural networks with substantial image-level labeled data, face challenges in accurately identifying small defects, especially with limited samples. To address these issues, we developed an innovative Segmentation-Assisted Classification with Convolutional Neural Networks (SACNN) model. SACNN integrates a common feature extraction subnet, a segmentation subnet enhanced by a multi-scale feature fusion module, and a classification subnet specifically designed for precise defect detection. A joint loss function co-trains the segmentation and classification subnets using both image-level and pixel-level labels, enhancing the model’s ability to accurately detect small defect regions. Our model demonstrates notable improvement, achieving accuracy gains ranging from 2% to 18% compared to existing state-of-the-art methods, with an overall accuracy of 94.09% on an industrial dataset of battery tray welds. To further evaluate the generalization capability of our model, we evaluated it on the publicly available Magnetic Tile dataset, achieving state-of-the-art results in this challenging context. Additionally, we conducted comprehensive ablation studies to validate the contribution of each component in our approach and utilized visualization techniques to enhance the interpretability of our model. These advancements represent a significant contribution to the state of the art in aluminum alloy weld defect detection.

MS-CGAN: Fusion of conditional generative adversarial networks and multi-scale spatio-temporal features for lithology identification

Lithology identification constitutes a crucial undertaking in formation evaluation and reservoir characterization. However, the need for improved precision arises in conventional lithology identification models due to the difficulties presented by unequal distributions of small-sample logging data. An effective combination of domain expertise and data-driven models to predict lithology is essential due to the intricate and nonlinear connection between logging parameters and lithology, combined with the distinct characteristics of the oilfield environments. In this paper, we proposed a multi-scale conditional generative adversarial network(MS-CGAN) method, which combines conditional generative adversarial networks with multi-scale spatio-temporal features to address data imbalance issues and enhance the accuracy of lithology classification. Our approach, tested on two small datasets from the Hugoton and Panoma fields, USA, and the Daqing production wells, China, stands out as the optimal choice compared to other models. Comprehensive evaluation results indicate promising practical applications and potential benefits of the new model in enhancing lithology identification using limited data.

Lightweight algorithm based on you only look once version 5 for multiple class defect detection on wind turbine blade surfaces

A lightweight wind turbine blade surface multiple class defect detection algorithm based on You Only Look Once version 5 (YOLOv5) is proposed to address the issues of low accuracy, poor efficiency, and high power consumption associated with manual and traditional automation algorithms used for detecting surface defects on wind turbine blades. Firstly, by replacing the Concentrated-Comprehensive Convolution Block module in YOLOv5 with the GhostBottleneckv2 module, the overall parameter count of the model is significantly reduced. Additionally, advocating the inclusion of the Convolutional Block Attention Module (CBAM) into the YOLOv5 backbone network to address the issue of small targets and significant scale variations in surface defects found on wind turbine blades. The YOLOv5 feature fusion module replaces the Feature Pyramid Network (FPN) structure with the Bidirectional Feature Pyramid Network (BiFPN) structure to enhance the fusion capability of multi-scale weighted features. Finally, the optimized model files underwent structured pruning through the application of pruning strategies, retraining fine-tuning, and knowledge distillation techniques. The experimental results on the self-made dataset of surface defects on wind turbine blades indicate that the optimized model has a parameter count that is only 48.79% of the small version of the YOLOv5 model. However, its mean Average Precision (mAP) and frame rate have increased by 5.05% and 8.1 frames/s, respectively, reaching 94.56% and 43.7 frames/s. The lightweight algorithm proposed in this study meets the accuracy and real-time requirements for surface defect detection on wind turbine blades.

Beyond Granularity: Enhancing Continuous Sign Language Recognition with Granularity-Aware Feature Fusion and Attention Optimization

The advancement of deep learning techniques has significantly propelled the development of the continuous sign language recognition (cSLR) task. However, the spatial feature extraction of sign language videos in the RGB space tends to focus on the overall image information while neglecting the perception of traits at different granularities, such as eye gaze and lip shape, which are more detailed, or posture and gestures, which are more macroscopic. Exploring the efficient fusion of visual information of different granularities is crucial for accurate sign language recognition. In addition, applying a vanilla Transformer to sequence modeling in cSLR exhibits weak performance because specific video frames could interfere with the attention mechanism. These limitations constrain the capability to understand potential semantic characteristics. We introduce a feature fusion method for integrating visual features of disparate granularities and refine the metric of attention to enhance the Transformer’s comprehension of video content. Specifically, we extract CNN feature maps with varying receptive fields and employ a self-attention mechanism to fuse feature maps of different granularities, thereby obtaining multi-scale spatial features of the sign language framework. As for video modeling, we first analyze why the vanilla Transformer failed in cSLR and observe that the magnitude of the feature vectors of video frames could interfere with the distribution of attention weights. Therefore, we utilize the Euclidean distance among vectors to measure the attention weights instead of scaled-dot to enhance dynamic temporal modeling capabilities. Finally, we integrate the two components to construct the model MSF-ET (Multi-Scaled feature Fusion–Euclidean Transformer) for cSLR and train the model end-to-end. We perform experiments on large-scale cSLR benchmarks—PHOENIX-2014 and Chinese Sign Language (CSL)—to validate the effectiveness.