Multi-scale Features Research Articles

Self-supervised monocular depth estimation based on improved dense network and wavelet decomposition

Traditional self-supervised monocular depth estimation models do not adequately extract and fuse shallow features, which can easily lead to problems such as missed detection of small objects and blurred object edges. To address the above problems, this paper proposes a self-supervised monocular depth estimation model based on improved dense network and wavelet decomposition. The model follows the structure of U-net, in which the encoder adopts an improved dense connection method to improve the feature extraction and fusion capabilities of the encoder; at the same time, a detail enhancement module is added to the jump connection to further refine and integrate the multi-scale features output by the encoder; finally, the wavelet decomposition method is introduced in the decoder, forcing the decoder to pay more attention to high-frequency information and achieve refined processing of image edges. Experimental results show that the depth estimation model proposed in this paper has a stronger ability to capture small object features, and the edges of the generated depth map are clearer and more accurate.

AI-based automated breast cancer segmentation in ultrasound imaging based on Attention Gated Multi ResU-Net

Breast cancer is a leading cause of death among women worldwide, making early detection and diagnosis critical for effective treatment and improved patient outcomes. Ultrasound imaging is a common diagnostic tool for breast cancer, but interpreting ultrasound images can be challenging due to the complexity of breast tissue and the variability of image quality. This study proposed an Attention Gated Multi ResU-Net model for medical image segmentation tasks, that has shown promising results for breast cancer ultrasound image segmentation. The model’s multi-scale feature extraction and attention-gating mechanism enable it to accurately identify and segment areas of abnormality in the breast tissue, such as masses, cysts, and calcifications. The model’s quantitative test showed an adequate degree of agreement with expert manual annotations, demonstrating its potential for improving early identification and diagnosis of breast cancer. The model’s multi-scale feature extraction and attention-gating mechanism enable it to accurately identify and segment areas of abnormality in the breast tissue, such as masses, cysts, and calcifications, achieving a Dice coefficient of 0.93, sensitivity of 93%, and specificity of 99%. These results underscore the model’s high precision and reliability in medical image analysis.

Welding Defect Monitoring Based on Multi-Scale Feature Fusion of Molten Pool Videos.

Real-time quality monitoring through molten pool images is a critical focus in researching high-quality, intelligent automated welding. However, challenges such as the dynamic nature of the molten pool, changes in camera perspective, and variations in pool shape make defect detection using single-frame images difficult. We propose a multi-scale fusion method for defect monitoring based on molten pool videos to address these issues. This method analyzes the temporal changes in light spots on the molten pool surface, transferring features between frames to capture dynamic behavior. Our approach employs multi-scale feature fusion using row and column convolutions along with a gated fusion module to accommodate variations in pool size and position, enabling the detection of light spot changes of different sizes and directions from coarse to fine. Additionally, incorporating mixed attention with row and column features enables the model to capture the characteristics of the molten pool more efficiently. Our method achieves an accuracy of 97.416% on a molten pool video dataset, with a processing time of 16 ms per sample. Experimental results on the UCF101-24 and JHMDB datasets also demonstrate the method's generalization capability.

A Study on Enhancing the Visual Fidelity of Aviation Simulators Using WGAN-GP for Remote Sensing Image Color Correction

When implementing outside-the-window (OTW) visuals in aviation tactical simulators, maintaining terrain image color consistency is critical for enhancing pilot immersion and focus. However, due to various environmental factors, inconsistent image colors in terrain can cause visual confusion and diminish realism. To address these issues, a color correction technique based on a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) is proposed. The proposed WGAN-GP model utilizes multi-scale feature extraction and Wasserstein distance to effectively measure and adjust the color distribution difference between the input image and the reference image. This approach can preserve the texture and structural characteristics of the image while maintaining color consistency. In particular, by converting Bands 2, 3, and 4 of the BigEarthNet-S2 dataset into RGB images as the reference image and preprocessing the reference image to serve as the input image, it is demonstrated that the proposed WGAN-GP model can handle large-scale remote sensing images containing various lighting conditions and color differences. The experimental results showed that the proposed WGAN-GP model outperformed traditional methods, such as histogram matching and color transfer, and was effective in reflecting the style of the reference image to the target image while maintaining the structural elements of the target image during the training process. Quantitative analysis demonstrated that the mid-stage model achieved a PSNR of 28.93 dB and an SSIM of 0.7116, which significantly outperforms traditional methods. Furthermore, the LPIPS score was reduced to 0.3978, indicating improved perceptual similarity. This approach can contribute to improving the visual elements of the simulator to enhance pilot immersion and has the potential to significantly reduce time and costs compared to the manual methods currently used by the Republic of Korea Air Force.

CEDNet: A cascade encoder–decoder network for dense prediction

The prevailing methods for dense prediction tasks typically utilize a heavy classification backbone to extract multi-scale features and then fuse these features using a lightweight module. However, these methods allocate most computational resources to the classification backbone, which delays the multi-scale feature fusion and potentially leads to inadequate feature fusion. Although some methods perform feature fusion from early stages, they either fail to fully leverage high-level features to guide low-level feature learning or have complex structures, resulting in sub-optimal performance. We propose a streamlined cascade encoder–decoder network, named CEDNet, tailored for dense prediction tasks. All stages in CEDNet share the same encoder–decoder structure and perform multi-scale feature fusion within each decoder, thereby enhancing the effectiveness of multi-scale feature fusion. We explored three well-known encoder–decoder structures: Hourglass, UNet, and FPN, all of which yielded promising results. Experiments on various dense prediction tasks demonstrated the effectiveness of our method.11Code: https://github.com/zhanggang001/CEDNet.

Multistage Spatial-Spectral Fusion Network for Spectral Super-Resolution.

Spectral super-resolution (SSR) aims to restore a hyperspectral image (HSI) from a single RGB image, in which deep learning has shown impressive performance. However, the majority of the existing deep-learning-based SSR methods inadequately address the modeling of spatial-spectral features in HSI. That is to say, they only sufficiently capture either the spatial correlations or the spectral self-similarity, which results in a loss of discriminative spatial-spectral features and hence limits the fidelity of the reconstructed HSI. To solve this issue, we propose a novel SSR network dubbed multistage spatial-spectral fusion network (MSFN). From the perspective of network design, we build a multistage Unet-like architecture that differentially captures the multiscale features of HSI both spatialwisely and spectralwisely. It consists of two types of the self-attention mechanism, which enables the proposed network to achieve global modeling of HSI comprehensively. From the perspective of feature alignment, we innovatively design the spatial fusion module (SpatialFM) and spectral fusion module (SpectralFM), aiming to preserve the comprehensively captured spatial correlations and spectral self-similarity. In this manner, the multiscale features can be better fused and the accuracy of reconstructed HSI can be significantly enhanced. Quantitative and qualitative experiments on the two largest SSR datasets (i.e., NTIRE2022 and NTIRE2020) demonstrate that our MSFN outperforms the state-of-the-art SSR methods. The code implementation will be uploaded at https://github.com/Matsuri247/MSFN-for-Spectral-Super-Resolution.

YOLO-RWY: A Novel Runway Detection Model for Vision-Based Autonomous Landing of Fixed-Wing Unmanned Aerial Vehicles

In scenarios where global navigation satellite systems (GNSSs) and radio navigation systems are denied, vision-based autonomous landing (VAL) for fixed-wing unmanned aerial vehicles (UAVs) becomes essential. Accurate and real-time runway detection in VAL is vital for providing precise positional and orientational guidance. However, existing research faces significant challenges, including insufficient accuracy, inadequate real-time performance, poor robustness, and high susceptibility to disturbances. To address these challenges, this paper introduces a novel single-stage, anchor-free, and decoupled vision-based runway detection framework, referred to as YOLO-RWY. First, an enhanced data augmentation (EDA) module is incorporated to perform various augmentations, enriching image diversity, and introducing perturbations that improve generalization and safety. Second, a large separable kernel attention (LSKA) module is integrated into the backbone structure to provide a lightweight attention mechanism with a broad receptive field, enhancing feature representation. Third, the neck structure is reorganized as a bidirectional feature pyramid network (BiFPN) module with skip connections and attention allocation, enabling efficient multi-scale and across-stage feature fusion. Finally, the regression loss and task-aligned learning (TAL) assigner are optimized using efficient intersection over union (EIoU) to improve localization evaluation, resulting in faster and more accurate convergence. Comprehensive experiments demonstrate that YOLO-RWY achieves AP50:95 scores of 0.760, 0.611, and 0.413 on synthetic, real nominal, and real edge test sets of the landing approach runway detection (LARD) dataset, respectively. Deployment experiments on an edge device show that YOLO-RWY achieves an inference speed of 154.4 FPS under FP32 quantization with an image size of 640. The results indicate that the proposed YOLO-RWY model possesses strong generalization and real-time capabilities, enabling accurate runway detection in complex and challenging visual environments, and providing support for the onboard VAL systems of fixed-wing UAVs.

Contour Detection Algorithm for α Phase Structure of TB6 Titanium Alloy fused with Multi-Scale Fretting Features

Aiming at the problems of inaccuracy in detecting the α phase contour of TB6 titanium alloy. By combining computer vision technology with human vision mechanisms, the spatial characteristics of the α phase can be simulated to obtain the contour accurately. Therefore, an algorithm for α phase contour detection of TB6 titanium alloy fused with multi-scale fretting features is proposed. Firstly, through the response of the classical receptive field model based on fretting and the suppression of new non-classical receptive field model based on fretting, the information maps of the α phase contour of the TB6 titanium alloy at different scales are obtained; then the information map of the smallest scale contour is used as a benchmark, the neighborhood is constructed to judge the deviation of other scale contour information, and the corresponding weight value is calculated; finally, Gaussian function is used to weight and fuse the deviation information, and the contour detection result of TB6 titanium alloy α phase is obtained. In the Visual Studio 2013 environment, 484 metallographic images with different temperatures, strain rates, and magnifications were tested. The results show that the performance evaluation F value of the proposed algorithm is 0.915, which can effectively improve the accuracy of α phase contour detection of TB6 titanium alloy.

An Efficient UAV Image Object Detection Algorithm Based on Global Attention and Multi-Scale Feature Fusion

An Efficient UAV Image Object Detection Algorithm Based on Global Attention and Multi-Scale Feature Fusion

Semantic Segmentation of Satellite Images for Landslide Detection Using Foreground-Aware and Multi-Scale Convolutional Attention Mechanism.

Advancements in satellite and aerial imagery technology have made it easier to obtain high-resolution remote sensing images, leading to widespread research and applications in various fields. Remote sensing image semantic segmentation is a crucial task that provides semantic and localization information for target objects. In addition to the large-scale variation issues common in most semantic segmentation datasets, aerial images present unique challenges, including high background complexity and imbalanced foreground-background ratios. However, general semantic segmentation methods primarily address scale variations in natural scenes and often neglect the specific challenges in remote sensing images, such as inadequate foreground modeling. In this paper, we present a foreground-aware remote sensing semantic segmentation model. The model introduces a multi-scale convolutional attention mechanism and utilizes a feature pyramid network architecture to extract multi-scale features, addressing the multi-scale problem. Additionally, we introduce a Foreground-Scene Relation Module to mitigate false alarms. The model enhances the foreground features by modeling the relationship between the foreground and the scene. In the loss function, a Soft Focal Loss is employed to focus on foreground samples during training, alleviating the foreground-background imbalance issue. Experimental results indicate that our proposed method outperforms current state-of-the-art general semantic segmentation methods and transformer-based methods on the LS dataset benchmark.

New Energy Vehicle Tire Defect Detection Algorithm Based on Improved YOLOv8

Aiming at the traditional tire defect detection is difficult to meet the practical application requirements, a defect detection algorithm based on YOLOv8 is constructed. Firstly, the GAM attention mechanism is added to the backbone to improve the feature expression of tire defects and improve the detection efficiency. Secondly, the SPPF is replaced by SimSPPF to effectively capture multi-scale features. Finally, the CIoU loss function is replaced by Wise-IoU to improve the convergence ability of the model. The experimental results show that the constructed improved models P, mAP0.5 and mAP0.5:0.95 reach 86.4%, 88.2% and 46.9%, respectively, which are improved by 2.7%, 2.3% and 0.9% compared with the original YOLOv8n model. The algorithm can better improve the problems faced by traditional new energy vehicle tire defect detection.

Classification and extraction method of hidden dangers along railway lines based on semantic segmentation network

ABSTRACT Foreign objects invading high-speed railway lines can cause danger. One existing solution is to use remote sensing images to analyse the dangerous areas along the railway line, thereby providing a certain amount of investigation time. Considering the spatial and temporal resolution characteristics of existing remote sensing technologies in identifying floating objects and the reality of rapid land use changes, this paper identifies areas on the ground where floating objects may be generated by using semantic segmentation techniques oriented to remotely sensed imagery and provides early warnings to staff along the route. However, these regions that need to be analysed have different semantics and scales. To address these challenges, this paper proposes a Dual-branch Parallel Fusion Network (DPFNet) based on Transformer, aimed at enhancing multi-class semantic segmentation in remote sensing images. To leverage global contextual information, we introduce a Swin Transformer-based backbone network, which employs self-attention to capture a comprehensive scene context, facilitating better segmentation by considering the entire scene’s context. For multi-scale semantic features, we propose one approach that involves independent branching feature expression and a Multi-scale Feature Space Fusion Module (MFSFM). The former enriches multi-scale information, while the latter fuses features across different levels to capture diverse semantic features. Experimental results demonstrate that DPFNet can effectively identify the hidden danger area, and the fusion of multi-scale features makes the network more accurately identify and segment the risk area of different sizes, improving the segmentation accuracy and robustness, and is of great significance to the formation of the ‘prevention’ as the core of the railway safety operation.

MSE-TCN: Multi-scale temporal convolutional network with channel attention for open-set gas classification

Currently, researches on closed-set gas classification tasks has achieved great success in the electronic nose (E-nose) field. E-nose, however, faces a more challenging and realistic task in open-set gas classification. To find an accurate open-set gas classification model, we proposed a MSE-TCN, which integrates squeeze-and-excitation residual network (SE-ResNet) internally into the temporal convolutional network (TCN) and expands the channels number of TCN, forming a multi-scale feature extraction encoder, and realizes open-set gas classification using the OpenMax algorithm. The underlying SE-ResNet module focuses on the relatively important sensor channels, while Multi-scale TCN thoroughly captures temporal relationships between the E-nose data from three scales, and the OpenMax identifies the unknown classes by redistributing the probability. The gas sensing performance of the sensors used in the self-sampling dataset was analyzed, demonstrating the reliability of the data acquisition. Subsequently, comparative experiments based on self-sampling dataset and public dataset were performed to determine the number of encoders and demonstrate the necessity of the SE-ResNet module. Meanwhile, ablation experiments demonstrate the effectiveness of our proposed model. In addition, the comparative experiments of the open-set classifiers show MSE-TCN achieves the highest accuracies of 0.9245 and 0.9387 among all the models on both datasets for open-set classification, respectively. As a result, this model provides an effective method for high accuracy gas classification for both closed-set and open-set in E-nose field.

A Lightweight Strip Steel Surface Defect Detection Network Based on Improved YOLOv8

Strip steel surface defect detection has become a crucial step in ensuring the quality of strip steel production. To address the issues of low detection accuracy and long detection times in strip steel surface defect detection algorithms caused by varying defect sizes and blurred images during acquisition, this paper proposes a lightweight strip steel surface defect detection network, YOLO-SDS, based on an improved YOLOv8. Firstly, StarNet is utilized to replace the backbone network of YOLOv8, achieving lightweight optimization while maintaining accuracy. Secondly, a lightweight module DWR is introduced into the neck and combined with the C2f feature extraction module to enhance the model’s multi-scale feature extraction capability. Finally, an occlusion-aware attention mechanism SEAM is incorporated into the detection head, enabling the model to better capture and process features of occluded objects, thus improving performance in complex scenarios. Experimental results on the open-source NEU-DET dataset show that the improved model reduces parameters by 34.4% compared with the original YOLOv8 algorithm while increasing average detection accuracy by 1.5%. And it shows good generalization performance on the deepPCB dataset. Compared with other defect detection models, YOLO-SDS offers significant advantages in terms of parameter count and detection speed. Additionally, ablation experiments validate the effectiveness of each module.

ZeroD-fender: A Resource-aware IoT Malware Detection Engine via Fine-grained Side-channel Analysis

In early 2023, cyberattacks experienced a significant rise due to unknown (zero-day) malware targeting Internet of Things (IoT) devices. To tackle the challenge of zero-day detection within a highly resource-constrained IoT environment, we propose a novel design that utilizes fine-grained power side-channel analysis with deep learning techniques. Our approach introduces an innovative concept called multiscale feature extraction to identify the most representative malware features across diverse architectures, thereby enhancing deep learning based detection performance against zero-day malware. Specifically, we employ a fine-grained power side-channel analysis of more than 120,000 honeypot-collected malware files across a hierarchy of commands , functions , and modules to identify the unique zero-day malware behaviors. With these identified features to train our model, ZeroD-fender’s performance in detecting zero-day malware has significantly improved. In pursuit of on-device detection, we present a resource-aware online inference customization framework. This framework features our lightweight network, ThingNetV2, which uses specialized 1-D depthwise separable convolution paired with h-swish activation, leading to significant resource savings. By applying the fine-grained power analysis, ZeroD-fender demonstrates a detection rate of 95.88% across various architecture zero-day malware, achieving detection speeds ranging between 16.083 ms and 23.961 ms , depending on the specific scenario.

Sparse-View Photoacoustic Reconstruction Method for Diabetic Retinopathy Using Feature Fusion Network.

Diabetic retinopathy is one of the most prevalent microvascular complications of diabetes mellitus, and photoacoustic imaging is an effective method for imaging diabetic retinal vessels. Photoacoustic imaging is an emerging noninvasive imaging method based on the photoacoustic effect, which offers advantages of contrast, resolution, and depth imaging. Appropriate photoacoustic reconstruction methods are essential for obtaining high-quality photoacoustic images. In this study, a multi-input self-attention multiscale feature fusion network (SAMF-Net) is proposed for photoacoustic reconstruction. The algorithm accepts two inputs, namely the original photoacoustic signal and the traditional reconstructed image. Furthermore, a global feature extraction module based on the self-attention mechanism is employed to focus on the global information. The results demonstrate that the proposed method exhibits superior reconstruction capability under different sparse detection views. The method has instructive value for photoacoustic image reconstruction and has the potential for further application in the diagnosis of diabetic retinopathy.

Image Super-Resolution Reconstruction Based on Dense Residual Attention and Multi-Scale Feature Fusion

In order to obtain super-resolution images with richer details and clearer textures, a method for image super-resolution reconstruction using dense residual attention and multi-scale fusion is proposed. First, different scale convolutions are used to fully extract shallow features of the image; then high-frequency features of the image are extracted through one three-layer cascaded multi-scale feature fusion and dense residual attention module, and the reuse of feature map is achieved; finally, residual branches are used to introduce shallow features and high-frequency features of each channel image, and the high-resolution images are reconstructed through up-sampling and sub-pixel convolution. The test results on the Set5, Set14, Bsd100, and Urban100 datasets show that the PSNR and SSIM of our model are superior to most current algorithms, especially in the case of [Formula: see text] reconstruction results. PSNR has improved by 0.2 dB on the Set5 and Bsd100 datasets, and the algorithm has a better subjective visual effect.

Multi-Scale Feature Attention Fusion for Image Splicing Forgery Detection

Image splicing is a widely occurrence image tampering technology. With the rapid development of digital image processing technology, detecting image splicing forgery has become significantly challenging. Although various methods have been devised to identify such tampered images, existing approaches have not achieved optimal performance due to limitations in effectively leveraging feature maps of different scales. To address this issue, we propose a novel method for image splicing forgery detection called multi-scale feature attention fusion network (MFAF-Net). We propose a multi-scale atrous feature attention (MAFA) module designed to capture rich contextual features for multi-scale high-level feature fusion. Additionally, we present the multi-branch attention mechanism (MBAM) module to fuse contextual information from various branches for low-level features. This integration enhances the capability of low-level features to produce more refined pixel-level attention. We employ the weighted binary cross-entropy loss and dice loss in the MFAF-Net to overcome the imbalance between positive and negative samples. Extensive experiments demonstrate that the proposed MFAF-Net outperforms state-of-the-art methods. Robustness experiments also show our model exhibits image splicing forgery detection robustness under common attacks.

DLCH-YOLO: An Object Detection Algorithm for Monitoring the Operation Status of Circuit Breakers in Power Scenarios

In the scenario of power system monitoring, detecting the operating status of circuit breakers is often inaccurate due to variable object scales and background interference. This paper introduces DLCH-YOLO, an object detection algorithm aimed at identifying the operating status of circuit breakers. Firstly, we propose a novel C2f_DLKA module based on Deformable Large Kernel Attention. This module adapts to objects of varying scales within a large receptive field, thereby more effectively extracting multi-scale features. Secondly, we propose a Semantic Screening Feature Pyramid Network designed to fuse multi-scale features. By filtering low-level semantic information, it effectively suppresses background interference to enhance localization accuracy. Finally, the feature extraction network incorporates Generalized-Sparse Convolution, which combines depth-wise separable convolution and channel mixing operations, reducing computational load. The DLCH-YOLO algorithm achieved a 91.8% mAP on our self-built power equipment dataset, representing a 4.7% improvement over the baseline network Yolov8. With its superior detection accuracy and real-time performance, DLCH-YOLO outperforms mainstream detection algorithms. This algorithm provides an efficient and viable solution for circuit breaker status detection.

Contrastive Learning based Speech Spoofing Detection for Multimedia Security in Edge Intelligence

Artificial intelligence (AI) empowered edge computing has given rise to a new paradigm and effectively facilitated the promotion and development of multimedia applications. The speech assistant is one of the significant services provided by multimedia applications, which aims to offer intelligent interactive experiences between humans and machines. However, malicious attackers may exploit spoofed speeches to deceive speech assistants, posing great challenges to the security of multimedia applications. The limited resources of multimedia terminal devices hinder their ability to effectively load speech spoofing detection models. Furthermore, processing and analyzing speech in the cloud can result in poor real-time performance and potential privacy risks. Existing speech spoofing detection methods rely heavily on annotated data and exhibit poor generalization capabilities for unseen spoofed speeches. To address these challenges, this paper first proposes the Coordinate Attention Network (CA2Net) that consists of coordinate attention blocks and Res2Net blocks. CA2Net can simultaneously extract temporal and spectral speech feature information and represent multi-scale speech features at a granularity level. Besides, a contrastive learning-based speech spoofing detection framework named GEMINI is proposed. GEMINI can be effectively deployed on edge nodes and autonomously learn speech features with strong generalization capabilities. GEMINI first performs data augmentation on speech signals and extracts conventional acoustic features to enhance the feature robustness. Subsequently, GEMINI utilizes the proposed CA2Net to further explore the discriminative speech features. Then, a tensor-based multi-attention comparison model is employed to maximize the consistency between speech contexts. GEMINI continuously updates CA2Net with contrastive learning, which enables CA2Net to effectively represent speech signals and accurately detect spoofed speeches. Extensive experiments on the ASVspoof2019 dataset show that GEMINI reduces the Equal Error Rate and tandem Detection Cost Function by up to 96.75% and 96.35% in the physical access scenario, and by up to 86.62% and 87.71% in the logical access scenario compared to peer methods.