Local Details Research Articles

GMmorph: dynamic spatial matching registration model for 3D medical image based on gated Mamba.

Deformable registration aims to achieve nonlinear alignment of image space by estimating a dense displacement field. It is commonly used as a preprocessing step in clinical and image analysis applications, such as surgical planning, diagnostic assistance, and surgical navigation. We aim to overcome these challenges: Deep learning-based registration methods often struggle with complex displacements and lack effective interaction between global and local feature information. They also neglect the spatial position matching process, leading to insufficient registration accuracy and reduced robustness when handling abnormal tissues.
Approach: We propose a dual-branch interactive registration model architecture from the perspective of spatial matching. Implicit regularization is achieved through a consistency loss, enabling the network to balance high accuracy with a low folding rate. We introduced the Dynamic Matching Module (DMM) between the two branches of the registration, which generates learnable offsets based on all the tokens across the entire resolution range of the base branch features. Using trilinear interpolation, the model adjusts its feature expression range according to the learned offsets, capturing highly flexible positional differences. To facilitate the spatial matching process, we designed the Gated Mamba Layer (GML) to globally model pixel-level features by associating all voxel information, while the Detail Enhancement Module (DEM), which is based on channel and spatial attention, enhances the richness of local feature details.
Main results: Our study explores the model's performance in single-modal and multi-modal image registration, including normal brain, brain tumor, and lung images. We propose unsupervised and semi-supervised registration modes and conduct extensive validation experiments. The results demonstrate that the model achieves state-of-the-art performance across multiple datasets..
Significance: By introducing a novel perspective of position matching, the model achieves precise registration of various types of medical data, offering significant clinical value in medical applications.&#xD.

Power Converter Fault Detection Using MLCA–SpikingShuffleNet

With the widespread adoption of electric vehicles, the power converter, as a key component, plays a crucial role. Traditional fault detection methods often face challenges in real-time performance and computational efficiency, making it difficult to meet the demands of electric vehicle power converters for efficient and accurate fault diagnosis. To address this challenge, this paper proposes a novel fault detection model—SpikingShuffleNet. This paper first designs an efficient SpikingShuffle Unit that integrates grouped convolutions and channel shuffle techniques, effectively reducing the model’s computational complexity by optimizing feature extraction and channel interaction. Next, by appropriately stacking SpikingShuffle Units and refining the network architecture, a complete lightweight diagnostic network is constructed for real-time fault detection in electric vehicle power converters. Finally, the Mixed Local Channel Attention mechanism is introduced to address the potential limitations in feature representation caused by grouped convolutions, further enhancing fault detection accuracy and robustness by balancing local detail preservation and global feature integration. Experimental results show that SpikingShuffleNet exhibits excellent accuracy and robustness in the fault detection task for power converters, fulfilling the real-time fault diagnosis requirements for low-power embedded devices.

Bird Species Detection Net: Bird Species Detection Based on the Extraction of Local Details and Global Information Using a Dual-Feature Mixer.

Bird species detection is critical for applications such as the analysis of bird population dynamics and species diversity. However, this task remains challenging due to local structural similarities and class imbalances among bird species. Currently, most deep learning algorithms focus on designing local feature extraction modules while ignoring the importance of global information. However, this global information is essential for accurate bird species detection. To address this limitation, we propose BSD-Net, a bird species detection network. BSD-Net efficiently learns local and global information in pixels to accurately detect bird species. BSD-Net consists of two main components: a dual-branch feature mixer (DBFM) and a prediction balancing module (PBM). The dual-branch feature mixer extracts features from dichotomous feature segments using global attention and deep convolution, expanding the network's receptive field and achieving a strong inductive bias, allowing the network to distinguish between similar local details. The prediction balance module balances the difference in feature space based on the pixel values of each category, thereby resolving category imbalances and improving the network's detection accuracy. The experimental results using two public benchmarks and a self-constructed Poyang Lake Bird dataset demonstrate that BSD-Net outperforms existing methods, achieving 45.71% and 80.00% mAP50 with the CUB-200-2011 and Poyang Lake Bird datasets, respectively, and 66.03% AP with FBD-SV-2024, allowing for more accurate location and species information for bird detection tasks in video surveillance.

End-to-End Multi-Scale Adaptive Remote Sensing Image Dehazing Network.

Satellites frequently encounter atmospheric haze during imaging, leading to the loss of detailed information in remote sensing images and significantly compromising image quality. This detailed information is crucial for applications such as Earth observation and environmental monitoring. In response to the above issues, this paper proposes an end-to-end multi-scale adaptive feature extraction method for remote sensing image dehazing (MSD-Net). In our network model, we introduce a dilated convolution adaptive module to extract global and local detail features of remote sensing images. The design of this module can extract important image features at different scales. By expanding convolution, the receptive field is expanded to capture broader contextual information, thereby obtaining a more global feature representation. At the same time, a self-adaptive attention mechanism is also used, allowing the module to automatically adjust the size of its receptive field based on image content. In this way, important features suitable for different scales can be flexibly extracted to better adapt to the changes in details in remote sensing images. To fully utilize the features at different scales, we also adopted feature fusion technology. By fusing features from different scales and integrating information from different scales, more accurate and rich feature representations can be obtained. This process aids in retrieving lost detailed information from remote sensing images, thereby enhancing the overall image quality. A large number of experiments were conducted on the HRRSD and RICE datasets, and the results showed that our proposed method can better restore the original details and texture information of remote sensing images in the field of dehazing and is superior to current state-of-the-art methods.

Novel View Synthesis with Depth Priors Using Neural Radiance Fields and CycleGAN with Attention Transformer

Novel view synthesis aims to generate new perspectives from a limited number of input views. Neural Radiance Field (NeRF) is a key method for this task, and it produces high-fidelity images from a comprehensive set of inputs. However, a NeRF’s performance drops significantly with sparse views. To mitigate this, depth information can be used to guide training, with coarse depth maps often readily available in practical settings. We propose an improved sparse view NeRF model, ATGANNeRF, which integrates an enhanced U-Net generator with a dual-discriminator framework, CBAM, and Multi-Head Self-Attention mechanisms. The symmetric design enhances the model’s ability to capture and preserve spatial relationships, ensuring a more consistent generation of novel views. Additionally, local depth ranking is employed to ensure depth consistency with coarse maps, and spatial continuity constraints are introduced to synthesize novel views from sparse samples. SSIM loss is also added to preserve local structural details like edges and textures. Evaluation on LLFF, DTU, and our own datasets shows that ATGANNeRF significantly outperforms state-of-the-art methods, both quantitatively and qualitatively.

Infrared Image Enhancement Based on Guided Filtering and Adaptive Algorithm and Its FPGA Implementation

ABSTRACTDetail enhancement and noise suppression are critical to the performance of infrared imaging systems, and a method for enhancing infrared images based on guided filters is introduced in this paper. Firstly, the guidance filter and Gaussian filter are used to decompose the original infrared image into the base layer and the detail layer, and then the adaptive histogram equalization method is used to enhance the overall brightness and contrast of the base layer. For the detail layer, we will guide the filtering to obtain the gain factor through the “” rule processing, on this basis, the detail gain function is constructed to apply to the processing of the detail layer, and finally the basic layer and the detail layer are fused to obtain the final result image. The proposed method is compared with six other infrared image enhancement algorithms under different data sets and analyzed using subjective and objective evaluation indicators. The experimental data show that the proposed method can effectively improve the overall contrast and local details of the image and has good scene adaptability. Finally, we transplanted the algorithm to the FPGA bottom layer for implementation, and for 640 512 resolution images, the final implementation effect is almost no different from that on the PC side.

Multi-view brain functional connectivity and hierarchical fusion for EEG-based emotion recognition

Emotional states evolve gradually from their onset to full manifestation, reflected in changes in brain functional connectivity. Existing research often focuses only on the final emotional state, missing the rich, dynamic information of emotional evolution. This paper proposes a novel network framework leveraging global and local brain functional connectivity through a multi-view approach. We introduce a global and local connectivity modeling method using electroencephalogram signals to simulate brain connectivity changes across time and specific moments. A residual-based local detail feature extractor combined with a coordinate attention module captures long-range dependencies and maintains positional accuracy. A global feature extractor is also designed for the effective extraction of emotional information. Finally, a hierarchical multi-scale cross-attention feature fusion module integrates features across different levels to enhance emotion recognition performance and robustness. Extensive experiments on three publicly available datasets demonstrate the framework’s superiority and generalization capability.

Vibrating screen fault detection based on video frame prediction

ABSTRACT As a key technical equipment for improving coal quality, efficiency and promoting clean and efficient utilization, it is important for vibrating screens to detect their faults quickly and accurately. Traditional vibrating screen fault detection methods usually rely on the vibration signals collected by sensors, and identify the faults through spectral analysis, time domain feature extraction, etc. However, these methods face the problems of inaccurate detection, insufficient real-time performance and poor adaptability to environmental changes under complex working conditions. To address this limitation, this paper proposes a video frame prediction model, DST-UNet, to detect the operation of vibrating screens through video. The model combines convolutional units with different kernel sizes to simultaneously extract local detail information and global structural information in the operation of vibrating screens; uses ECA as a temporal attention mechanism to improve the model’s understanding and sensitivity to temporal sequences and employs convolutional units to generate spatio-temporal weights to dynamically fuse spatial and temporal features. In addition, an evaluation metric MTR is proposed to assess the model performance. The experimental results show that, compared with models such as SimVP, 3D convolution and PredRNN, the method improves the sensitivity, accuracy and processing speed of fault detection while reducing the model complexity and can realize efficient detection with limited data sets, which helps to identify equipment faults quickly and reduce the risk of damage.

Rain removal method for single image of dual-branch joint network based on sparse transformer

In response to image degradation caused by rain during image acquisition, this paper proposes a rain removal method for single image of dual-branch joint network based on a sparse Transformer (DBSTNet). The developed model comprises a rain removal subnet and a background recovery subnet. The former extracts rain trace information utilizing a rain removal strategy, while the latter employs this information to restore background details. Furthermore, a U-shaped encoder-decoder branch (UEDB) focuses on local features to mitigate the impact of rainwater on background detail textures. UEDB incorporates a feature refinement unit to maximize the contribution of the channel attention mechanism in recovering local detail features. Additionally, since tokens with low relevance in the Transformer may influence image recovery, this study introduces a residual sparse Transformer branch (RSTB) to overcome the limitations of the Convolutional Neural Network’s (CNN’s) receptive field. Indeed, RSTB preserves the most valuable self-attention values for the aggregation of features, facilitating high-quality image reconstruction from a global perspective. Finally, the parallel dual-branch joint module, composed of RSTB and UEDB branches, effectively captures the local context and global structure, culminating in a clear background image. Experimental validation on synthetic and real datasets demonstrates that rain removal images exhibit richer detail information, significantly improving the overall visual effect.

LKAFFNet: A Novel Large-Kernel Attention Feature Fusion Network for Land Cover Segmentation

The accurate segmentation of land cover in high-resolution remote sensing imagery is crucial for applications such as urban planning, environmental monitoring, and disaster management. However, traditional convolutional neural networks (CNNs) struggle to balance fine-grained local detail with large-scale contextual information. To tackle these challenges, we combine large-kernel convolutions, attention mechanisms, and multi-scale feature fusion to form a novel LKAFFNet framework that introduces the following three key modules: LkResNet, which enhances feature extraction through parameterizable large-kernel convolutions; Large-Kernel Attention Aggregation (LKAA), integrating spatial and channel attention; and Channel Difference Features Shift Fusion (CDFSF), which enables efficient multi-scale feature fusion. Experimental comparisons demonstrate that LKAFFNet outperforms previous models on both the LandCover dataset and WHU Building dataset, particularly in cases with diverse scales. Specifically, it achieved a mIoU of 0.8155 on the LandCover dataset and 0.9326 on the WHU Building dataset. These findings suggest that LKAFFNet significantly improves land cover segmentation performance, offering a more effective tool for remote sensing applications.

Development of Women’s Shirt Style Recognition Model (WSSRM) based on a ResNet and Transformer integrated method

The rise of e-commerce has brought women’s shirt images to the forefront, showcasing complex styles and diverse features. This presents a challenge for consumers in finding their preferred styles. Deep learning, known for its speed and superior retrieval performance, offers a solution by learning image features through multi-layer neural networks and extracting high-level semantic information, making it highly effective for recognizing and classifying clothing styles. The ResNet model excels in extracting detailed pixel information, but has limitations: (1) Jump connections in ResNet, which add input directly to the output in the residual block, can cause feature distortion, especially when input and output sizes do not match; (2) Despite many network layers, the effective receptive field of ResNet is smaller than theoretically expected. To address these limitations, this paper introduces the Transformer model into ResNet, proposing a new method for recognizing and classifying women’s shirt styles based on local detail feature extraction. The Transformer’s attention mechanism enhances the model’s ability to focus on important features and suppress less relevant ones, improving the accuracy of local detail feature extraction. This study examines six typical women’s shirt style features, applying the improved ResNet to their recognition and classification, resulting in a highly accurate and reliable model. This theoretical and practical advancement enhances the recognition of detailed features in women’s shirts, significantly contributing to the development of intelligent clothing design.

MS-CLSTM: Myoelectric Manipulator Gesture Recognition Based on Multi-Scale Feature Fusion CNN-LSTM Network.

Surface electromyography (sEMG) signals reflect the local electrical activity of muscle fibers and the synergistic action of the overall muscle group, making them useful for gesture control of myoelectric manipulators. In recent years, deep learning methods have increasingly been applied to sEMG gesture recognition due to their powerful automatic feature extraction capabilities. sEMG signals contain rich local details and global patterns, but single-scale convolutional networks are limited in their ability to capture both comprehensively, which restricts model performance. This paper proposes a deep learning model based on multi-scale feature fusion-MS-CLSTM (MS Block-ResCBAM-Bi-LSTM). The MS Block extracts local details, global patterns, and inter-channel correlations in sEMG signals using convolutional kernels of different scales. The ResCBAM, which integrates CBAM and Simple-ResNet, enhances attention to key gesture information while alleviating overfitting issues common in small-sample datasets. Experimental results demonstrate that the MS-CLSTM model achieves recognition accuracies of 86.66% and 83.27% on the Ninapro DB2 and DB4 datasets, respectively, and the accuracy can reach 89% in real-time myoelectric manipulator gesture prediction experiments. The proposed model exhibits superior performance in sEMG gesture recognition tasks, offering an effective solution for applications in prosthetic hand control, robotic control, and other human-computer interaction fields.

YOLOv8-STE: Enhancing Object Detection Performance Under Adverse Weather Conditions with Deep Learning

Object detection powered by deep learning is extensively utilized across diverse sectors, yielding substantial outcomes. However, adverse weather conditions such as rain, snow, and haze interfere with images, leading to a decline in quality and making it extremely challenging for existing methods to detect images captured in such environments. In response to the problem, our research put forth a detection approach grounded in the YOLOv8 model, which we named YOLOv8-STE. Specifically, we introduced a new detection module, ST, on the basis of YOLOv8, which integrates global information step-by-step through window movement while capturing local details. This is particularly important in adverse weather conditions and effectively enhances detection accuracy. Additionally, an EMA mechanism was incorporated into the neck network, which reduced computational burdens through streamlined operations and enriched the original features, making them more hierarchical, thus improving detection stability and generalization. Finally, soft-NMS was used to replace the traditional non-maximum suppression method. Experimental results indicate that our proposed YOLOv8-STE demonstrates excellent performance under adverse weather conditions. Compared to the baseline model YOLOv8, it exhibits superior results on the RTTS dataset, providing a more efficient method for object detection in adverse weather.

A Weakly Supervised Crowd Counting Method via Combining CNN and Transformer

During the past five years, there has been an increasing trend of weakly supervised crowd counting methods being developed since such methods just rely on count-level annotations and avoid a laborious labeling process. But, the existing weakly supervised methods usually fail to achieve comparable counting performance to the fully supervised methods. To improve the accuracy of crowd counting tasks, we propose to combine the convolutional neural network (CNN) and Transformer frameworks. Since CNN focuses on capturing local detail information and Transformer can effectively extract global context information, we believe that the combination of CNN and Transformer could learn more efficient feature representations for crowd images. Our proposed framework is named CrowdCCT (Crowd Counting via CNN and Transformer), and it is composed of a CNN feature extraction part, a Transformer feature extraction part, and a counting regression part. In the CNN part, we utilize DenseNet121 to learn rich semantic features with its inherent dense connection structure. In the Transformer part, we introduce two attention modules, Multi-Scale Dilated Attention (MSDA) and Location-Enhanced Attention (LEA), working together to extract more expressive features. The output features are then fed into the regression part to generate the predicted counting results. Experiments on four crowd counting benchmark datasets demonstrate that our proposed CrowdCCT can achieve superior performance. Also, the experimental results validate the feasibility and effectiveness of combining CNN and Transformer for weakly supervised counting tasks. Our work could be expected to promote further combination research on CNN and Transformer.

Offset attention with seed generation for point cloud completion

Abstract Point cloud data acquired through 3D scanning is frequently subject to fragmentation due to the constraints of the scanner’s field of view and occlusions within the scanned object. The ensuing incompleteness in the data can significantly degrade the accuracy of subsequent computational tasks. Traditional methods for predicting complete point clouds from these fragments often fail to capture the fine-grained local details, leading to inaccurate reconstructions. In this work, we introduce a novel neural network architecture designed for point cloud completion that addresses these limitations.Our network accepts an incomplete point cloud and employs a multi-scale feature extraction module, which integrates an offset attention mechanism alongside a feature aggregation module operating across various scales. This dual approach significantly bolsters the network’s capacity to discern both local and global features inherent in the point cloud data. Furthermore, we incorporate a seed generation module within our missing point cloud generator, harnessing a hierarchical feature pyramid network to forecast the entirety of the point cloud. This innovative strategy allows our network to accurately predict the structure of missing regions.Empirical evaluations conducted on the Shapenet-Part and ModelNet10 datasets substantiate the efficacy of our proposed methodology. Our approach outperforms the state-of-the-art PF-Net algorithm, achieving a remarkable reduction in chamfer distance by 16.15$\%$ and 41.87$\%$ on the respective datasets. Visual inspection of the results underscores the robust generalization capabilities of our algorithm, which is particularly evident in scenarios with limited dataset sizes. It adeptly predicts the contours of the missing regions and synthesizes a more comprehensive point cloud shape.

GLMAFuse: A Dual-Stream Infrared and Visible Image Fusion Framework Integrating Local and Global Features with Multi-Scale Attention

Integrating infrared and visible-light images facilitates a more comprehensive understanding of scenes by amalgamating dual-sensor data derived from identical environments. Traditional CNN-based fusion techniques are predominantly confined to local feature emphasis due to their inherently limited receptive fields. Conversely, Transformer-based models tend to prioritize global information, which can lead to a deficiency in feature diversity and detail retention. Furthermore, methods reliant on single-scale feature extraction are inadequate for capturing extensive scene information. To address these limitations, this study presents GLMAFuse, an innovative dual-stream encoder–decoder network, which utilizes a multi-scale attention mechanism to harmoniously integrate global and local features. This framework is designed to maximize the extraction of multi-scale features from source images while effectively synthesizing local and global information across all layers. We introduce the global-aware and local embedding (GALE) module to adeptly capture and merge global structural attributes and localized details from infrared and visible imagery via a parallel dual-branch architecture. Additionally, the multi-scale attention fusion (MSAF) module is engineered to optimize attention weights at the channel level, facilitating an enhanced synergy between high-frequency edge details and global backgrounds. This promotes effective interaction and fusion of dual-modal features. Extensive evaluations using standard datasets demonstrate that GLMAFuse surpasses the existing leading methods in both qualitative and quantitative assessments, highlighting its superior capability in infrared and visible image fusion. On the TNO and MSRS datasets, our method achieves outstanding performance across multiple metrics, including EN (7.15, 6.75), SD (46.72, 47.55), SF (12.79, 12.56), MI (2.21, 3.22), SCD (1.75, 1.80), VIF (0.79, 1.08), Qbaf (0.58, 0.71), and SSIM (0.99, 1.00). These results underscore its exceptional proficiency in infrared and visible image fusion.

Dual-Path Beat Tracking: Combining Temporal Convolutional Networks and Transformers in Parallel

The Transformer, a deep learning architecture, has shown exceptional adaptability across fields, including music information retrieval (MIR). Transformers excel at capturing global, long-range dependencies in sequences, which is valuable for tracking rhythmic patterns over time. Temporal Convolutional Networks (TCNs), with their dilated convolutions, are effective at processing local, temporal patterns with reduced complexity. Combining these complementary characteristics, global sequence modeling from Transformers and local temporal detail from TCNs enhances beat tracking while reducing the model’s overall complexity. To capture beat intervals of varying lengths and ensure optimal alignment of beat predictions, the model employs a Dynamic Bayesian Network (DBN), followed by Viterbi decoding for effective post-processing. This system is evaluated across diverse public datasets spanning various music genres and styles, achieving performance on par with current state-of-the-art methods yet with fewer trainable parameters. Additionally, we also explore the interpretability of the model using Grad-CAM to visualize the model’s learned features, offering insights into how the TCN-Transformer hybrid captures rhythmic patterns in the data.

AeroNet: Efficient YOLOv7 for Tiny-Object Detection in UAV Imagery

The detection of multiple tiny objects from diverse perspectives using unmanned aerial vehicles (UAVs) and onboard edge devices presents a significant challenge in computer vision. To address that, this study proposes AeroNet, a lightweight and efficient detection algorithm based on YOLOv7 (You Only Look Once version7).This algorithm features the LHGNet (Lightweight High-Performance GhostNet) backbone, an advanced feature extraction network that integrates depth-wise separable convolution and channel shuffle modules.These modules enable deeper exploration of network features, promoting the fusion of local detail information and channel characteristics. Additionally, this research introduces the LGS(Lightweight Gradient-Sensitive) bottleneck and LGSCSP(Lightweight Gradient-Sensitive Cross Stage Partial Network) fusion module in the neck to reduce computational complexity while maintaining accuracy. Structural modifications and adjusted feature map sizes further enhance detection accuracy. Evaluated on the SkyFusion dataset,this method demonstrated a 25.0% reduction in parameter count and a 12.8% increase in mAP (0.5) compared to YOLOv7. These results underscore the effectiveness of this proposed approach in improving detection accuracy and model efficiency through the proposed enhancements.

Cross-attention swin-transformer for detailed segmentation of ancient architectural color patterns.

Segmentation tasks in computer vision play a crucial role in various applications, ranging from object detection to medical imaging and cultural heritage preservation. Traditional approaches, including convolutional neural networks (CNNs) and standard transformer-based models, have achieved significant success; however, they often face challenges in capturing fine-grained details and maintaining efficiency across diverse datasets. These methods struggle with balancing precision and computational efficiency, especially when dealing with complex patterns and high-resolution images. To address these limitations, we propose a novel segmentation model that integrates a hierarchical vision transformer backbone with multi-scale self-attention, cascaded attention decoding, and diffusion-based robustness enhancement. Our approach aims to capture both local details and global contexts effectively while maintaining lower computational overhead. Experiments conducted on four diverse datasets, including Ancient Architecture, MS COCO, Cityscapes, and ScanNet, demonstrate that our model outperforms state-of-the-art methods in accuracy, recall, and computational efficiency. The results highlight the model's ability to generalize well across different tasks and provide robust segmentation, even in challenging scenarios. Our work paves the way for more efficient and precise segmentation techniques, making it valuable for applications where both detail and speed are critical.

Magnetization reversal of perpendicular magnetic anisotropy multilayers on polymeric substrates for flexible spintronics applications

Earlier demonstrations of spintronic functionality on flexible substrates have highlighted the potential for spintronics in flexible electronics applications. However, for device applications, the relationship between global magnetization reversal, as measured by hysteresis, and the local reversal processes of nucleation and growth of magnetic domains need to be understood for magnetic systems on flexible substrates. This study compares the local magnetization reversal behavior of perpendicularly magnetized Pt/CoFeB/Pt and Pt/Co/Pt on rigid and flexible polymeric substrates using magneto-optical Kerr effect magnetometry and microscopy. It is shown that while the magnetic hysteresis is comparable, the local details of the nucleation and field driven reversal are quite different and are attributed to the greater variability of the surface structures of the polymeric substrates, which has implications for consistency in device performance.