Multi-scale Features Research Articles

ShiftTransUNet: An Efficient Deep Learning Model for Medical Image Segmentation Using ShiftViT Framework

Deep learning has significantly advanced the field of medical image segmentation. However, the complexity of network structures often leads to high computational demands, limiting their practical efficiency. To enhance the efficiency of image segmentation, this paper introduces an innovative, concise, and lightweight deep learning network. First, to reduce model complexity, we replaced the attention mechanism in the traditional vision transformer (ViT) structure with a shift operation, creating the ShiftViT architecture. This substitution significantly decreased computation and the number of parameters while preserving model performance. Second, to retain and enhance fine-grained features and facilitate more precise information transfer across different layers, we employed a full-scale progressive skip connection strategy. This approach effectively integrates multi-scale feature information, further enhancing model performance. Additionally, to further reduce network complexity, inspired by the independence of probabilities, we opted for depth-wise separable convolution over traditional convolution. This enhances the relative independence between layers. Together, these modifications achieved superior segmentation results on both the Synapse and Automated Cardiac Diagnostic Challenge (ACDC) datasets compared to mainstream models, with substantial advantages in terms of computational efficiency and parameter count. The proposed approach represents an effective solution for medical image applications with limited computational resources and holds great promise for clinical practice.

MSMP-Net: A Multi-Scale Neural Network for End-to-End Monkeypox Virus Skin Lesion Classification

Monkeypox is a zoonotic disease caused by monkeypox virus infection. It is easily transmitted among people and poses a major threat to human health, making it of great significance in public health. Therefore, this paper proposes MSMP-Net, a multi-scale neural network for end-to-end monkeypox virus skin lesion classification ConvNeXt is used as the backbone network, and designs such as inverse bottleneck layers and large convolution kernels are used to enhance the network’s feature extraction capabilities. In order to effectively utilize the multi-level feature maps generated by the backbone network, a multi-scale feature fusion structure was designed. By fusing the deepest feature maps of multi-scale features, the model’s ability to represent monkeypox image features is enhanced. Experimental results show that the accuracy, precision, recall, and F1-score of this method on the MSLD v2.0 dataset are 87.03 ± 3.43%, 87.59 ± 3.37%, 87.03 ± 3.43%, and 86.58 ± 3.66%, respectively.

TSAE-UNet: A Novel Network for Multi-Scene and Multi-Temporal Water Body Detection Based on Spatiotemporal Feature Extraction

The application of remote sensing technology in water body detection has become increasingly widespread, offering significant value for environmental monitoring, hydrological research, and disaster early warning. However, the existing methods face challenges in multi-scene and multi-temporal water body detection, including the diverse variations in water body shapes and sizes that complicate detection; the complexity of land cover types, which easily leads to false positives and missed detections; the high cost of acquiring high-resolution images, limiting long-term applications; and the lack of effective handling of multi-temporal data, making it difficult to capture the dynamic changes in water bodies. To address these challenges, this study proposes a novel network for multi-scene and multi-temporal water body detection based on spatiotemporal feature extraction, named TSAE-UNet. TSAE-UNet integrates convolutional neural networks (CNN), depthwise separable convolutions, ConvLSTM, and attention mechanisms, significantly improving the accuracy and robustness of water body detection by capturing multi-scale features and establishing long-term dependencies. The Otsu method was employed to quickly process Sentinel-1A and Sentinel-2 images, generating a high-quality training dataset. In the first experiment, five rectangular areas of approximately 37.5 km2 each were selected to validate the water body detection performance of the TSAE-UNet model across different scenes. The second experiment focused on Jining City, Shandong Province, China, analyzing the monthly water body changes from 2020 to 2022 and the quarterly changes in 2022. The experimental results demonstrate that TSAE-UNet excels in multi-scene and long-term water body detection, achieving a precision of 0.989, a recall of 0.983, an F1 score of 0.986, and an IoU of 0.974, significantly outperforming FCN, PSPNet, DeepLabV3+, ADCNN, and MECNet.

A multi-source domain feature-decision dual fusion adversarial transfer network for cross-domain anti-noise mechanical fault diagnosis in sustainable city

Rotating machinery forms the critical backbone of infrastructure in a sustainable city, with bearings playing a pivotal role as key mechanical transmission components. Therefore, the health status of these bearings directly influences the safe operation of the infrastructure. Accurate and reliable diagnosis of defects in these components minimizes downtime, reduces maintenance costs, and prevents major accidents, ultimately providing insights in the construction and management of a sustainable city. Typically, in actual industrial scenarios, varying working conditions and various types of machines can result in significant discrepancies in the distribution of sample data. Moreover, the non-negligible noise may degrade the diagnostic performance. Therefore, realizing an accurate and reliable bearing diagnosis considering the cross-domain and noise environment remains a challenge. Leveraging the merits of information fusion and multi-source domain transfer learning, this article proposes a multi-source domain feature-decision dual fusion adversarial transfer network (DFATN) to break through the aforesaid limitations. Initially, an adversarial transfer framework is developed, incorporating novel feature matching evaluation and joint distribution difference losses. This framework is designed to facilitate the learning of feature invariants across domains and to enhance the sharing of domain-specific knowledge, even in noise. Relying on channel-spatial interactive feature fusion, a multi-scale feature extractor (MFE) is constructed to share the interaction and enhance the modeling of complex features in multiple dimensions. Additionally, a fault state-related decision fusion mechanism (SDF) is also implemented to integrate diagnostic information, significantly enhancing the generalization performance and robustness of the proposed network. By employing both public Paderborn University (PU) and laboratory-collected (Lab) datasets, the effectiveness and superiority of the proposed DFATN on bearing fault diagnosis are validated. For cross-working condition tasks, the proposed method realizes impressive performance, with average accuracies of 96.52% and 98.76% for Paderborn University (PU) and laboratory-collected (Lab) datasets, respectively. For cross-machine tasks, the average accuracy is 83.36%, outperforming other latest cross-domain fault diagnosis techniques.

A Multi-Scale Feature Extraction Method Based on Improved Transformer for Intrusion Detection

Network traffic is a crucial indicator of network performance and network intrusions typically result in traffic anomalies. Capturing the differences and commonalities between different input features is challenging due to high-dimensional traffic data. To address this, we propose a multi-scale feature extraction method based on global additive attention (MSFE-GAA), which integrates time position information encoded by trigonometric functions to capture multi-scale temporal features. An improved Transformer with a similarity matrix captures the commonalities and differences, enhanced by global additive attention for long-term dependencies. Experiments on two public datasets show that the MSFE-GAA model outperforms other baseline models.

DBAII-Net with multiscale feature aggregation and cross-modal attention for enhancing infant brain injury classification in MRI.

MRI is pivotal in diagnosing brain injuries in infants. However, the dynamic development of the brain introduces variability in infant MRI characteristics, posing challenges for MRI-based classification in this population. Furthermore, manual data selection in large-scale studies is labor-intensive, and existing algorithms often underperform with thick-slice MRI data. To enhance research efficiency and classification accuracy in large datasets, we propose an advanced classification model. We introduce the Dual-Branch Attention Information Interactive Neural Network (DBAII-Net), a cutting-edge model inspired by radiologists' use of multiple MRI sequences. DBAII-Net features two innovative modules: (1) the Convolutional Enhancement Module (CEM), which leverages advanced convolutional techniques to aggregate multi-scale features, significantly enhancing information representation; and (2) the Cross-Modal Attention Module (CMAM), which employs state-of-the-art attention mechanisms to fuse data across branches, dramatically improving positional and channel feature extraction. Performances (accuracy, sensitivity, specificity, area under the curve, etc.) of DBAII-Net were compared with eight benchmark models for brain MRI classification in infants aged 6 months to 2 years. Utilizing a self-constructed dataset of 240 thick-slice brain MRI scans (122 with brain injuries, 118 without), DBAII-Net demonstrated superior performance. On a test set of approximately 50 cases, DBAII-Net achieved average performance metrics of 92.53% accuracy, 90.20% sensitivity, 94.93% specificity, and an area under the curve (AUC) of 0.9603. Ablation studies confirmed the effectiveness of CEM and CMAM, with CMAM significantly boosting classification metrics. DBAII-Net with CEM and CMAM outperforms existing benchmarks in enhancing the precision of brain MRI classification in infants, significantly reducing manual effort in infant brain research. Our code is available at https://github.com/jiazhen4585/DBAII-Net.

Multi-feature fusion and multi-attention deep network for enhancing road extraction in remote sensing images

ABSTRACT Road detection in remote sensing (RS) images plays a critical role in applications ranging from urban planning to autonomous navigation systems. However, accurate road extraction remains a challenging task due to the presence of textual-similar objects that can be visually confused with roads, and shadows that can obscure road features. For this regard, we present a novel end-to-end network that leverages multi-feature fusion and multi-attention. Multi-level dual residual blocks are introduced to capture multi-scale and multi-level road features. A channel attention feature fusion module fuses road features from the decoder while suppressing coarse-grain noises. Moreover, a difference channel feature fusion module filters out the interfering texture-similar features and irrelevant background features. Subsequently, a spatial channel feature enhancement module is designed to identify the mis-segmented regions and reclassify these pixels. We conducted extensive experiments on diverse datasets, demonstrating the effectiveness of our approach in improving road detection accuracy, especially in areas with shadows and textual-similar objects. The results indicate that our approach outperforms state-of-the-art methods in handling textual-similar objects and shadow-occluded roads, making it a valuable contribution to the field of road extraction in remote sensing images.

Multi-scale feature correspondence and restriction mechanism for visible X-ray baggage re-Identification

Multi-scale feature correspondence and restriction mechanism for visible X-ray baggage re-Identification

Infrared Image Generation Based on Visual State Space and Contrastive Learning

The preparation of infrared reference images is of great significance for improving the accuracy and precision of infrared imaging guidance. However, collecting infrared data on-site is difficult and time-consuming. Fortunately, the infrared images can be obtained from the corresponding visible-light images to enrich the infrared data. To this end, this present work proposes an image translation algorithm that converts visible-light images to infrared images. This algorithm, named V2IGAN, is founded on the visual state space attention module and multi-scale feature contrastive learning loss. Firstly, we introduce a visual state space attention module designed to sharpen the generative network’s focus on critical regions within visible-light images. This enhancement not only improves feature extraction but also bolsters the generator’s capacity to accurately model features, ultimately enhancing the quality of generated images. Furthermore, the method incorporates a multi-scale feature contrastive learning loss function, which serves to bolster the robustness of the model and refine the detail of the generated images. Experimental results show that the V2IGAN method outperforms existing typical infrared image generation techniques in both subjective visual assessments and objective metric evaluations. This suggests that the V2IGAN method is adept at enhancing the feature representation in images, refining the details of the generated infrared images, and yielding reliable, high-quality results.

Enhanced Vehicle Logo Detection Method Based on Self-Attention Mechanism for Electric Vehicle Application

Vehicle logo detection plays a crucial role in various computer vision applications, such as vehicle classification and detection. In this research, we propose an improved vehicle logo detection method leveraging the self-attention mechanism. Our feature-sampling structure integrates multiple attention mechanisms and bidirectional feature aggregation to enhance the discriminative power of the detection model. Specifically, we introduce the multi-head attention for multi-scale feature fusion module to capture multi-scale contextual information effectively. Moreover, we incorporate the bidirectional aggregation mechanism to facilitate information exchange between different layers of the detection network. Experimental results on a benchmark dataset (VLD-45 dataset) demonstrate that our proposed method outperforms baseline models in terms of both detection accuracy and efficiency. Our experimental evaluation using the VLD-45 dataset achieves a state-of-the-art result of 90.3% mAP. Our method has also improved AP by 10% for difficult samples, such as HAVAL and LAND ROVER. Our method provides a new detection framework for small-size objects, with potential applications in various fields.

Contraband classification method for X-ray security images considering sample imbalance

X-ray security image contraband classification is widely used to assist in maintaining aviation and transportation security. This paper suggests an end-to-end X-ray security inspection image classification method that takes sample imbalance into account in order to address the issues of different scales of contraband in X-ray images, challenging samples, and unbalanced positive and negative samples inherent in passenger baggage security inspection. The feature fusion module is used to enhance the model's ability to express picture edge and texture features while the multi-scale feature extraction network is used to capture the features of numerous sorts of illegal goods with various scales. Based on the cost-sensitive idea, the loss function is designed to solve the problem of dataset imbalance, and improve the classification accuracy of difficult samples. experimental results of the subset constructed on the public dataset SIXray show that the proposed method improves the mean AP index by compared with the current optimal end-to-end classification model, especially for hard-to-classify samples such as scissors, the AP index has a significant improvement effect.

Hair-YOLO: a hair follicle detection model based on YOLOv8

Abstract Hair follicle detection technology has developed rapidly in recent years. Traditional manual detection methods are labor-intensive and inefficient. To address this problem, we propose a real-time hair follicle detection model called Hair-YOLO based on YOLOv8. This model focuses on accurately identifying the number of hairs within each follicle, providing precise data that help doctors assess hair density and follicle health of patients. First of all, we incorporate the re-parameterization Ghost module into the backbone, reducing the parameters and computational load. Then, the deformable convolution v3 operator is integrated into the neck network, enhancing adaptation to follicle shapes. Next, we propose a novel multi-scale feature perception separated and enhancement attention (Multi-SEAM) module, building upon the SEAM module, to address complex scalp scenarios. Furthermore, we enhance bounding box regression by replacing the standard complete intersection over union loss, with a modified point distance intersection over union loss. Finally, we construct a new hair follicle dataset and use it for a comparative analysis of Hair-YOLO and established models. Our model shows excellent performance, with a 14.26% increase in mAP@0.5:0.95, a 2.98% increase in Recall, and a 4.31% increase in Precision compared to the baseline.

Interactive Segmentation for Medical Images Using Spatial Modeling Mamba

Interactive segmentation methods utilize user-provided positive and negative clicks to guide the model in accurately segmenting target objects. Compared to fully automatic medical image segmentation, these methods can achieve higher segmentation accuracy with limited image data, demonstrating significant potential in clinical applications. Typically, for each new click provided by the user, conventional interactive segmentation methods reprocess the entire network by re-inputting the click into the segmentation model, which greatly increases the user’s interaction burden and deviates from the intended goal of interactive segmentation tasks. To address this issue, we propose an efficient segmentation network, ESM-Net, for interactive medical image segmentation. It obtains high-quality segmentation masks based on the user’s initial clicks, reducing the complexity of subsequent refinement steps. Recent studies have demonstrated the strong performance of the Mamba model in various vision tasks; however, its application in interactive segmentation remains unexplored. In our study, we incorporate the Mamba module into our framework for the first time and enhance its spatial representation capabilities by developing a Spatial Augmented Convolution (SAC) module. These components are combined as the fundamental building blocks of our network. Furthermore, we designed a novel and efficient segmentation head to fuse multi-scale features extracted from the encoder, optimizing the generation of the predicted segmentation masks. Through comprehensive experiments, our method achieved state-of-the-art performance on three medical image datasets. Specifically, we achieved 1.43 NoC@90 on the Kvasir-SEG dataset, 1.57 NoC@90 on the CVC-ClinicDB polyp segmentation dataset, and 1.03 NoC@90 on the ADAM retinal disk segmentation dataset. The assessments on these three medical image datasets highlight the effectiveness of our approach in interactive medical image segmentation.

Nutritional composition analysis in food images: an innovative Swin Transformer approach.

Accurate recognition of nutritional components in food is crucial for dietary management and health monitoring. Current methods often rely on traditional chemical analysis techniques, which are time-consuming, require destructive sampling, and are not suitable for large-scale or real-time applications. Therefore, there is a pressing need for efficient, non-destructive, and accurate methods to identify and quantify nutrients in food. In this study, we propose a novel deep learning model that integrates EfficientNet, Swin Transformer, and Feature Pyramid Network (FPN) to enhance the accuracy and efficiency of food nutrient recognition. Our model combines the strengths of EfficientNet for feature extraction, Swin Transformer for capturing long-range dependencies, and FPN for multi-scale feature fusion. Experimental results demonstrate that our model significantly outperforms existing methods. On the Nutrition5k dataset, it achieves a Top-1 accuracy of 79.50% and a Mean Absolute Percentage Error (MAPE) for calorie prediction of 14.72%. On the ChinaMartFood109 dataset, the model achieves a Top-1 accuracy of 80.25% and a calorie MAPE of 15.21%. These results highlight the model's robustness and adaptability across diverse food images, providing a reliable and efficient tool for rapid, non-destructive nutrient detection. This advancement supports better dietary management and enhances the understanding of food nutrition, potentially leading to more effective health monitoring applications.

Lightweight Detection of Broccoli Heads in Complex Field Environments Based on LBDC-YOLO

Robotically selective broccoli harvesting requires precise lightweight detection models to efficiently detect broccoli heads. Therefore, this study introduces a lightweight and high-precision detection model named LBDC-YOLO (Lightweight Broccoli Detection in Complex Environment—You Look Only Once), based on the improved YOLOv8 (You Look Only Once, Version 8). The model incorporates the Slim-neck design paradigm based on GSConv to reduce computational complexity. Furthermore, Triplet Attention is integrated into the backbone network to capture cross-dimensional interactions between spatial and channel dimensions, enhancing the model’s feature extraction capability under multiple interfering factors. The original neck network structure is replaced with a BiFPN (Bidirectional Feature Pyramid Network), optimizing the cross-layer connection structure, and employing weighted fusion methods for better integration of multi-scale features. The model undergoes training and testing on a dataset constructed in real field conditions, featuring broccoli images under various influencing factors. Experimental results demonstrate that LBDC-YOLO achieves an average detection accuracy of 94.44% for broccoli. Compared to the original YOLOv8n, LBDC-YOLO achieves a 32.1% reduction in computational complexity, a 47.8% decrease in parameters, a 44.4% reduction in model size, and a 0.47 percentage point accuracy improvement. When compared to models such as YOLOv5n, YOLOv5s, and YOLOv7-tiny, LBDC-YOLO exhibits higher detection accuracy and lower computational complexity, presenting clear advantages for broccoli detection tasks in complex field environments. The results of this study provide an accurate and lightweight method for the detection of broccoli heads in complex field environments. This work aims to inspire further research in precision agriculture and to advance knowledge in model-assisted agricultural practices.

Remaining Useful Life Prediction Method Based on Dual-Path Interaction Network with Multiscale Feature Fusion and Dynamic Weight Adaptation

In fields such as manufacturing and aerospace, remaining useful life (RUL) prediction estimates the failure time of high-value assets like industrial equipment and aircraft engines by analyzing time series data collected from various sensors, enabling more effective predictive maintenance. However, significant temporal diversity and operational complexity during equipment operation make it difficult for traditional single-scale, single-dimensional feature extraction methods to effectively capture complex temporal dependencies and multi-dimensional feature interactions. To address this issue, we propose a Dual-Path Interaction Network, integrating the Multiscale Temporal-Feature Convolution Fusion Module (MTF-CFM) and the Dynamic Weight Adaptation Module (DWAM). This approach adaptively extracts information across different temporal and feature scales, enabling effective interaction of multi-dimensional information. Using the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset for comprehensive performance evaluation, our method achieved RMSE values of 0.0969, 0.1316, 0.086, and 0.1148; MAPE values of 9.72%, 14.51%, 8.04%, and 11.27%; and Score results of 59.93, 209.39, 67.56, and 215.35 across four different data categories. Furthermore, the MTF-CFM module demonstrated an average improvement of 7.12%, 10.62%, and 7.21% in RMSE, MAPE, and Score across multiple baseline models. These results validate the effectiveness and potential of the proposed model in improving the accuracy and robustness of RUL prediction.

Transformer assisted by DSC and BiLSTM for bearing fault pattern recognition under strong noise interference

ABSTRACT Deep learning has greatly promoted intelligent fault diagnosis for bearings and has led to the development of many intelligent fault recognition methods based on deep neural networks. However, bearing fault signals in industrial applications are susceptible to noise interference, and intelligent recognition methods for bearing faults under strong noise interference are lacking. An intelligent fault pattern recognition model termed IFPR-MDL, which is based on a Transformer assisted by depthwise separable convolutional (DSC) and bidirectional long short-term memory (BiLSTM) networks, is designed to identify bearing faults under strong noise conditions. In this model, multiscale features are first extracted by a multiscale feature extraction module designed on the basis of DSC; then, contextual information is further extracted by a BiLSTM, and information from different scales is integrated by Transformer encoders. The DSC network facilitates model parameter reduction, the BiLSTM network has expertise in mining temporal features, and the Transformer network excels at capturing long-range feature correlations. This model fully utilises the advantages of these models and complements them to improve the anti-interference ability of bearing fault pattern recognition. Tests on several public datasets show that the proposed model has higher recognition accuracy, stronger generalisation ability, and noise resistance in strong noise situations.

GroupFormer for hyperspectral image classification through group attention

Hyperspectral image (HSI) data has a wide range of valuable spectral information for numerous tasks. HSI data encounters challenges such as small training samples, scarcity, and redundant information. Researchers have introduced various research works to address these challenges. Convolution Neural Network (CNN) has gained significant success in the field of HSI classification. CNN’s primary focus is to extract low-level features from HSI data, and it has a limited ability to detect long-range dependencies due to the confined filter size. In contrast, vision transformers exhibit great success in the HSI classification field due to the use of attention mechanisms to learn the long-range dependencies. As mentioned earlier, the primary issue with these models is that they require sufficient labeled training data. To address this challenge, we proposed a spectral-spatial feature extractor group attention transformer that consists of a multiscale feature extractor to extract low-level or shallow features. For high-level semantic feature extraction, we proposed a group attention mechanism. Our proposed model is evaluated using four publicly available HSI datasets, which are Indian Pines, Pavia University, Salinas, and the KSC dataset. Our proposed approach achieved the best classification results in terms of overall accuracy (OA), average accuracy (AA), and Kappa coefficient. As mentioned earlier, the proposed approach utilized only 5%, 1%, 1%, and 10% of the training samples from the publicly available four datasets.

Hybrid Swin-CSRNet: A Novel and Efficient Fish Counting Network in Aquaculture

Real-time estimation of fish biomass plays a crucial role in real-world fishery production, as it helps formulate feeding strategies and other management decisions. In this paper, a dense fish counting network called Swin-CSRNet is proposed. Specifically, the VGG16 layer in the front-end is replaced with the Swin transformer to extract image features more efficiently. Additionally, a squeeze-and-excitation (SE) module is introduced to enhance feature representation by dynamically adjusting the importance of each channel through “squeeze” and “excitation”, making the extracted features more focused and effective. Finally, a multi-scale fusion (MSF) module is added after the back-end to fully utilize the multi-scale feature information, enhancing the model’s ability to capture multi-scale details. The experiment demonstrates that Swin-CSRNet achieved excellent results with MAE, RMSE, and MAPE and a correlation coefficient R2 of 11.22, 15.32, 5.18%, and 0.954, respectively. Meanwhile, compared to the original network, the parameter size and computational complexity of Swin-CSRNet were reduced by 70.17% and 79.05%, respectively. Therefore, the proposed method not only counts the number of fish with higher speed and accuracy but also contributes to advancing the automation of aquaculture.

Enhanced Abandoned Object Detection through Adaptive Dual-Background Modeling and SAO-YOLO Integration

Abandoned object detection is a critical task in the field of public safety. However, existing methods perform poorly when detecting small and occluded objects, leading to high false detection and missed detection rates. To address this issue, this paper proposes an abandoned object detection method that integrates an adaptive dual-background model with SAO-YOLO (Small Abandoned Object YOLO). The goal is to reduce false and missed detection rates for small and occluded objects, thereby improving overall detection accuracy. First, the paper introduces an adaptive dual-background model that adjusts according to scene changes, reducing noise interference in the background model. When combined with an improved PFSM (Pixel-based Finite State Machine) model, this enhances detection accuracy and robustness. Next, a network model called SAO-YOLO is designed. Key improvements within this model include the SAO-FPN (Small Abandoned Object FPN) feature extraction network, which fully extracts features of small objects, and a lightweight decoupled head, SODHead (Small Object Detection Head), which precisely extracts local features and enhances detection accuracy through multi-scale feature fusion. Finally, experimental results show that SAO-YOLO increases mAP@0.5 and mAP@0.5:0.95 by 9.0% and 5.1%, respectively, over the baseline model. It outperforms other advanced detection models. Ultimately, after a series of experiments on the ABODA, PETS2006, and AVSS2007 datasets, the proposed method achieved an average detection precious of 91.1%, surpassing other advanced methods. It significantly outperforms other advanced detection methods. This approach notably reduces false and missed detections, especially for small and occluded objects.