Multi-scale Contextual Information Research Articles

Self-Supervised Monocular Depth Estimation for Endoscopic Imaging.

Endoscopy holds a pivotal role in the early detection and treatment of diverse diseases, with artificial intelligence (AI)-assisted methods increasingly gaining prominence in disease screening. Among them, the depth estimation from endoscopic sequences is crucial for a spectrum of AI-assisted surgical techniques. However, the development of endoscopic depth estimation algorithms presents a formidable challenge due to the unique environmental intricacies and constraints within the dataset. This paper proposes a self-supervised depth estimation network to comprehensively explore the brightness changes in endoscopic images, and fuse different features at multiple levels to achieve an accurate prediction of endoscopic depth. First, a FlowNet is designed to evaluate the brightness changes of adjacent frames by calculating the multi-scale structural similarity. Second, a feature fusion module is presented to capture multi-scale contextual information. Experiments show that the average accuracy of the algorithm is 97.03% in the Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED dataset). Based on the training parameters of the SCARED dataset, the algorithm achieves superior performance on the other two datasets (EndoSLAM and KVASIR dataset), indicating that the algorithm has good generalization performance.

Efficient SpineUNetX for X-ray: A spine segmentation network based on ConvNeXt and UNet

Accurate localization and delineation of vertebrae are crucial for diagnosing and treating spinal disorders. To achieve this, we propose an efficient X-ray full-spine vertebra instance segmentation method based on an enhanced U-Net architecture. Several key improvements have been made: the ConvNeXt encoder is employed to effectively capture complex features, and IFE feature extraction is introduced in the skip connections to focus on texture-rich and edge-clear clues. The CBAM attention mechanism is used in the bottleneck to integrate coarse and fine-grained semantic information. The decoder employs a residual structure combined with skip connections to achieve multi-scale contextual information and feature fusion. Our method has been validated through experiments on anterior-posterior and lateral spinal segmentation, demonstrating robust feature extraction and precise semantic segmentation capabilities. It effectively handles various spinal disorders, including scoliosis, vertebral wedging, lumbar spondylolisthesis and spondylolysis. This segmentation foundation enables rapid calibration of vertebral parameters and the computation of relevant metrics, providing valuable references and guidance for advancements in medical imaging.

REDef-DETR: real-time and efficient DETR for industrial surface defect detection

Abstract Industrial surface defect detection is an important part of industrial production, which aims to identify and detecting various defects on the surface of product to ensure quality and meet customer requirements. With the development of deep learning and image processing technologies, the surface defect detection methods based on computer vision has become the mainstream method. However, the prevalent convolutional neural network-based defect detection methods also have many problems. For example, these methods rely on post-processing of Non-Maximum Suppression and have poor detection ability for small targets, which affects the speed and accuracy of surface defect detection in industrial scenarios. Therefore, we propose a novel DEtection TRansformer-based surface defect detection method. Firstly, we propose a Multi-scale Contextual Information Dilated module and fuse it into the backbone. The module is mainly composed of large kernel convolutions, which aims to expand the receptive field of the model, thus reducing the leakage rate of the model. Moreover, we design an efficient encoder which mainly contains two important modules, namely feature enhancement based on cascaded group attention module and efficient feature fusion module based on content-aware. The former module effectively enhances the high-level semantic information extracted by the backbone, thus enabling the model to better interpret features, and it can improve the problem of high computational cost of transformer encoder, thus increasing the detection speed. The latter module performs multi-scale feature fusion across the feature information of various scales, thus improving the detection accuracy of the model for small-size defects. Experimental results show that the proposed method achieves 80.6%mAP and 80.3FPS on NEU-DET, and 98.0%mAP and 79.4FPS on PCB-DET. Our proposed method exhibits excellent detection performance and achieves real-time and efficient surface defect detection capability to meet the needs of industrial surface defect detection.

CDT-CAD: Context-Aware Deformable Transformers for End-to-End Chest Abnormality Detection on X-Ray Images.

Deep learning methods have achieved great success in medical image analysis domain. However, most of them suffer from slow convergency and high computing cost, which prevents their further widely usage in practical scenarios. Moreover, it has been proved that exploring and embedding context knowledge in deep network can significantly improve accuracy. To emphasize these tips, we present CDT-CAD, i.e., context-aware deformable transformers for end-to-end chest abnormality detection on X-Ray images. CDT-CAD first constructs an iterative context-aware feature extractor, which not only enlarges receptive fields to encode multi-scale context information via dilated context encoding blocks, but also captures unique and scalable feature variation patterns in wavelet frequency domain via frequency pooling blocks. Afterwards, a deformable transformer detector on the extracted context features is built to accurately classify disease categories and locate regions, where a small set of key points are sampled, thus leading the detector to focus on informative feature subspace and accelerate convergence speed. Through comparative experiments on Vinbig Chest and Chest Det 10 Datasets, CDT-CAD demonstrates its effectiveness in recognizing chest abnormities and outperforms 1.4% and 6.0% than the existing methods in AP50 and AR on VinBig dateset, and 0.9% and 2.1% on Chest Det-10 dataset, respectively.

DFCAFNet: Dual-feature co-attentive fusion network for diabetic retinopathy grading

Diabetic retinopathy (DR) grading is a complicated task characterized by inter-class variations among several categories, the subtle detection of tiny lesions, and uneven data distribution. To achieve precise DR grading, extracting distinct features that can effectively capture minor visual variations in lesions is necessary. The constraints of conventional convolutional neural networks (CNNs) are evident in their difficulty in detecting tiny lesions, which is further enhanced by their biased focus due to an uneven distribution of data. Furthermore, comparable color and texture attributes among different classes add to the complexity of achieving accurate grading results. This paper presents a new multi-scale attention fusion network (MS-AFNet) incorporating multi-scale contextual information. By successfully combining local and global features, the network improves the diagnostic accuracy of the model in handling inter-class variances. A concurrent attention module (CAM) with multiple attention blocks enhances the ability to distinguish information by capturing complex features within the lesions through numerous attention mechanisms. This novel approach greatly improves the model’s capacity to distinguish nuanced features in minor lesions, resulting in more precise diagnostic outcomes. A dual-feature co-attentive fusion network (DFCAFNet) is proposed by combining CAM and MS-AFNet, which capture spatial patterns and highlight inter-channel interactions to detect complex patterns and enhance accuracy. An extensive assessment of two datasets on DR demonstrates the exceptional accuracy and computational efficiency of DFCAFNet.

Steel Surface Defect Detection Algorithm Based on Improved YOLOv8n

The traditional detection methods of steel surface defects have some problems, such as a lack of feature extraction ability, sluggish detection speed, and subpar detection performance. In this paper, a YOLOv8-based DDI-YOLO model is suggested for effective steel surface defect detection. First, on the Backbone network, the extended residual module (DWR) is fused with the C2f module to obtain C2f_DWR, and the two-step approach is used to carry out the effective extraction of multiscale contextual information, and then fusing feature maps formed from the multiscale receptive fields to enhance the capacity for feature extraction. Also based on the above, an extended heavy parameter module (DRB) is added to the structure of C2f_DWR to make up for the lack of C2f’s ability to capture small-scale pattern defects between training to enhance the training fluency of the model. Finally, the Inner-IoU loss function is employed to enhance the regression accuracy and training speed of the model. The experimental results show that the detection of DDI-YOLO on the NEU-DET dataset improves the mAP by 2.4%, the accuracy by 3.3%, and the FPS by 59 frames/s compared with the original YOLOv8n.Therefore, this paper’s proposed model has a superior mAP, accuracy, and FPS in identifying surface defects in steel.

Multi-scale Segformer: image segmentation algorithm for electric vehicles in fire accident

ABSTRACT Electric vehicles (EVs) have played a significant role in sustainability, and EVs fire accidents have raised doubts in recent years. To solve the mobile analytical equipment limitation in EVs fire accident and help staff receive prompt results at on-spot inspection, we provide a lightweight but accurate Transformer that can ideally adapt to the mobile environment. First, we built on the simple Segformer and extended it to aggregate the representations of amorphous objects, such as fire traces, in image recognition. Second, we used shunt-based self-attention (SSA) to enhance the model for capturing multi-scale contextual information and help distinguish the deformed level of EVs after combustion. Third, we redesigned a simple multi-level information aggregation (MIA) decoder to obtain the relationship between pixels in the channel dimensions by a weighted aggregation. Furthermore, to foster image trace recognition, we put forwards and evaluated the accuracy of models on electric vehicle fire traces (EVFTrace), a dataset of images of burnt EVs. On EVFTrace, the mean intersection over union (mIoU) achieves 72.24%. The float point operations (Flops) and parameters (Params) achieve 114.83 G and 89.5 M. Our model shows excellent efficiency and accuracy for burnt EVs segmentation tasks.

Coreference Resolution Based on High-Dimensional Multi-Scale Information.

Coreference resolution is a key task in Natural Language Processing. It is difficult to evaluate the similarity of long-span texts, which makes text-level encoding somewhat challenging. This paper first compares the impact of commonly used methods to improve the global information collection ability of the model on the BERT encoding performance. Based on this, a multi-scale context information module is designed to improve the applicability of the BERT encoding model under different text spans. In addition, improving linear separability through dimension expansion. Finally, cross-entropy loss is used as the loss function. After adding BERT and span BERT to the module designed in this article, F1 increased by 0.5% and 0.2%, respectively.

A context-ensembled refinement network for image segmentation of coated fuel particles

The thickness of the coating layer of coated fuel particles plays a vital role in the reliable operation of a high-temperature gas-cooled reactor. Coating thickness remains extremely challenging to measure it efficiently, accurately, and robotically. The existing methods struggle with poorly refined segmentation results and easy adhesion of boundaries, complicating the accurate image segmentation of coating layers, which is fundamental to achieving the measurement goal. To overcome these issues, we designed the Context-Ensembled Refinement Network. This network refines the segmentation results to accurately extract different coating regions, leveraging its context fusion module and boundary refinement module to effectively utilize multiscale contextual information and provide strong boundary refinement capability. Experimental results demonstrate that our proposed method outperforms recent state-of-the-art methods, achieving an mIoU value of 95.8 %, and holds promising potential for automatic measurement of coating thickness.

Lightweight multi-scale dynamic selection network for medical image segmentation

Medical image segmentation technology can assist doctors in accurately diagnosing and treating diseases. However, the diversity of segmented targets in scale and shape, as well as the complexity of the background environment, pose challenges for existing segmentation methods. In this context, we propose two effective deep neural networks, the multi-scale dynamic selection network (MDSNet) and its lightweight version MDSNet-Light for medical image segmentation. In MDSNet, we design a multi-scale dynamic selection module to capture multi-scale contextual information and dynamically adjust the acceptance domain of the feature map to achieve feature fusion at the optimal scale. We introduce PixelShuffle and PixelUnshuffle as sampling methods to effectively alleviate information loss issues for small-scale objects caused by pooling. Experimental results show that MDSNet outperforms eight existing superior neural networks in the test data of the publicly available dataset LIDC-IDRI on four evaluation indicators. And MDSNet-Light achieves a good balance between network performance and computational complexity, providing an option for applications in limited computing resource scenarios.

Incorporating convolutional and transformer architectures to enhance semantic segmentation of fine-resolution urban images

ABSTRACT Though convolutional neural networks (CNN) exhibit promise in image semantic segmentation, they have limitations in capturing global context information, resulting in inaccuracies in segmenting small object features and object boundaries. This study introduces a hybrid network, ICTANet, which incorporate convolutional and Transformer architectures to improve the segmentation performance of fine-resolution remote sensing urban imagery. The ICTANet model is essentially a Transformer-based encoder-decoder structure. The dual-encoder architecture, which combines CNN and Swin Transformer modules, is designed to extract both global and local detail information. The feature information at various stages is collected by the Feature Extraction and Fusion modules (FEF), enabling multi-scale contextual information fusion. In addition, an Auxiliary Boundary Detection (ABD) module is introduced at the end of the decoder to enhance the model’s ability to capture object boundary information. Numerous ablation experiments have been conducted to demonstrate the efficacy of various components within the network. The testing results have proven that the proposed model can achieve satisfactory performance on the ISPRS Vaihingen and Potsdam dataset, with overall accuracies of 91.9% and 92.0%, respectively. Simultaneously, the proposed model is also compared to the current state-of-the-art methods, exhibiting competitive performance, particularly in the segmentation of diminutive objects like cars and trees.

Boundary-enhanced dual-stream network for semantic segmentation of high-resolution remote sensing images

ABSTRACT Deep convolutional neural networks (DCNNs) have been successfully used in semantic segmentation of high-resolution remote sensing images (HRSIs). However, this task still suffers from intra-class inconsistency and boundary blur due to high intra-class heterogeneity and inter-class homogeneity, considerable scale variance, and spatial information loss in conventional DCNN-based methods. Therefore, a novel boundary-enhanced dual-stream network (BEDSN) is proposed, in which an edge detection branch stream (EDBS) with a composite loss function is introduced to compensate for boundary loss in semantic segmentation branch stream (SSBS). EDBS and SSBS are integrated by highly coupled encoder and feature extractor. A lightweight multilevel information fusion module guided by channel attention mechanism is designed to reuse intermediate boundary information effectively. For aggregating multiscale contextual information, SSBS is enhanced by multiscale feature extraction module and hybrid atrous convolution module. Extensive experiments have been tested on ISPRS Vaihingen and Potsdam datasets. Results show that BEDSN can achieve significant improvements in intra-class consistency and boundary refinement. Compared with 11 state-of-the-art methods, BEDSN exhibits higher-level performance in both quantitative and visual assessments with low model complexity. The code will be available at https://github.com/lixinghua5540/BEDSN.

GMAlignNet: multi-scale lightweight brain tumor image segmentation with enhanced semantic information consistency

Although the U-shaped architecture, represented by UNet, has become a major network model for brain tumor segmentation, the repeated convolution and sampling operations can easily lead to the loss of crucial information. Additionally, directly fusing features from different levels without distinction can easily result in feature misalignment, affecting segmentation accuracy. On the other hand, traditional convolutional blocks used for feature extraction cannot capture the abundant multi-scale information present in brain tumor images. This paper proposes a multi-scale feature-aligned segmentation model called GMAlignNet that fully utilizes Ghost convolution to solve these problems. Ghost hierarchical decoupled fusion unit and Ghost hierarchical decoupled unit are used instead of standard convolutions in the encoding and decoding paths. This transformation replaces the holistic learning of volume structures by traditional convolutional blocks with multi-level learning on a specific view, facilitating the acquisition of abundant multi-scale contextual information through low-cost operations. Furthermore, a feature alignment unit is proposed that can utilize semantic information flow to guide the recovery of upsampled features. It performs pixel-level semantic information correction on misaligned features due to feature fusion. The proposed method is also employed to optimize three classic networks, namely DMFNet, HDCNet, and 3D UNet, demonstrating its effectiveness in automatic brain tumor segmentation. The proposed network model was applied to the BraTS 2018 dataset, and the results indicate that the proposed GMAlignNet achieved Dice coefficients of 81.65%, 90.07%, and 85.16% for enhancing tumor, whole tumor, and tumor core segmentation, respectively. Moreover, with only 0.29 M parameters and 26.88G FLOPs, it demonstrates better potential in terms of computational efficiency and possesses the advantages of lightweight. Extensive experiments on the BraTS 2018, BraTS 2019, and BraTS 2020 datasets suggest that the proposed model exhibits better potential in handling edge details and contour recognition.

STF-Net: sparsification transformer coding guided network for subcortical brain structure segmentation.

Subcortical brain structure segmentation plays an important role in the diagnosis of neuroimaging and has become the basis of computer-aided diagnosis. Due to the blurred boundaries and complex shapes of subcortical brain structures, labeling these structures by hand becomes a time-consuming and subjective task, greatly limiting their potential for clinical applications. Thus, this paper proposes the sparsification transformer (STF) module for accurate brain structure segmentation. The self-attention mechanism is used to establish global dependencies to efficiently extract the global information of the feature map with low computational complexity. Also, the shallow network is used to compensate for low-level detail information through the localization of convolutional operations to promote the representation capability of the network. In addition, a hybrid residual dilated convolution (HRDC) module is introduced at the bottom layer of the network to extend the receptive field and extract multi-scale contextual information. Meanwhile, the octave convolution edge feature extraction (OCT) module is applied at the skip connections of the network to pay more attention to the edge features of brain structures. The proposed network is trained with ahybrid loss function. The experimental evaluation on two public datasets: IBSR and MALC, shows outstanding performance in terms of objective and subjective quality.

An IIoT Temporal Data Anomaly Detection Method Combining Transformer and Adversarial Training

The existing Industrial Internet of Things (IIoT) temporal data analysis methods often suffer from issues such as information loss, difficulty balancing spatial and temporal features, and being affected by training data noise, which can lead to varying degrees of reduced model accuracy. Therefore, a new anomaly detection method was proposed, which integrated Transformer and adversarial training. Firstly, a bidirectional spatiotemporal feature extraction module was constructed by combining Graph Attention Networks (GAT) and Bidirectional Gated Recurrent Unit (BiGRU), which can simultaneously extract spatial and temporal features. Then, by combining multi-scale convolution with Long Short-Term Memory (LSTM), multi-scale contextual information was captured. Finally, an improved Transformer was used to fuse multi-dimensional features, combined with an adversarial-trained variational autoencoder to calculate the anomalies of the input data. This method outperforms other comparison models by conducting experiments on four publicly available datasets.

Multi-Scale Contextual Swin Transformer for Crop Image Segmentation

Combining UAV-based remote sensing with deep learning for image segmentation is a particularly innovative and effective technology in modern agriculture. This approach allows for detailed and precise analysis of agricultural fields, such as crop monitoring, yield prediction, and irrigation management, enhancing decision-making and farm management practices. Inspired by the recent advancements of Transformers in computer vision, this paper introduces the Multi-Scale Contextual Swin Transformer (MSC-Swin), a novel model for precise segmentation of UAV crop images. MSC-Swin innovatively combines a Swin Transformer architecture for detailed feature extraction with pooling operations to utilize multi-scale contextual information. Our extensive experimentation demonstrates that MSC-Swin not only achieves state-of-the-art performance on the Barley Remote Sensing dataset, with a record mIoU of 86.4% on the test set, but also exhibits robustness and excellent generalizability.

Multi-scale deep residual shrinkage networks with a hybrid attention mechanism for rolling bearing fault diagnosis

The fault diagnosis of rolling bearings based on deep networks is hindered by the unexpected noise involved with accessible vibration signals and global information abatement in deepened networks. To combat the degradation, a multi-scale deep residual shrinkage network with a hybrid attention mechanism (MH-DRSN) is proposed in this paper. First, a spatial domain attention mechanism is introduced into the residual shrinkage module to represent the distance dependence of the feature maps. Then, a hybrid attention mechanism considering both the inner-channeled and cross-channeled characteristics is constructed. Through the comprehensive evaluation of the feature map, it provides a soft threshold for the activation function and realizes the feature-map selection adaptively. Second, the dilated convolution with different dilation rates is implemented for multi-scale context information extraction. Through the feature combination of the DRSN and the dilated convolution, the global information of the rolling bearing fault is strengthened and preserved as the fault diagnosis network is deepened. Finally, the performance of the proposed fault-diagnosis model is validated on the dataset from Case Western Reserve University (CWRU). The experimental results show that, compared with common convolution neural networks, the proposed neural diagnosis model provides a higher identification accuracy and better robustness under noise interference.

基于DeepLabV3的复杂果园环境下樱桃分割与识别

Aiming at the problems of less research on cherry segmentation and identification, with slow recognition speed and low classification accuracy in agricultural products, a method based on DeepLabV3 was proposed to realize the rapid segmentation and identification of cherry in complex orchard environment. Complex environment mainly includes front lighting, back lighting, cloudy and rainy days, single fruit, multi fruit, fruit overlap, and branch and leaf occlusion. This model proposed the Atrous Spatial Pyramid Pooling (ASPP) module to effectively extract multi-scale contextual information, and solved the problem of target segmentation at multiple scales. The obtained data was divided into training, validation and testing sets in 7:1:2 ratios, and the residual network 50 (ResNet50) was selected as backbone of the DeepLabV3. Experimental results show that the algorithm in this paper can segment cherry quickly and accurately, the mean intersection over union (MIoU) was 91.06%, the mean pixel accuracy (MPA) was 93.05%, and the kappa coefficient was 0.89, which was better than fully convolutional networks (FCN), SegNet, DeepLabV1 and DeepLabV2. It is demonstrated that this study can provide technical support for intelligent segmentation of agricultural products.

MSGAT: Multi-scale gated axial reverse attention transformer network for medical image segmentation

Medical image segmentation can help doctors accurately identify and locate various structures, tissues, or lesions in the patient's body, providing an important basis for clinical diagnosis. However, the segmentation of medical images is seriously affected by the complexity of the structures in the images, the variety of morphologies, the uneven contrast, and the blurring of the boundaries between the target tissues and the background. Therefore, we propose a Multiscale Gated Axial Reverse Attention Transformer (MSGAT) for medical image segmentation. First, we use the Pyramid Vision Transformer (PVT) as the backbone network to extract the high-level features and introduce the Channel Received Field Block (CRFB) to filter the interfering information in the high-level features, and then we augment and aggregate the high-level features separately. Specifically, on the one hand, the Multiscale Feature Enhancement Module (MSFEM) is used to obtain multiscale contextual information to enhance the positional information of the target, and on the other hand, Multiscale Parallel Partial Decoder (MSPPD) is employed to aggregate the high-level feature maps to generate rough segmentation maps. Finally, the rough segmentation map is used to guide the gated axial reverse attention (GARA) combined with the position information in the enhanced feature map to refine the segmentation boundaries and gradually mine the detailed information of the target. MSGAT is evaluated against state-of-the-art methods on skin cancer (ISIC2016, PH2), polyp (Kvair), and gland (GlaS) datasets. Results demonstrate MSGAT's superiority, with an average improvement of 6.19% and 9.35% in mIoU and DSC metrics, respectively, compared to existing methods.

FNeXter: A Multi-Scale Feature Fusion Network Based on ConvNeXt and Transformer for Retinal OCT Fluid Segmentation.

The accurate segmentation and quantification of retinal fluid in Optical Coherence Tomography (OCT) images are crucial for the diagnosis and treatment of ophthalmic diseases such as age-related macular degeneration. However, the accurate segmentation of retinal fluid is challenging due to significant variations in the size, position, and shape of fluid, as well as their complex, curved boundaries. To address these challenges, we propose a novel multi-scale feature fusion attention network (FNeXter), based on ConvNeXt and Transformer, for OCT fluid segmentation. In FNeXter, we introduce a novel global multi-scale hybrid encoder module that integrates ConvNeXt, Transformer, and region-aware spatial attention. This module can capture long-range dependencies and non-local similarities while also focusing on local features. Moreover, this module possesses the spatial region-aware capabilities, enabling it to adaptively focus on the lesions regions. Additionally, we propose a novel self-adaptive multi-scale feature fusion attention module to enhance the skip connections between the encoder and the decoder. The inclusion of this module elevates the model's capacity to learn global features and multi-scale contextual information effectively. Finally, we conduct comprehensive experiments to evaluate the performance of the proposed FNeXter. Experimental results demonstrate that our proposed approach outperforms other state-of-the-art methods in the task of fluid segmentation.