Multi-receptive Field Research Articles

Flue-cured tobacco leaf grade detection through multi-receptive field features fusing adaptively and dynamic loss adjustment

Flue-cured tobacco leaf grade detection through multi-receptive field features fusing adaptively and dynamic loss adjustment

An image deblurring method using improved U-Net model based on multilayer fusion and attention mechanism

The investigation of image deblurring techniques in dynamic scenes represents a prominent area of research. Recently, deep learning technology has gained extensive traction within the field of image deblurring methodologies. However, such methods often suffer from limited inherent interconnections across various hierarchical levels, resulting in inadequate receptive fields and suboptimal deblurring outcomes. In U-Net, a more adaptable approach is employed, integrating diverse levels of features effectively. Such design not only significantly reduces the number of parameters but also maintains an acceptable accuracy range. Based on such advantages, an improved U-Net model for enhancing the image deblurring effect was proposed in the present study. Firstly, the model structure was designed, incorporating two key components: the MLFF (multilayer feature fusion) module and the DMRFAB (dense multi-receptive field attention block). The aim of these modules is to improve the feature extraction ability. The MLFF module facilitates the integration of feature information across various layers, while the DMRFAB module, enriched with an attention mechanism, extracts crucial and intricate image details, thereby enhancing the overall information extraction process. Finally, in combination with fast Fourier transform, the FRLF (Frequency Reconstruction Loss Function) was proposed. The FRLF obtains the frequency value of the image by reducing the frequency difference. The present experiment results reveal that the proposed method exhibited higher-quality visual effects. Specifically, for the GoPro dataset, the PSNR (peak signal-to-noise ratio) reached 31.53, while the SSIM (structural similarity index) attained a value of 0.948. Additionally, for the Real Blur dataset, the PSNR achieved 31.32, accompanied by an SSIM score of 0.934.

Brain tumor image segmentation based on improved FPN

PurposeAutomatic segmentation of brain tumors by deep learning algorithm is one of the research hotspots in the field of medical image segmentation. An improved FPN network for brain tumor segmentation is proposed to improve the segmentation effect of brain tumor.Materials and methodsAiming at the problem that the traditional full convolutional neural network (FCN) has weak processing ability, which leads to the loss of details in tumor segmentation, this paper proposes a brain tumor image segmentation method based on the improved feature pyramid networks (FPN) convolutional neural network. In order to improve the segmentation effect of brain tumors, we improved the model, introduced the FPN structure into the U-Net structure, captured the context multi-scale information by using the different scale information in the U-Net model and the multi receptive field high-level features in the FPN convolutional neural network, and improved the adaptability of the model to different scale features.ResultsPerformance evaluation indicators show that the proposed improved FPN model has 99.1% accuracy, 92% DICE rating and 86% Jaccard index. The performance of the proposed method outperforms other segmentation models in each metric. In addition, the schematic diagram of the segmentation results shows that the segmentation results of our algorithm are closer to the ground truth, showing more brain tumour details, while the segmentation results of other algorithms are smoother.ConclusionsThe experimental results show that this method can effectively segment brain tumor regions and has certain generalization, and the segmentation effect is better than other networks. It has positive significance for clinical diagnosis of brain tumors.

Multiclass classification of imagined speech EEG using noise-assisted multivariate empirical mode decomposition and multireceptive field convolutional neural network.

In this study, we classified electroencephalography (EEG) data of imagined speech using signal decomposition and multireceptive convolutional neural network. The imagined speech EEG with five vowels /a/, /e/, /i/, /o/, and /u/, and mute (rest) sounds were obtained from ten study participants. First, two different signal decomposition methods were applied for comparison: noise-assisted multivariate empirical mode decomposition and wavelet packet decomposition. Six statistical features were calculated from the decomposed eight sub-frequency bands EEG. Next, all features obtained from each channel of the trial were vectorized and used as the input vector of classifiers. Lastly, EEG was classified using multireceptive field convolutional neural network and several other classifiers for comparison. We achieved an average classification rate of 73.09 and up to 80.41% in a multiclass (six classes) setup (Chance: 16.67%). In comparison with various other classifiers, significant improvements for other classifiers were achieved (p-value < 0.05). From the frequency sub-band analysis, high-frequency band regions and the lowest-frequency band region contain more information about imagined vowel EEG data. The misclassification and classification rate of each vowel imaginary EEG was analyzed through a confusion matrix. Imagined speech EEG can be classified successfully using the proposed signal decomposition method and a convolutional neural network. The proposed classification method for imagined speech EEG can contribute to developing a practical imagined speech-based brain-computer interfaces system.

CFHA-Net: A polyp segmentation method with cross-scale fusion strategy and hybrid attention

Colorectal cancer is a prevalent disease in modern times, with most cases being caused by polyps. Therefore, the segmentation of polyps has garnered significant attention in the field of medical image segmentation. In recent years, the variant network derived from the U-Net network has demonstrated a good segmentation effect on polyp segmentation challenges. In this paper, a polyp segmentation model, called CFHA-Net, is proposed, that combines a cross-scale feature fusion strategy and a hybrid attention mechanism. Inspired by feature learning, the encoder unit incorporates a cross-scale context fusion (CCF) module that performs cross-layer feature fusion and enhances the feature information of different scales. The skip connection is optimized by proposed triple hybrid attention (THA) module that aggregates spatial and channel attention features from three directions to improve the long-range dependence between features and help identify subsequent polyp lesion boundaries. Additionally, a dense-receptive feature fusion (DFF) module, which combines dense connections and multi-receptive field fusion modules, is added at the bottleneck layer to capture more comprehensive context information. Furthermore, a hybrid pooling (HP) module and a hybrid upsampling (HU) module are proposed to help the segmentation network acquire more contextual features. A series of experiments have been conducted on three typical datasets for polyp segmentation (CVC-ClinicDB, Kvasir-SEG, EndoTect) to evaluate the effectiveness and generalization of the proposed CFHA-Net. The experimental results demonstrate the validity and generalization of the proposed method, with many performance metrics surpassing those of related advanced segmentation networks. Therefore, proposed CFHA-Net could present a promising solution to the challenges of polyp segmentation in medical image analysis. The source code of proposed CFHA-Net is available at https://github.com/CXzhai/CFHA-Net.git.

A Decoupled Semantic–Detail Learning Network for Remote Sensing Object Detection in Complex Backgrounds

Detecting multi-scale objects in complex backgrounds is a crucial challenge in remote sensing. The main challenge is that the localization and identification of objects in complex backgrounds can be inaccurate. To address this issue, a decoupled semantic–detail learning network (DSDL-Net) was proposed. Our proposed approach comprises two components. Firstly, we introduce a multi-receptive field feature fusion and detail mining (MRF-DM) module, which learns higher semantic-level representations by fusing multi-scale receptive fields. Subsequently, it uses multi-scale pooling to preserve detail texture information at different scales. Secondly, we present an adaptive cross-level semantic–detail fusion (CSDF) network that leverages a feature pyramid with fusion between detailed features extracted from the backbone network and high-level semantic features obtained from the topmost layer of the pyramid. The fusion is accomplished through two rounds of parallel global–local contextual feature extraction, with shared learning for global context information between the two rounds. Furthermore, to effectively enhance fine-grained texture features conducive to object localization and features conducive to object semantic recognition, we adopt and improve two enhancement modules with attention mechanisms, making them simpler and more lightweight. Our experimental results demonstrate that our approach outperforms 12 benchmark models on three publicly available remote sensing datasets (DIOR, HRRSD, and RSOD) regarding average precision (AP) at small, medium, and large scales. On the DIOR dataset, our model achieved a 2.19% improvement in mAP@0.5 compared to the baseline model, with a parameter reduction of 14.07%.

MAPD: multi-receptive field and attention mechanism for multispectral pedestrian detection

MAPD: multi-receptive field and attention mechanism for multispectral pedestrian detection

Placental MRI segmentation based on multi-receptive field and mixed attention separation mechanism

ObjectiveTo reduce the occurrence of massive bleeding during placental abruption in patients with placenta accrete, we established a medical imaging based on multi-receptive field and mixed attention separation mechanism (MRF-MAS) model to improve the accuracy of MRI placenta segmentation and provide a basis for subsequent placenta accreta. MethodsWe propose a placenta MRI segmentation technology using the MRF-MAS framework to develop a medical image diagnostic technique. The model first uses the multi-receptive field feature structure to obtain multi-level information, and improves the expression of features at differing scales. Note that the hybrid attention mechanism combines channel attention and spatial attention, separates the input feature sets and computes the attention separately, and finally reorganizes the feature maps. To show that the model can improve the accuracy of segmenting the placenta, we adopt mean Intersection over Union (IoU), Dice similarity coefficient (Dice) and area under the receiver operating characteristic curve (AUC) with U-Net, Mask RCNN, Deeplab v3 for comparison. ResultsThe four models achieved different outcomes based on our placenta dataset, with our model IoU and Dice up to 0.8169 and 0.8992, which are 5.51% and 3.03% higher than the average of the three comparison models. ConclusionThe model proposed by us is helpful to assist the imaging diagnosis and at the same time provides a quantitative reference for the precise treatment of placenta accreta, assists the Equationtion of the clinical operation plan of the physician, and promotes the precision medicine of placenta accreta.

FI-FPN: Feature-integration feature pyramid network for object detection

The multi-layer feature pyramid structure, represented by FPN, is widely used in object detection. However, due to the aliasing effect brought by up-sampling, the current feature pyramid structure still has defects, such as loss of high-level feature information and weakening of low-level small object features. In this paper, we propose FI-FPN to solve these problems, which is mainly composed of a multi-receptive field fusion (MRF) module, contextual information filtering (CIF) module, and efficient semantic information fusion (ESF) module. Particularly, MRF stacks dilated convolutional layers and max-pooling layers to obtain receptive fields of different scales, reducing the information loss of high-level features; CIF introduces a channel attention mechanism, and the channel attention weights are reassigned; ESF introduces channel concatenation instead of element-wise operation for bottom-up feature fusion and alleviating aliasing effects, facilitating efficient information flow. Experiments show that under the ResNet50 backbone, our method improves the performance of Faster RCNN and RetinaNet by 3.5 and 4.6 mAP, respectively. Our method has competitive performance compared to other advanced methods.

Hierarchical interaction and pooling network for co-salient object detection

Co-salient object detection (CoSOD) aims to detect common and salient objects across the given image group. Due to the particularity of CoSOD, images in the given group are processed synergistically to excavate the relevance between them. Inspired by the tracking methods, previous works tend to utilize pixel-wise correspondences to measure the relevance. However, because of the complexity of the image groups, the obtained feature maps could be easily affected by the common interference. Moreover, current works tend to utilize classification labels to ensure intra-group coherence and inter-group separability, which may cause some overfitting problems. In this paper, we propose the hierarchical interaction and pooling network to alleviate the above problems. We first design a pyramid pooling interaction module and perform convolution with dimension permutation, making full use of convolution and multi-receptive field information. We further propose the coherence confirmation module along with the four-branch architecture. Without the classification labels, the module achieves comparable or even better performance. Extensive experiments demonstrate that the proposed method can detect common and salient objects more accurately and achieves the new state-of-the-art.

Multi-Receptive Field Soft Attention Part Learning for Vehicle Re-Identification.

Vehicle re-identification across multiple cameras is one of the main problems of intelligent transportation systems (ITSs). Since the differences in the appearance between different vehicles of the same model are small and the appearance of the same vehicle changes drastically from different viewpoints, vehicle re-identification is a challenging task. In this paper, we propose a model called multi-receptive field soft attention part learning (MRF-SAPL). The MRF-SAPL model learns semantically diverse vehicle part-level features under different receptive fields through multiple local branches, alleviating the problem of small differences in vehicle appearance. To align vehicle parts from different images, this study uses soft attention to adaptively locate the positions of the parts on the final feature map generated by a local branch and maintain the continuity of the internal semantics of the parts. In addition, to obtain parts with different semantic patterns, we propose a new loss function that punishes overlapping regions, forcing the positions of different parts on the same feature map to not overlap each other as much as possible. Extensive ablation experiments demonstrate the effectiveness of our part-level feature learning method MRF-SAPL, and our model achieves state-of-the-art performance on two benchmark datasets.

Extraction of vascular wall in carotid ultrasound via a novel boundary-delineation network

Ultrasound imaging plays an essential role in the diagnosis of vascular lesions. Accurate segmentation of the vascular wall is important for preventing, diagnosing, and treating vascular diseases. However, existing methods have inaccurate localization of the vascular wall boundary. Segmentation errors occur in discontinuous vascular wall boundaries and dark boundaries. To overcome these problems, we propose a new boundary-delineation network (BDNet). First, we design the feature extraction module to prevent the feature information loss of low-quality images by multi-scale information fusion and multi-receptive field feature fusion. Secondly, we generate the initial coarse prediction results based on the extracted features. Based on the coarse prediction results, we use the boundary refinement module to obtain the boundary point locations and re-delineate the boundary points to prevent the boundary points from the offset. Finally, we use region mutual information loss and our designed global pixel relationship loss to model the relationship between pixels from global and neighbourhood aspects using the structural features of the vessel wall to help the model extract important structured information. To facilitate clinical applications, we design the model to be lightweight. Experimental results show that our model achieves the best segmentation results and significantly reduces memory consumption compared to existing models.

Multi-receptive field attention for person re-identification

Multi-receptive field attention for person re-identification

Twist-Net: A multi-modality transfer learning network with the hybrid bilateral encoder for hypopharyngeal cancer segmentation

Hypopharyngeal cancer (HPC) is a rare disease. Therefore, it is a challenge to automatically segment HPC tumors and metastatic lymph nodes (HPC risk areas) from medical images with the small-scale dataset. Combining low-level details and high-level semantics from feature maps in different scales can improve the accuracy of segmentation. Herein, we propose a Multi-Modality Transfer Learning Network with Hybrid Bilateral Encoder (Twist-Net) for Hypopharyngeal Cancer Segmentation. Specifically, we propose a Bilateral Transition (BT) block and a Bilateral Gather (BG) block to twist (fuse) high-level semantic feature maps and low-level detailed feature maps. We design a block with multi-receptive field extraction capabilities, M Block, to capture multi-scale information. To avoid overfitting caused by the small scale of the dataset, we propose a transfer learning method that can transfer priors experience from large computer vision datasets to multi-modality medical imaging datasets. Compared with other methods, our method outperforms other methods on HPC dataset, achieving the highest Dice of 82.98%. Our method is also superior to other methods on two public medical segmentation datasets, i.e., the CHASE_DB1 dataset and BraTS2018 dataset. On these two datasets, the Dice of our method is 79.83% and 84.87%, respectively. The code is available at: https://github.com/zhongqiu1245/TwistNet.

Multireceptive field: An adaptive path aggregation graph neural framework for hyperspectral image classification

In recent years, the applications of graph convolutional networks (GCNs) in hyperspectral image (HSI) classification have attracted much attention. However, hyperspectral classification faces problems such as complex noise effects, spectral variability, labelled training sample deficiency, and high spectral mixing between materials. Furthermore, the available GCN-based methods are computationally complex and cannot automatically adjust aggregate paths. To mitigate these issues, we propose a novel multiadaptive receptive field-based graph neural framework (MARP) for HSI classification. In our method, an adaptive receptive path aggregation (ARP) mechanism is proposed to suppress the impact of noise nodes on classification and automatically explore an adaptive receptive field, where a graph attention (GAT) neural network is introduced to learn the importance of different-sized neighbourhoods and a long short-term memory (LSTM) method is adopted to update the nodes and preserve the local convolutional features of the nodes. To address the problem that ARP may fall into a local optimum, we design a multiscale receptive mechanism. Extensive experimental results obtained on four public HSI datasets demonstrate that the proposed MARP method can mitigate oversmoothing and reduce computational complexity while achieving competitive performance when compared to several state-of-the-art methods.

HIPA: Hierarchical Patch Transformer for Single Image Super Resolution.

Transformer-based architectures start to emerge in single image super resolution (SISR) and have achieved promising performance. However, most existing vision Transformer-based SISR methods still have two shortcomings: (1) they divide images into the same number of patches with a fixed size, which may not be optimal for restoring patches with different levels of texture richness; and (2) their position encodings treat all input tokens equally and hence, neglect the dependencies among them. This paper presents a HIPA, which stands for a novel Transformer architecture that progressively recovers the high resolution image using a hierarchical patch partition. Specifically, we build a cascaded model that processes an input image in multiple stages, where we start with tokens with small patch sizes and gradually merge them to form the full resolution. Such a hierarchical patch mechanism not only explicitly enables feature aggregation at multiple resolutions but also adaptively learns patch-aware features for different image regions, e.g., using a smaller patch for areas with fine details and a larger patch for textureless regions. Meanwhile, a new attention-based position encoding scheme for Transformer is proposed to let the network focus on which tokens should be paid more attention by assigning different weights to different tokens, which is the first time to our best knowledge. Furthermore, we also propose a multi-receptive field attention module to enlarge the convolution receptive field from different branches. The experimental results on several public datasets demonstrate the superior performance of the proposed HIPA over previous methods quantitatively and qualitatively. We will share our code and models when the paper is accepted.

MrFDDGAN: Multireceptive Field Feature Transfer and Dual Discriminator-Driven Generative Adversarial Network for Infrared and Color Visible Image Fusion

Most of the previous infrared and visible image fusion methods based on deep learning were on the basis of grayscale images and used the single convolution kernel receptive field to extract deep features, which would inevitably cause information loss in the process of feature transfer. Accordingly, this paper proposes a new generative adversarial network based on multi-receptive field feature transfer and dual discriminators, which is named MrFDDGAN. It is applied to infrared and color visible image fusion. Firstly, three different receptive fields are used to extract multi-scale and multi-level deep features of multi-modal images on three channels, so as to obtain significant features of source images from multi-view and multi-perspective in a more comprehensive way. Secondly, a feature interaction module is introduced in the encoder to realize the information interaction and information pre-fusion among the three feature channels. Thirdly, the multi-level features fused in the encoder are cascaded with the deep features in the decoder to enhance feature transfer and feature reuse. Finally, this experiment adopts a dual-discriminator adversarial network structure to keep the balance of infrared image intensity and visible image texture preserved by fusion results. And qualitative and quantitative experimental analyses are carried out on two public datasets of grayscale infrared as well as visible images and one dataset of infrared and color visible images. The experimental results prove that the proposed MrFDDGAN algorithm has a better subjective visual effect and objective performance than the existing state-of-the-art fusion methods.

Small unopened cotton boll counting by detection with MRF-YOLO in the wild

Accurate detection and counting of unopened cotton bolls at the early stage of cotton maturation is an effective way to develop crop load management and harvesting strategies in advance. However, robust and effective detection and counting of unopened bolls under complicated field conditions is a challenging task. In this study, we propose a deep learning method with multi-receptive field extraction based on YOLOX, called MRF-YOLO, to detect and count small targets, which we validate on a cotton boll dataset collected from a cotton farm. In the target detection part, a multi-scale residual block and an attention module are first introduced to enhance the extraction of cotton boll feature details. Second, a multi-receptive field extraction module is added to reduce the loss on small targets in the deep network. Finally, a small target detection layer is integrated to improve detection precision. Compared with existing methods, the proposed model achieves a 14.86% improvement in the average accuracy AP50:95 and AP50 of 92.75% while maintaining a high processing speed. In the counting part, we propose a detection-based counting method using MRF-YOLO. The mean squared error and coefficient of determination (R2) are 1.06 and 0.92, respectively. MRF-YOLO can be extended to a wide range of small target crop detection, enabling reliable yield predictions under real field conditions.

A Domain-Adversarial Multi-Graph Convolutional Network for Unsupervised Domain Adaptation Rolling Bearing Fault Diagnosis

The transfer learning method, based on unsupervised domain adaptation (UDA), has been broadly utilized in research on fault diagnosis under variable working conditions with certain results. However, traditional UDA methods pay more attention to extracting information for the class labels and domain labels of data, ignoring the influence of data structure information on the extracted features. Therefore, we propose a domain-adversarial multi-graph convolutional network (DAMGCN) for UDA. A multi-graph convolutional network (MGCN), integrating three graph convolutional layers (multi-receptive field graph convolutional (MRFConv) layer, local extreme value convolutional (LEConv) layer, and graph attention convolutional (GATConv) layer) was used to mine data structure information. The domain discriminators and classifiers were utilized to model domain labels and class labels, respectively, and align the data structure differences through the correlation alignment (CORAL) index. The classification and feature extraction ability of the DAMGCN was significantly enhanced compared with other UDA algorithms by two example validation results, which can effectively achieve rolling bearing cross-domain fault diagnosis.

APT-Net: Adaptive encoding and parallel decoding transformer for medical image segmentation

There are limitations in the study of transformer-based medical image segmentation networks for token position encoding and decoding of images. The position encoding module cannot encode the position information adequately, and the serial decoder cannot utilize the contextual information efficiently. In this paper, we propose a new CNN-transformer hybrid structure for the medical image segmentation network APT-Net based on the encoder-decoder architecture. The network introduces an adaptive position encoding module for the fusion of position information of a multi-receptive field to provide more adequate position information for the token sequences in the transformer. In addition, the dual-path parallel decoder's basic and guide information paths simultaneously process multiscale feature maps to efficiently utilize contextual information. We conducted extensive experiments and reported a number of important metrics from multiple perspectives on seven datasets containing skin lesions, polyps, and glands. The IoU reached 0.783 and 0.851 on the ISIC2017 and Glas datasets, respectively. To the best of our knowledge, APT-Net achieves state-of-the-art performance on the Glas dataset and polyp segmentation tasks. Ablation experiments validate the effectiveness of the proposed adaptive position encoding module and the dual-path parallel decoder. Comparative experiments with state-of-the-art methods demonstrate the high accuracy and portability of APT-Net.