Dual-branch Network Research Articles

A Dual-Branch Autoencoder Network for Underwater Low-Light Polarized Image Enhancement

Underwater detection faces uncomfortable illumination conditions, and traditional optical images sensitive to intensity often cannot work well in these conditions. Polarization imaging is a good solution for underwater detection under adverse lighting conditions. However, the process of obtaining polarization information causes it to be more sensitive to noise; serious noise reduces the quality of polarized images and subsequent performance in advanced visual tasks. Unfortunately, the flourishing low-light image enhancement methods applied to intensity images have not demonstrated satisfactory performance when transferred to polarized images. In this paper, we propose a low-light image enhancement paradigm based on the antagonistic properties of polarization parameters. Furthermore, we develop a dual-branch network that relies on a gradient residual dense feature extraction module (GRD) designed for polarized image characteristics and polarization loss, effectively avoiding noise introduced during the direct amplification of brightness, and capable of restoring target contour details. To facilitate a data-driven learning method, we propose a simulation method for underwater low-light polarized images. Extensive experimental results on real-world datasets demonstrate the effectiveness of our proposed approach and its superiority against other state-of-the-art methods.

CMNet: deep learning model for colon polyp segmentation based on dual-branch structure.

Colon cancer is one of the top three diseases in gastrointestinal cancers, and colon polyps are an important trigger of colon cancer. Early diagnosis and removal of colon polyps can avoid the incidence of colon cancer. Currently, colon polyp removal surgery is mainly based on artificial-intelligence (AI) colonoscopy, supplemented by deep-learning technology to help doctors remove colon polyps. With the development of deep learning, the use of advanced AI technology to assist in medical diagnosis has become mainstream and can maximize the doctor's diagnostic time and help doctors to better formulate medical plans. We propose a deep-learning model for segmenting colon polyps. The model adopts a dual-branch structure, combines a convolutional neural network (CNN) with a transformer, and replaces ordinary convolution with deeply separable convolution based on ResNet; a stripe pooling module is introduced to obtain more effective information. The aggregated attention module (AAM) is proposed for high-dimensional semantic information, which effectively combines two different structures for the high-dimensional information fusion problem. Deep supervision and multi-scale training are added in the model training process to enhance the learning effect and generalization performance of the model. The experimental results show that the proposed dual-branch structure is significantly better than the single-branch structure, and the model using the AAM has a significant performance improvement over the model not using the AAM. Our model leads 1.1% and 1.5% in mIoU and mDice, respectively, when compared with state-of-the-art models in a fivefold cross-validation on the Kvasir-SEG dataset. We propose and validate a deep learning model for segmenting colon polyps, using a dual-branch network structure. Our results demonstrate the feasibility of complementing traditional CNNs and transformer with each other. And we verified the feasibility of fusing different structures on high-dimensional semantics and successfully retained the high-dimensional information of different structures effectively.

DGCBG-Net: A dual-branch network with global cross-modal interaction and boundary guidance for tumor segmentation in PET/CT images

Background and objectivesAutomatic tumor segmentation plays a crucial role in cancer diagnosis and treatment planning. Computed tomography (CT) and positron emission tomography (PET) are extensively employed for their complementary medical information. However, existing methods ignore bilateral cross-modal interaction of global features during feature extraction, and they underutilize multi-stage tumor boundary features. MethodsTo address these limitations, we propose a dual-branch tumor segmentation network based on global cross-modal interaction and boundary guidance in PET/CT images (DGCBG-Net). DGCBG-Net consists of 1) a global cross-modal interaction module that extracts global contextual information from PET/CT images and promotes bilateral cross-modal interaction of global feature; 2) a shared multi-path downsampling module that learns complementary features from PET/CT modalities to mitigate the impact of misleading features and decrease the loss of discriminative features during downsampling; 3) a boundary prior-guided branch that extracts potential boundary features from CT images at multiple stages, assisting the semantic segmentation branch in improving the accuracy of tumor boundary segmentation. ResultsExtensive experiments are conducted on STS and Hecktor 2022 datasets to evaluate the proposed method. The average Dice scores of our DGCB-Net on the two datasets are 80.33% and 79.29%, with average IOU scores of 67.64% and 70.18%. DGCB-Net outperformed the current state-of-the-art methods with a 1.77% higher Dice score and a 2.12% higher IOU score. ConclusionsExtensive experimental results demonstrate that DGCBG-Net outperforms existing segmentation methods, and is competitive to state-of-arts.

Detection of Image Tampering Using Deep Learning, Error Levels and Noise Residuals

Images once were considered a reliable source of information. However, when photo-editing software started to get noticed it gave rise to illegal activities which is called image tampering. These days we can come across innumerable tampered images across the internet. Software such as Photoshop, GNU Image Manipulation Program, etc. are applied to form tampered images from real ones in just a few minutes. To discover hidden signs of tampering in an image deep learning models are an effective tool than any other methods. Models used in deep learning are capable of extracting intricate features from an image automatically. Here we proposed a combination of traditional handcrafted features along with a deep learning model to differentiate between authentic and tampered images. We have presented a dual-branch Convolutional Neural Network in conjunction with Error Level Analysis and noise residuals from Spatial Rich Model. For our experiment, we utilized the freely accessible CASIA dataset. After training the dual-branch network for 16 epochs, it generated an accuracy of 98.55%. We have also provided a comparative analysis with other previously proposed work in the field of image forgery detection. This hybrid approach proves that deep learning models along with some well-known traditional approaches can provide better results for detecting tampered images.

A dual-branch neural network for crop disease recognition by integrating frequency domain and spatial domain information

Crop diseases severely affect crop yield and quality, and accurate identification of crop diseases is crucial for disease management. Although deep neural networks have made progress in the task of crop disease identification, the complex environment of crop diseases, including background interference, morphological differences, and scale variations, has led to limited accuracy in disease recognition. To address these issues, this study proposes a dual-branch deep neural network for crop disease identification, integrating both frequency domain and spatial domain information. The frequency branch takes frequency domain information as input to extract rich crop disease frequency component features, while the deformable attention Transformer branch excels in representing global features and selectively focusing on local features of crop diseases. A new fusion method, Multi-Spectral Channel Attention Fusion (MSAF), is adopted to better integrate crop disease frequency and spatial features. Additionally, an improved bias loss function (cv_bias) is proposed to optimize the dual-branch network model, achieving an accuracy of 96.7 % on the test dataset, which is 2.0 % higher than the existing state-of-the-art deformable attention Transformer model. With only 14 M model parameters, this study's model provides an effective method for future applications in complex environment.

A New Dual-Branch Network With Global Information for the Surface Defect Detection on Solar PV Wafer

A New Dual-Branch Network With Global Information for the Surface Defect Detection on Solar PV Wafer

A Dual-Branch Framework With Prior Knowledge for Precise Segmentation of Lung Nodules in Challenging CT Scans.

Lung cancer is one of the deadliest cancers globally, and early diagnosis is crucial for patient survival. Pulmonary nodules are the main manifestation of early lung cancer, usually assessed using CT scans. Nowadays, computer-aided diagnostic systems are widely used to assist physicians in disease diagnosis. The accurate segmentation of pulmonary nodules is affected by internal heterogeneity and external data factors. In order to overcome the segmentation challenges of subtle, mixed, adhesion-type, benign, and uncertain categories of nodules, a new mixed manual feature network that enhances sensitivity and accuracy is proposed. This method integrates feature information through a dual-branch network framework and multi-dimensional fusion module. By training and validating with multiple data sources and different data qualities, our method demonstrates leading performance on the LUNA16, Multi-thickness Slice Image dataset, LIDC, and UniToChest, with Dice similarity coefficients reaching 86.89%, 75.72%, 84.12%, and 80.74% respectively, surpassing most current methods for pulmonary nodule segmentation. Our method further improved the accuracy, reliability, and stability of lung nodule segmentation tasks even on challenging CT scans.

Robust Localization-Guided Dual-Branch Network for Camouflaged Object Segmentation

The existence of camouflage targets is widespread in the natural world, as they blend seamlessly or closely resemble their surrounding environment, making it difficult for the human eye to identify them accurately. In camouflage target segmentation, challenges often arise from the high similarity between the foreground and background, resulting in segmentation errors, imprecise edge detection, and overlooking of small targets. To address these issues, this paper presents a robust localization-guided dual-branch network for the recognition of camouflaged targets. Two crucial branches, i.e., a localization branch and an overall refinement branch are designed and incorporated. The localization branch achieves accurate preliminary localization of camouflaged targets by incorporating the robust localization module, which integrates different high-level feature maps in a partially decoded manner. The overall refinement branch optimizes segmentation accuracy based on the output predictions of the localization branch. Within this branch, the edge refinement module is devised to effectively reduce false negative and false positive interference. By conducting context exploration on each feature layer from top to bottom, this module further enhances the precision of target edge segmentation. Additionally, our network employs five jointly trained output prediction maps and introduces attention-guided heads for diverse prediction maps in the overall refinement branch. This design adjusts the spatial positions and channel weights of different prediction maps, generating output prediction maps based on the emphasis of each output, thereby further strengthening the perception and feature representation capabilities of the model. To improve its ability to generate highly confident and accurate prediction candidate regions, tailored loss functions are designed to cater to the objectives of different prediction maps. We conducted experiments on three publicly available datasets for camouflaged object detection to assess our methodology and compared it with state-of-the-art network models. On the largest dataset COD10K, our method achieved a Structure-measure of 0.827 and demonstrated superior performance in other evaluation metrics, outperforming recent network models.

CGTD-Net: Channel-Wise Global Transformer-Based Dual-Branch Network for Industrial Strip Steel Surface Defect Detection

CGTD-Net: Channel-Wise Global Transformer-Based Dual-Branch Network for Industrial Strip Steel Surface Defect Detection

DBNet-SI: Dual branch network of shift window attention and inception structure for skin lesion segmentation

The U-shaped convolutional neural network (CNN) has attained remarkable achievements in the segmentation of skin lesion. However, given the inherent locality of convolution, this architecture cannot capture long-range pixel dependencies and multiscale global contextual information effectively. Moreover, repeated convolutions and downsampling operations can readily result in the omission of intricate local fine-grained details. In this paper, we proposed a U-shaped network (DBNet-SI) equipped with a dual-branch module that combines shift window attention and inception structures. First, we proposed a dual-branch module that combines shift window attention and inception structures (MSI) to better capture multiscale global contextual information and long-range pixel dependencies. Specifically, we have devised a cross-branch bidirectional interaction module within the MSI module to enable information complementarity between the two branches in the channel and spatial dimensions. Therefore, MSI is capable of extracting distinguishing and comprehensive features to accurately identify the skin lesion boundaries. Second, we have devised a progressive feature enhancement and information compensation module (PFEIC), which progressively compensates for fine-grained features through reconstructed skip connections and integrated global context attention modules. The results of the experiment show the superior segmentation performance of DBNet-SI compared with other deep learning models for skin lesion segmentation in the ISIC2017 and ISIC2018 datasets. Ablation studies demonstrate that our model can effectively extract rich multiscale global contextual information and compensate for the loss of local details.

Duet of ViT and CNN: multi-scale dual-branch network for fine-grained image classification of marine organisms

Fine-grained image classification of marine organisms involves dividing subcategories within a larger category. For instance, this could mean distinguishing specific species of fish or types of algae. This type of classification is more intricate than regular image classification, as the minor feature differences between subcategories are often concentrated in one or a few specific areas. Therefore, accurately identifying these critical regions and effectively using local features are crucial in improving the accuracy of fine-grained image classification. Existing methods for fine-grained image classification primarily rely on single-branch models based on convolutional neural networks (CNNs) or vision transformers (ViTs). Consequently, merging them allows for a more comprehensive understanding of marine organism images. In addition, marine organism images are affected by the distance and angle of the shot, making it challenging to capture detailed local nuances at a single scale. To address these challenges, we propose a multi-scale dual-branch network (MSDBN) that combines the strengths of ViT and CNN for fine-grained image classification of marine organisms. Our model uses a novel two-stage selection module to select discriminative regions from the ViT branch. Following this, the CNN branch executes a more detailed feature extraction on the local regions. To effectively utilise the multi-scale information of marine organisms, we introduce our designed multi-scale shift-window self-attention, specifically for the ViT branch. MSDBN demonstrates improved performance compared to existing classical methods and the best-performing dual-branch methods on three marine datasets. Our code is released publicly at https://github.com/Xiaosigz/MSDBN.

Dual convolutional network based on hypergraph and multilevel feature fusion for road extraction from high-resolution remote sensing images

ABSTRACT Road extraction from high-resolution remote sensing images (HRSI) is confronted with the challenge that roads are occluded by other objects, including opaque obstructions and similarly colored areas. This paper proposes a dual convolutional network based on hypergraph and multilevel feature fusion (DHM) for road extraction to address these challenges. The DHM consists of two branch networks (HGNN branch and CNN branch) and a bimodal feature fusion module (BFFM). In the HGNN branch, an algorithm is developed to exploit the shape features of roads and construct hypergraphs on the HRSI. Then, hypergraph neural networks are employed for the first time to capture the long-range context of roads to enhance road connectivity. In the CNN branch, a bottleneck fusion module integrated with an encoder-decoder network structure is built to aggregate multiscale local features. In BFFM, the long-range context from the HGNN branch and the local features from the CNN branch are fused through the designed position converter and enhanced graph reasoning module to achieve the complementary advantages of the dual-branch network. Extensive experiments on three datasets show that DHM outperforms other state-of-the-art methods, especially on the GS-Mountain road dataset. Furthermore, DHM significantly improves road extraction in occluded and similar road regions.

Fake News Detection Based on Knowledge-Guided Semantic Analysis

Recently, fake news, such as low-quality news with intentionally false information, has threatened the authenticity of news information. However, existing detection methods are inefficient in modeling complicated data and leveraging external knowledge. To address these limitations, we propose a fake news detection framework based on knowledge-guided semantic analysis, which compares the news to external knowledge through triplets for fake news detection. Considering that equivalent elements of triplets may be presented in different forms, a triplet alignment method is designed to construct the bridge between news documents and knowledge graphs. Then, a dual-branch network is developed to conduct interaction and comparison between text and knowledge embeddings. Specifically, text semantics is analyzed with the guidance generated by a triplet aggregation module to capture the inconsistency between news content and external knowledge. In addition, a triplet scoring module is designed to measure rationality in view of general knowledge as a complementary clue. Finally, an interaction module is proposed to fuse rationality scores in aspects of text semantics and external knowledge to obtain detection results. Extensive experiments are conducted on publicly available datasets and several state-of-the-art methods are considered for comparison. The results verify the superiority of the proposed method in achieving more reliable detection results of fake news.

DBPF-net: dual-branch structural feature extraction reinforcement network for ocular surface disease image classification.

Pterygium and subconjunctival hemorrhage are two common types of ocular surface diseases that can cause distress and anxiety in patients. In this study, 2855 ocular surface images were collected in four categories: normal ocular surface, subconjunctival hemorrhage, pterygium to be observed, and pterygium requiring surgery. We propose a diagnostic classification model for ocular surface diseases, dual-branch network reinforced by PFM block (DBPF-Net), which adopts the conformer model with two-branch architectural properties as the backbone of a four-way classification model for ocular surface diseases. In addition, we propose a block composed of a patch merging layer and a FReLU layer (PFM block) for extracting spatial structure features to further strengthen the feature extraction capability of the model. In practice, only the ocular surface images need to be input into the model to discriminate automatically between the disease categories. We also trained the VGG16, ResNet50, EfficientNetB7, and Conformer models, and evaluated and analyzed the results of all models on the test set. The main evaluation indicators were sensitivity, specificity, F1-score, area under the receiver operating characteristics curve (AUC), kappa coefficient, and accuracy. The accuracy and kappa coefficient of the proposed diagnostic model in several experiments were averaged at 0.9789 and 0.9681, respectively. The sensitivity, specificity, F1-score, and AUC were, respectively, 0.9723, 0.9836, 0.9688, and 0.9869 for diagnosing pterygium to be observed, and, respectively, 0.9210, 0.9905, 0.9292, and 0.9776 for diagnosing pterygium requiring surgery. The proposed method has high clinical reference value for recognizing these four types of ocular surface images.

TrUNet: Dual-Branch Network by Fusing CNN and Transformer for Skin Lesion Segmentation

TrUNet: Dual-Branch Network by Fusing CNN and Transformer for Skin Lesion Segmentation

A Dual-Branch Cross-Modality-Attention Network for Thyroid Nodule Diagnosis Based on Ultrasound Images and Contrast-Enhanced Ultrasound Videos.

Contrast-enhanced ultrasound (CEUS) has been extensively employed as an imaging modality in thyroid nodule diagnosis due to its capacity to visualise the distribution and circulation of micro-vessels in organs and lesions in a non-invasive manner. However, current CEUS-based thyroid nodule diagnosis methods suffered from: 1) the blurred spatial boundaries between nodules and other anatomies in CEUS videos, and 2) the insufficient representations of the local structural information of nodule tissues by the features extracted only from CEUS videos. In this paper, we propose a novel dual-branch network with a cross-modality-attention mechanism for thyroid nodule diagnosis by integrating the information from tow related modalities, i.e., CEUS videos and ultrasound image. The mechanism has two parts: US-attention-from-CEUS transformer (UAC-T) and CEUS-attention-from-US transformer (CAU-T). As such, this network imitates the manner of human radiologists by decomposing the diagnosis into two correlated tasks: 1) the spatio-temporal features extracted from CEUS are hierarchically embedded into the spatial features extracted from US with UAC-T for the nodule segmentation; 2) the US spatial features are used to guide the extraction of the CEUS spatio-temporal features with CAU-T for the nodule classification. The two tasks are intertwined in the dual-branch end-to-end network and optimized with the multi-task learning (MTL) strategy. The proposed method is evaluated on our collected thyroid US-CEUS dataset. Experimental results show that our method achieves the classification accuracy of 86.92%, specificity of 66.41%, and sensitivity of 97.01%, outperforming the state-of-the-art methods. As a general contribution in the field of multi-modality diagnosis of diseases, the proposed method has provided an effective way to combine static information with its related dynamic information, improving the quality of deep learning based diagnosis with an additional benefit of explainability.

Local and Global Feature-Aware Dual-Branch Networks for Plant Disease Recognition.

Accurate identification of plant diseases is important for ensuring the safety of agricultural production. Convolutional neural networks (CNNs) and visual transformers (VTs) can extract effective representations of images and have been widely used for the intelligent recognition of plant disease images. However, CNNs have excellent local perception with poor global perception, and VTs have excellent global perception with poor local perception. This makes it difficult to further improve the performance of both CNNs and VTs on plant disease recognition tasks. In this paper, we propose a local and global feature-aware dual-branch network, named LGNet, for the identification of plant diseases. More specifically, we first design a dual-branch structure based on CNNs and VTs to extract the local and global features. Then, an adaptive feature fusion (AFF) module is designed to fuse the local and global features, thus driving the model to dynamically perceive the weights of different features. Finally, we design a hierarchical mixed-scale unit-guided feature fusion (HMUFF) module to mine the key information in the features at different levels and fuse the differentiated information among them, thereby enhancing the model's multiscale perception capability. Subsequently, extensive experiments were conducted on the AI Challenger 2018 dataset and the self-collected corn disease (SCD) dataset. The experimental results demonstrate that our proposed LGNet achieves state-of-the-art recognition performance on both the AI Challenger 2018 dataset and the SCD dataset, with accuracies of 88.74% and 99.08%, respectively.

LG-DBNet: Local and Global Dual-Branch Network for SAR Image Denoising

LG-DBNet: Local and Global Dual-Branch Network for SAR Image Denoising

A Lightweight Dual-Branch Network for Building Change Detection in Remote Sensing Images Integrating Cross-Scale Coupling and Boundary Constraint

A Lightweight Dual-Branch Network for Building Change Detection in Remote Sensing Images Integrating Cross-Scale Coupling and Boundary Constraint

DCTP-Net: Dual-Branch CLIP-Enhance Textual Prompt-Aware Network for Acute Ischemic Stroke Lesion Segmentation From CT Image.

Detecting early ischemic lesions (EIL) in computed tomography (CT) images is crucial for reducing diagnostic time and minimizing neuron loss due to oxygen deprivation. This paper introduces DCTP-Net, a dual-branch network for segmenting acute ischemic stroke lesions in CT images, consisting of a segmentation branch and a prompt-aware branch. The segmentation branch uses an encoder-decoder network as the backbone to identify lesions, where the encoder fuses CT image features with prompt features from the prompt-aware branch. To enhance semantic feature extraction and reduce the impact of cerebral structural details, we introduce a cross-collaboration dynamic connection (CCDC) module to link the encoder and decoder. The prompt-aware branch includes a learnable prompt (LP) block to incorporate cerebral prior knowledge, and the prompt-aware encoder (PAE) combines the LP block with multi-level features from the segmentation branch for more precise representation. Additionally, we propose a CLIP-enhance textual prompt (CETP) module that utilizes the CLIP text encoder to generate specialized convolutional parameters for the segmentation head. These parameters are tailored to the unique characteristics of each input image, improving segmentation performance. Qualitative and quantitative studies reveal that DCTP-Net outperforms the current state-of-the-art, IS-Net, with Dice score increases of 3.9% on AISD and 3.8% on ISLES2018, demonstrating its superiority in EIL segmentation.