Dual Fusion Research Articles

CRISPR-Based Split Luciferase as a Biosensor for Unique DNA Sequences In Situ.

To date, CRISPR-based DNA targeting approaches have typically used fusion proteins between full fluorescent reporters and catalytically inactive Cas9 (dCas9) for imaging rather than detection of endogenous genomic DNA sequences. A promising alternative strategy for DNA targeting is the direct biosensing of user-defined sequences at single copy with single-cell resolution. Our recently described DNA biosensing approach using a dual fusion protein biosensor comprised of two independently optimized fragments of NanoLuc luciferase (NLuc) directionally fused to dCas9 paired with user-defined single-guide RNAs (sgRNAs) could allow users to sensitively detect unique copies of a target sequence in individual living cells using common laboratory equipment such as a microscope or a luminescence-equipped microplate reader. Here we describe a protocol for using such a DNA biosensor noninvasively in situ.

ASSOCIATION OF THE PRESENCE OF BCR-ABL1 GENE REARRANGEMENTS AND MYELOID ABERRANT ANTIGENS IN PRECURSOR B-ACUTE LYMPHOBLASTIC LEUKEMIA (PRE-B-ALL) PATIENTS

Objective: To determine the association of the presence of BCR-ABL1 gene rearrangements and myeloid aberrant antigens in precursor B-ALL patients. Material and Methods: Both males and females, of all age groups, diagnosed with precursor B-ALL on flow cytometry were included in the study. BCR-ABL1 gene rearrangement was identified by performing Fluorescence in situ hybridization (FISH) using Vysis LSI BCR/ABL, Dual color, Dual fusion translocation probe set. The results were noted and compared with the results of flow cytometry. Results: Male patients were 36 (60%) and female patients were 24 (40%) out of total 60 patients. Median age was 22 years (range 1-73). BCR-ABL1 fusion gene was identified in 47 (78%) patients while 13 (22%) patients were negative for BCR-ABL1. Aberrant myeloid antigens were expressed in 24 (40%) patients and all of these patients were BCR-ABL1 positive. None of BCR-ABL1 negative patients expressed aberrant antigens. There was statistically significant association between BCR-ABL1 positivity and CD117 expression (P<0.05). Conclusion: Keywords: BCR-ABL1, Pre B-ALL, Myeloid aberrant antigens, Flow cytometry, FISH.

Dual encoding DDS‐UNet liver tumour segmentation based on multi‐scale deep and shallow feature fusion

AbstractThe fusion and utilization of multi‐scale deep and shallow features are of great significance in liver tumour segmentation. This study proposes a dual encoding DDS‐UNet liver tumour segmentation method based on multi‐scale deep and shallow feature fusion, aiming to fully achieve the fusion and utilization of deep and shallow features and achieve accurate segmentation. The proposed method mainly consists of residual convolution module fusion residual convolution (FRC), dual encoding end fusion module dual encoding end fusion (DEF), and skip connection fusion module jump connection fusion module (JCF). In the residual convolution module, a layer by layer fused residual convolution is used instead of traditional convolution to achieve better training. In the dual encoding end fusion module, multi‐scale feature fusion at the end provides more comprehensive contextual information, solves the loss of spatial geometric information. The skip connection fusion module reduces the interference of invalid features by changing the weights of spatial attention and channel attention on important features. The Dice coefficient, average intersection to union ratio, accuracy, recall, and accuracy indicators tested on the LiTS dataset were 90.37%, 90.16%, 93.78%, 94.91%, and 98.84%, respectively, which are superior to many advanced liver tumor segmentation methods.

Thermal infrared and visible sequences tracking via dual adversarial pixel fusion

Thermal infrared and visible sequences tracking via dual adversarial pixel fusion

Dual contrast attention-guided multi-frequency fusion for multi-contrast MRI super-resolution

Objective. Multi-contrast magnetic resonance (MR) imaging super-resolution (SR) reconstruction is an effective solution for acquiring high-resolution MR images. It utilizes anatomical information from auxiliary contrast images to improve the quality of the target contrast images. However, existing studies have simply explored the relationships between auxiliary contrast and target contrast images but did not fully consider different anatomical information contained in multi-contrast images, resulting in texture details and artifacts unrelated to the target contrast images. Approach. To address these issues, we propose a dual contrast attention-guided multi-frequency fusion (DCAMF) network to reconstruct SR MR images from low-resolution MR images, which adaptively captures relevant anatomical information and processes the texture details and low-frequency information from multi-contrast images in parallel. Specifically, after the feature extraction, a feature selection module based on a dual contrast attention mechanism is proposed to focus on the texture details of the auxiliary contrast images and the low-frequency features of the target contrast images. Then, based on the characteristics of the selected features, a high- and low-frequency fusion decoder is constructed to fuse these features. In addition, a texture-enhancing module is embedded in the high-frequency fusion decoder, to highlight and refine the texture details of the auxiliary contrast and target contrast images. Finally, the high- and low-frequency fusion process is constrained by integrating a deeply-supervised mechanism into the DCAMF network. Main results. The experimental results show that the DCAMF outperforms other state-of-the-art methods. The peak signal-to-noise ratio and structural similarity of DCAMF are 39.02 dB and 0.9771 on the IXI dataset and 37.59 dB and 0.9770 on the BraTS2018 dataset, respectively. The image recovery is further validated in segmentation tasks. Significance. Our proposed SR model can enhance the quality of MR images. The results of the SR study provide a reliable basis for clinical diagnosis and subsequent image-guided treatment.

PCNN orchard heterologous image fusion with semantic segmentation of significance regions

The apple fruits captured from red green and blue (RGB) color camera cannot be accurately identified and localized under natural scenes of orchards due to lighting and shading. We propose a pulse coupled neural network (PCNN) orchard heterogeneous image fusion model for semantic segmentation of saliency regions. The purpose of the model is to improve the accuracy of image segmentation and the quality of heterogeneous source image fusion in complex orchard scenes. The model fuses the time of flight (ToF) and RGB images acquired from orchard scenes. This model consists of an orchard image semantic segmentation module, a parameter optimization module, and a dual PCNN image fusion module. Image semantic segmentation module extracts the significant regions of fruit targets in orchard images. And then, in parameter optimization module, we introduce tent chaos and ranking strategies into human mental search algorithm to optimize the parameters of PCNN model. Finally, we complete the fusion through a dual PCNN module. Six fusion algorithms were used to evaluate the fusion quality of public and self-built datasets. The results showed that the chaotic hierarchical mental search algorithm could better solve the problem of difficult-to-determine PCNN parameters. The improved DeeplabV3 + could also segment clearer and more complete apple regions. Compared with other six fusion algorithms, the fusion experimental results show that the fused images with clearer targets and more prominent textures could be obtained in the proposed model. The proposed model reduces the influence of light and other disturbing factors in fruit recognition. And the fused image contains rich and accurate image information, which is more in line with the perception habit of human eyes.

TeenyNet: a novel lightweight attention model for sunflower disease detection

Sunflower is one of the oilseed crops which is popularly and widely cultivated globally and contributes significantly to human health. Leaf diseases of sunflower seriously affect the growth and yield of sunflower, which directly leads to the loss of agricultural economy. However, existing machine learning algorithms and deep learning techniques are mainly based on large models with attention mechanisms, which lack considerations in computational performance, especially model size. Therefore, this study proposes a lightweight model called TeenyNet to break through the dilemma. First, the designed global multi-frequency feature extraction module decomposes the image to extract multi-frequency multi-scale features. Then, a parameter-free maximum pooling layer further extracts edge and texture features and simplifies the network complexity through downsampling, after which the proposed lightweight dual fusion attention and multi-branching structure fuses all the feature vectors to enhance multidimensional feature learning and accelerate the model convergence. Finally, the fully connected linear layer solves the multi-classification problem of sunflower disease under natural illumination background conditions. The experimental results show that TeenyNet obtains the highest accuracy of 98.94% for sunflower disease recognition with a minimum size of 143 KB and has better recognition performance in comparison experiments. TeenyNet can be effectively used for the detection of sunflower leaf diseases to achieve disease prevention and control.

A TBGAV-Based Image-Text Multimodal Sentiment Analysis Method for Tourism Reviews

To overcome limitations in existing methods for sentiment analysis of tourism reviews, the authors propose an image-text multimodal sentiment analysis method (TBGAV). It consists of three modules: image sentiment extraction, text sentiment extraction, and image-text fusion. The image sentiment extraction module employs a pre-trained VGG19 model to capture sentiment features. The text sentiment extraction module utilizes the tiny bidirectional encoder representations from transformers (TinyBERT) model, incorporating the bidirectional recurrent neural network and attention (BiGRU-Attention) module for deeper sentiment semantics. The image-text fusion module employs the dual linear fusion approach to correlate image-text links and the maximum decision-making approach for high-precision sentiment prediction. TBGAV achieves superior performance on the Yelp dataset with accuracy, recall rates, and F1 scores of 77.51%, 78.01%, and 78.34%, respectively, outperforming existing methods. Accordingly, TBGAV is expected to help improve travel-related recommender systems and marketing strategies.

Reciprocal transformer for hyperspectral and multispectral image fusion

Hyperspectral and multispectral (HS–MS) image fusion aims to reconstruct high-resolution hyperspectral images from low-resolution hyperspectral images and high-resolution multispectral images. While convolutional neural networks have been widely used for HS–MS fusion, their potential is curtailed due to the limited receptive field of each neuron, resulting in inadequate long-range modeling capabilities. Although several Transformer-based HS–MS fusion methods have been proposed, most of them often fail to fully integrate and coordinate the data from the two modalities (i.e., hyperspectral images and multispectral images). Such ineffective interactions significantly compromise the quality of the reconstructed hyperspectral images. In this paper, we introduce a novel reciprocal fusion strategy called the dual cross Transformer-based fusion (DCTransformer) for HS–MS fusion. The model excels in capturing the interplay between different modalities by utilizing directional pairwise multi-head cross-attention, which facilitates the transfer of information between modalities. Additionally, we incorporate a Swin Transformer block post cross-attention to enhance the self-attention within the context. Extensive experiments show that our DCTransformer performs favorably against other recent works on both simulation HSI datasets and real HSI datasets. The source code and pre-trained models are availabe at https://github.com/qingma2016/DCTransformer.

Dual mode information fusion with pre-trained CNN models and transformer for video-based non-invasive anaemia detection

Anaemia, caused by low haemoglobin levels, affects billions of people worldwide. India has one of the highest anaemia rates among developing nations, according to WHO data. Non-invasive, low-cost, and accurate anaemia screening technologies are in high demand because invasive methods are expensive and difficult to administer globally. The current effort aims to construct a reliable anaemia detection system by combining cutting-edge computational methods with the age-old practise of imprecise assessment of haemoglobin levels from pallor in palm. The proposed method utilises time-domain analysis to establish the connection between blood haemoglobin concentration and colour changes in the palm caused by pressure application and release. A smartphone camera sensor which records the full event of palm colour changes generated by a customised device is processed and analysed. Some key frames are selected from extracted frames and passed to the pre-trained deep learning models for feature extractions. These features are then passed through vision transformers for classification and regression tasks. This approach have the power of pre-trained models for feature extraction and utilises the flexibility of transformer encoders and MLP networks for efficient and effective representation learning, classification, and regression. Two different approaches are proposed i.e., decision level and feature level fusion. The suggested approach accurately estimates blood haemoglobin levels with RMSE and accuracy of 0.483 and 96.296%, respectively based on 531 video samples of palm evidence.

Mask-VRDet: A robust riverway panoptic perception model based on dual graph fusion of vision and 4D mmWave radar

With the development of Unmanned Surface Vehicles (USVs), the perception of inland waterways has become significant to autonomous navigation. RGB cameras can capture images with rich semantic features, but they would fail in adverse weather and at night. As a perception sensor that has initially emerged in recent years, 4D millimeter-wave radar (4D mmWave radar) can work in all weather and has more abundant point-cloud features than ordinary radar, but it also suffers from water-surface clutter seriously. Furthermore, the shape and outline of dense point cloud captured by 4D mmWave radar are irregular. CNN-based neural networks treat features as 2D rectangle grids, which excessively favor image modality and are unfriendly to radar modality. Therefore, we transform both features of image and radar into non-Euclidean space as graph structures. In this paper, we focus on robust panoptic perception in inland waterways. Firstly, we propose the first Clutter-Point-Removal (CPR) algorithm for 4D mmWave radar, removing water-surface clutter and improving the recall of radar targets. Secondly, we propose a high-performance panoptic perception model based on the graph neural network called Mask-VRDet, fusing features of vision and radar to simultaneously perform object detection and semantic segmentation. To the best of our knowledge, Mask-VRDet is the first riverway panoptic perception model based on vision-radar graphical fusion. It outperforms other single-modal and fusion models, and achieves state-of-the-art performance on our collected dataset. We release our code at https://github.com/GuanRunwei/Mask-VRDet-Official.

Efficient Person Reidentification for IoT-Assisted Cyber–Physical Systems

The main objective of this study is to propose a cyber-physical system (CPS) based person re-identification framework for smart surveillance. The Internet of Things (IoT) based interconnected vision sensors in smart cities are considered essential elements of a CPS, and contribute significantly to urban security. However, the re-identification of targeted persons using emerging edge AI techniques still faces certain challenges. To improve efficiency at the edge and overcome the traditional sensing of video cameras, we employed an AI-based person re-identification framework for CPS that is functional in IoT environments. In addition, we present dual attention dilated network (DADNet), which integrates an energy-efficient convolutional neural network (CNN) with a self-attention module to substantially improve the person matching probability. Further, we applied dual feature fusion to intelligently integrate discriminative and robust features using early and late fusion strategies that allow DADNet to significantly consider the foreground and marginally utilize the background information. Furthermore, we impose diversity orthogonality regularization over several CNN layers, which boosts the performance of DADNet, resulting in an appropriate usage over IoT networks. A comprehensive set of ablation studies, comparison with other state-of-the-art approaches, and a time complexity analysis confirm the strength of our DADNet for re-identification tasks in AI-enabled IoT settings that are well-suited for a CPS.

DualGGAN: A New Facial Style Transfer Network

In response to the background penetration problem of unsupervised style transfer algorithms in most cases, a Transformer style transfer network DualGGAN based on dual generators and fusion of relative position encoding is proposed. The network is trained using the least squares generative adversarial network, and the neural network is used as the image feature extractor to generate feature maps to obtain facial image features with attention weights from the feature maps, utilizing relative position encoding and mask loss to jointly constrain feature region style transfer. The experimental results show that the DualGGAN network effectively reduces artifact generation when implementing facial style transfer, maintains good background consistency, and has good generalization ability. Experiments have shown that the FID and KID indicators on the cat2dog and older2adult datasets are significantly improved compared to other algorithms.

FineStyle: Semantic-Aware Fine-Grained Motion Style Transfer with Dual Interactive-Flow Fusion.

We present FineStyle, a novel framework for motion style transfer that generates expressive human animations with specific styles for virtual reality and vision fields. It incorporates semantic awareness, which improves motion representation and allows for precise and stylish animation generation. Existing methods for motion style transfer have all failed to consider the semantic meaning behind the motion, resulting in limited controls over the generated human animations. To improve, FineStyle introduces a new cross-modality fusion module called Dual Interactive-Flow Fusion (DIFF). As the first attempt, DIFF integrates motion style features and semantic flows, producing semantic-aware style codes for fine-grained motion style transfer. FineStyle uses an innovative two-stage semantic guidance approach that leverages semantic clues to enhance the discriminative power of both semantic and style features. At an early stage, a semantic-guided encoder introduces distinct semantic clues into the style flow. Then, at a fine stage, both flows are further fused interactively, selecting the matched and critical clues from both flows. Extensive experiments demonstrate that FineStyle outperforms state-of-the-art methods in visual quality and controllability. By considering the semantic meaning behind motion style patterns, FineStyle allows for more precise control over motion styles. Source code and model are available on https://github.com/XingliangJin/Fine-Style.git.

Improving agricultural field parcel delineation with a dual branch spatiotemporal fusion network by integrating multimodal satellite data

Accurate spatial information for agricultural field parcels is important for agricultural production management and understanding agro-industrialization and intensification. However, traditional remote sensing methods that rely on single-modal or single-date data struggle to identify heterogeneous field parcels, particularly in regions dominated by smallholder farming systems. To address this challenge, we proposed a Dual branch Spatiotemporal Fusion Network (DSTFNet) that integrated very high-resolution (VHR) images and medium-resolution satellite image time series (MRSITS) to extract agricultural field parcels over various landscapes. The DSTFNet consisted of two branches: a spatial branch that extracted spatial features from VHR images and a temporal branch that explored seasonal spectral dynamics from MRSITS data by using ConvLSTM units and an attention module. We evaluated the DSTFNet in four regions across China by using GF-2 and Sentinel-2 data. The results showed that DSTFNet performed well in delineating agricultural field parcels, achieving the highest Matthew’s correlation coefficient (MCC) = 0.823 for the field extent, the highest F1-score of edge (Fedge) = 0.865 for field boundary, and the lowest errors of segmentation evaluation index (SEI) = 0.191 for the vectorized field parcels in Hubei province. In addition, DSTFNet significantly outperformed two single-branch models that used VHR or MRSITS alone, as well as existing BsiNet, ResUNet_a, UNet and RAUNet models. DSTFNet also showed good spatial transferability in distinct regions without training data (on average, MCC = 0.728, Fedge = 0.729, and SEI = 0.281 for three target regions). Using limited training data to fine-tune the DSTFNet can further improve its ability to delineate field parcels over complex regions. The visualization analysis of temporal attention weights demonstrated that DSTFNet can well capture cropland spectral dynamics, making it advantageous in extracting diverse cropland parcels. By exploiting important spectral, spatial and temporal information from multimodal satellite data, DSTFNet provided an effective, robust, and transferable solution for accurately delineating agricultural field parcels across heterogeneous farming systems.

RTAU-Net: A novel 3D rectal tumor segmentation model based on dual path fusion and attentional guidance

Background and objectiveAccording to the Global Cancer Statistics 2020, colorectal cancer has the third-highest diagnosis rate (10.0 %) and the second-highest mortality rate (9.4 %) among the 36 types. Rectal cancer accounts for a large proportion of colorectal cancer. The size and shape of the rectal tumor can directly affect the diagnosis and treatment by doctors. The existing rectal tumor segmentation methods are based on two-dimensional slices, which cannot analyze a patient's tumor as a whole and lose the correlation between slices of MRI image, so the practical application value is not high. MethodsIn this paper, a three-dimensional rectal tumor segmentation model is proposed. Firstly, image preprocessing is performed to reduce the effect caused by the unbalanced proportion of background region and target region, and improve the quality of the image. Secondly, a dual-path fusion network is designed to extract both global features and local detail features of rectal tumors. The network includes two encoders, a residual encoder for enhancing the spatial detail information and feature representation of the tumor and a transformer encoder for extracting global contour information of the tumor. In the decoding stage, we merge the information extracted from the dual paths and decode them. In addition, for the problem of the complex morphology and different sizes of rectal tumors, a multi-scale fusion channel attention mechanism is designed, which can capture important contextual information of different scales. Finally, visualize the 3D rectal tumor segmentation results. ResultsThe RTAU-Net is evaluated on the data set provided by Shanxi Provincial Cancer Hospital and Xinhua Hospital. The experimental results showed that the Dice of tumor segmentation reached 0.7978 and 0.6792, respectively, which improved by 2.78 % and 7.02 % compared with suboptimal model. ConclusionsAlthough the morphology of rectal tumors varies, RTAU-Net can precisely localize rectal tumors and learn the contour and details of tumors, which can relieve physicians' workload and improve diagnostic accuracy.

Automated Cellular-Level Dual Global Fusion of Whole-Slide Imaging for Lung Adenocarcinoma Prognosis.

Histopathologic whole-slide images (WSI) are generally considered the gold standard for cancer diagnosis and prognosis. Survival prediction based on WSI has recently attracted substantial attention. Nevertheless, it remains a central challenge owing to the inherent difficulties of predicting patient prognosis and effectively extracting informative survival-specific representations from WSI with highly compounded gigapixels. In this study, we present a fully automated cellular-level dual global fusion pipeline for survival prediction. Specifically, the proposed method first describes the composition of different cell populations on WSI. Then, it generates dimension-reduced WSI-embedded maps, allowing for efficient investigation of the tumor microenvironment. In addition, we introduce a novel dual global fusion network to incorporate global and inter-patch features of cell distribution, which enables the sufficient fusion of different types and locations of cells. We further validate the proposed pipeline using The Cancer Genome Atlas lung adenocarcinoma dataset. Our model achieves a C-index of 0.675 (±0.05) in the five-fold cross-validation setting and surpasses comparable methods. Further, we extensively analyze embedded map features and survival probabilities. These experimental results manifest the potential of our proposed pipeline for applications using WSI in lung adenocarcinoma and other malignancies.

Dual Parallel Branch Fusion Network for Road Segmentation in High-Resolution Optical Remote Sensing Imagery

Road segmentation from high-resolution (HR) remote sensing images plays a core role in a wide range of applications. Due to the complex background of HR images, most of the current methods struggle to extract a road network correctly and completely. Furthermore, they suffer from either the loss of context information or high redundancy of details information. To alleviate these problems, we employ a dual branch dilated pyramid network (DPBFN), which enables dual-branch feature passing between two parallel paths when it is merged to a typical road extraction structure. A DPBFN consists of three parts: a residual multi-scaled dilated convolutional network branch, a transformer branch, and a fusion module. Constructing pyramid features through parallel multi-scale dilated convolution operations with multi-head attention block can enhance road features while suppressing redundant information. Both branches after fusing can solve shadow or vision occlusions and maintain the continuity of the road network, especially on a complex background. Experiments were carried out on three datasets of HR images to showcase the stable performance of the proposed method, and the results are compared with those of other methods. The OA in the three data sets of Massachusetts, Deep Globe, and GF-2 can reach more than 98.26%, 95.25%, and 95.66%, respectively, which has a significant improvement compared with the traditional CNN network. The results and explanation analysis via Grad-CAMs showcase the effective performance in accurately extracting road segments from a complex scene.

Defogging Learning Based on an Improved DeepLabV3+ Model for Accurate Foggy Forest Fire Segmentation

In recent years, the utilization of deep learning for forest fire detection has yielded favorable outcomes. Nevertheless, the accurate segmentation of forest fires in foggy surroundings with limited visibility remains a formidable obstacle. To overcome this challenge, a collaborative defogging learning framework, known as Defog DeepLabV3+, predicated on an enhanced DeepLabV3+ model is presented. Improved learning and precise flame segmentation are accomplished by merging the defogging features produced by the defogging branch in the input image. Furthermore, dual fusion attention residual feature attention (DARA) is proposed to enhance the extraction of flame-related features. The FFLAD dataset was developed given the scarcity of specifically tailored datasets for flame recognition in foggy environments. The experimental findings attest to the efficacy of our model, with a Mean Precision Accuracy (mPA) of 94.26%, a mean recall (mRecall) of 94.04%, and a mean intersection over union (mIoU) of 89.51%. These results demonstrate improvements of 2.99%, 3.89%, and 5.22% respectively. The findings reveal that the suggested model exhibits exceptional accuracy in foggy conditions, surpassing other existing models across all evaluation metrics.

A two-stage tool wear prediction approach based on dual fusion of multi-feature and decision-making

A two-stage tool wear prediction approach based on dual fusion of multi-feature and decision-making