Fusion Method Research Articles

Model for Inverting the Leaf Area Index of Green Plums by Integrating IoT Environmental Monitoring Data and Leaf Relative Content of Chlorophyll Values

The quantitative inversion of the leaf area index (LAI) of green plum trees is crucial for orchard field management and yield prediction. The data on the relative content of chlorophyll (SPAD) in leaves and environmental data from orchards show a significant correlation with LAI. Effectively integrating these two data types for LAI inversion is important to explore. This study proposes a multi−source decision fusion LAI inversion model for green plums based on their adjusted determination coefficient (MDF−ADRS). First, three statistical methods—Pearson, Spearman rank, and Kendall rank correlation analyses—were used to measure the linear relationships between variables, and the six environmental factors most highly correlated with LAI were selected from the orchard’s environmental data. Then, using multivariate statistical analysis methods, LAI inversion models based on environmental feature factors (EFs−PM) and SPAD (SPAD−PM) were established. Finally, a weight optimization allocation strategy was employed to achieve a multi−source decision fusion LAI inversion model for green plums. This strategy adaptively allocates weights based on the predictive performance of each data source. Unlike traditional models that rely on fixed weights or a single data source, this approach allows the model to increase the influence of a key data source when its predictive strength is high and reduce noise interference when it is weaker. This dynamic adjustment not only enhances the model’s robustness under varying environmental conditions but also effectively mitigates potential biases when a particular data source becomes temporarily unreliable. Our experimental results show that the MDF−ADRS model achieves an R2 of 0.88 and an RMSE of 0.39 in the validation set, outperforming other fusion methods. Compared to the EFs−PM and SPAD−PM models, the R2 increased by 0.19 and 0.26, respectively, and the RMSE decreased by 0.16 and 0.22. This model effectively integrates multiple sources of data from green plum orchards, enabling rapid inversion and improving the accuracy of green plum LAI estimation, providing a technical reference for monitoring the growth and managing the production of green plums.

Refined subsidence monitoring and dynamic prediction in narrow and long mining areas based on InSAR and probabilistic integral method.

Continuous exploitation in mining areas damages the surrounding environment and has various severe geological impacts. Hence, long-term monitoring of mining areas is crucial to reducing these impacts. Differential interferometric synthetic aperture radar (D-InSAR) is widely applied to monitor the subsidence in mining areas, but it cannot obtain accurate large-gradient subsidence result in the centre of the subsidence basin in mining areas due to the de-coherence phenomenon. The probability integral method (PIM) is a prediction method that can cooperate with D-InSAR (D-InSAR PIM, DPIM) to solve the problem. However, with this method, there is early convergence of the edge subsidence in narrow and long mining areas. Moreover, the PIM can only predict the spatial domain; it cannot achieve dynamic prediction. To address the above problems in the traditional DPIM data processing process, in this study, firstly, the traditional PIM was improved by adjusting the radius of the parameter and constructed an improved DPIM (IDPIM) method. The hybrid algorithm was applied to solve the parameters of the IDPIM method and then acquire subsidence results, thus solving the early convergence of edge subsidence problem characteristic of traditional PIM prediction in mining. Additionally, an area-weighting based fusion method was proposed to integrate the IDPIM results and the D-InSAR results (Area-weighting based fusion of the IDPIM and D-InSAR results, AFID) achieving whole-basin refined subsidence in mining areas. Secondly, based on a summary of subsidence laws in mining areas, the Hossfeld model was introduced and combined with the IDPIM method (IDPIM Hossfeld, IDPIM-H) to construct a subsidence dynamic prediction method. This achieved dynamic prediction of the subsidence in mining areas. A coal mine in Ordos was used as the study area, and the feasibility of the IDPIM method, the AFID method and the DPIM-H method was verified through a comparative analysis of leveling data. The results demonstrated that: (1) The results of the IDPIM method showed 8% and 66% improvement in RMSE along the striking and dip lines, respectively, over the D-InSAR results, improving the early convergence of the DPIM along the dip direction of the mine. (2) The results of the AFID method provide a 69% improvement in whole-basin RMSE over the D-InSAR results, which improves the lack of monitoring capacity in the D-InSAR technology center. (3) The results of the DPIM-H method provide a 35% improvement in basin-wide RMSE over the D-InSAR results, solving the problem of low temporal resolution of the D-InSAR technology and realizing the dynamic prediction with high temporal resolution. These findings provide a theoretical basis for future refined exploration of dynamic subsidence in mining areas.

A Comparative approach of Depression Detection: A state of Art

Depression is an unconscious state of mind; it directly affects the mental health condition of patients, so it is required to accurately detect with a certain time to prevent severe depression. Depression detection with traditional approaches includes the patient's interviews and self-reporting to doctors or psychotherapists, but this approach faces challenges like time consumption and subjectivity. Another approach is to detect depression by using some computational techniques, including ‘machine learning (ML)’, ‘deep learning’, ‘natural language processing’, and also some wearable technologies or devices. The study compares the hybrid ensemble model and the late fusion method with other classifiers that are used for depression detection based on various performance parameters, like ‘accuracy’, classification report, ‘mean squared error’, and R2 score , etc. This study also discusses some ethical terms, such as the privacy of patient data and the risk of misdiagnosis. The study highlights the multimodal approach of various modality to build an effective and personalized system for depression detection.

A New Fusion Method of Fuzzy Numbers and Linguistic Terms Based on Individual Semantics in Mixed Decision Making Problems

A New Fusion Method of Fuzzy Numbers and Linguistic Terms Based on Individual Semantics in Mixed Decision Making Problems

Dual-encoder architecture for metal artifact reduction for kV-cone-beam CT images in head and neck cancer radiotherapy.

During a radiotherapy(RT) course, geometrical variations of target volumes, organs at risk, weight changes (loss/gain), tumor regression and/or progression can significantly affect the treatment outcome. Adaptive RT has become the effective methods along with technical advancements in imaging modalities including cone-beam computed tomography (CBCT). Planning CT (pCT) can be modified via deformable image registration (DIR), which is applied to the pair of pCT and CBCT. However, the artifact existed in both pCT and CBCT is a vulnerable factor in DIR. The dose calculation on CBCT is also suggested. Missing information due to the artifacts hinders the accurate dose calculation on CBCT. In this study, we aim to develop a deep learning-based metal artifact reduction(MAR) model to reduce the metal artifacts in CBCT for head and neck cancer RT. To train the proposed MAR model, we synthesized the kV-CBCT images including metallic implants, with and without metal artifacts (simulated image data pairs) through sinogram image handling process. We propose the deep learning architecture which focuses on both artifact removal and reconstruction of anatomic structure using a dual-encoder architecture. We designed four single-encoder models and three dual-encoder models based on UNet (for an artifact removal) and FusionNet (for a tissue restoration). Each single-encoder model contains either UNet or FusionNet, while the dual-encoder models have both UNet and FusionNet architectures. In the dual-encoder models, we implemented different feature fusion methods, including simple addition, spatial attention, and spatial/channel wise attention. Among the models, a dual-encoder model with spatial/channel wise attention showed the highest scores in terms of peak signal-to-noise ratio, mean squared error, structural similarity index, and Pearson correlation coefficient. CBCT images from 34 head and neck cancer patients were used to test the developed models. The dual-encoder model with spatial/channel wise attention showed the best results in terms of artifact index. By using the proposed model to CBCT, one can achieve more accurate synthetic pCT for head and neck patients as well as better tissue recognition and structure delineation for CBCT image itself.

3D Point Cloud Fusion Method Based on EMD Auto-Evolution and Local Parametric Network

Although the development of high-resolution remote sensing satellite technology has made it possible to reconstruct the 3D structure of object-level features using satellite imagery, the results from a single reconstruction are often insufficient to comprehensively describe the 3D structure of the target. Therefore, developing an effective 3D point cloud fusion method can fully utilize information from multiple observations to improve the accuracy of 3D reconstruction. To this end, this paper addresses the problems of shape distortion and sparse point cloud density in existing 3D point cloud fusion methods by proposing a 3D point cloud fusion method based on Earth mover’s distance (EMD) auto-evolution and local parameterization network. Our method is divided into two stages. In the first stage, EMD is introduced as a key metric for evaluating the fusion results, and a point cloud fusion method based on EMD auto-evolution is constructed. The method uses an alternating iterative technique to sequentially update the variables and produce an initial fusion result. The second stage focuses on point cloud optimization by constructing a local parameterization network for the point cloud, mapping the upsampled point cloud in the 2D parameter domain back to the 3D space to complete the optimization. Through these two steps, the method achieves the fusion of two sets of non-uniform point cloud data obtained from satellite stereo images into a single, denser 3D point cloud that more closely resembles the true target shape. Experimental results demonstrate that our fusion method outperforms other classical comparison algorithms for targets such as buildings, planes, and ships, and achieves a fused RMSE of approximately 2 m and an EMD accuracy better than 0.5.

Multimodal Sentiment Analysis for the Malay Language: New Corpus using CNN-based Framework

Sentiment analysis in the Malay language has traditionally focused on text-based data. Malay is the native language of Malaysia and other surrounding countries. While text-based sentiment analysis has shown good performance, it often lacks accuracy due to the absence of the speaker's affective state and intentions, which can lead to misinterpretations. Multimodal sentiment analysis addresses these shortcomings and has demonstrated improved performance in various prediction tasks. Unfortunately, there has been little research in this area for the Malay language, due to a lack of corpus and baseline studies. This paper introduces a new Malay Multimodal Sentiment Corpus, ‘MyMSC’, with annotations at both the multimodal and unimodal text levels. It contains 1208 segments covering political and social topics. The corpus development processes are described in detail, along with the necessary guidelines and considerations. This paper proposes a CNN-based framework with a late fusion method as the baseline model. Experiments with the proposed model demonstrate that the multimodal approach (F1 score = 0.77) outperforms the unimodal approach (F1 score = 0.68), validating the contribution of multimodality to classification performance. The differences between the two types of annotations and their impact are further elaborated. The full corpus is available at https://github.com/serenaroseaini/MyMSC .

RUSNet: Robust fish segmentation in underwater videos based on adaptive selection of optical flow

Fish segmentation in underwater videos can be used to accurately determine the silhouette size of fish objects, which provides key information for fish population monitoring and fishery resources survey. Some researchers have utilized underwater optical flow to improve the fish segmentation accuracy of underwater videos. However, the underwater optical flow is not evaluated and screen in existing works, and its predictions are easily disturbed by motion of non-fish. Therefore, in this paper, by analyzing underwater optical flow data, we propose a robust underwater segmentation network, RUSNet, with adaptive screening and fusion of input information. First, to enhance the robustness of the segmentation model to low-quality optical flow inputs, a global optical flow quality evaluation module is proposed for evaluating and aligning the underwater optical flow. Second, a decoder is designed by roughly localizing the fish object and then applying the proposed multidimension attention (MDA) module to iteratively recover the rough localization map from the spatial and edge dimensions of the fish. Finally, a multioutput selective fusion method is proposed in the testing stage, in which the mean absolute error (MAE) of the prediction using a single input is compared with that obtained using multisource input. Then, the information with the highest confidence is selected for predictive fusion, which facilitates the acquisition of the ultimate underwater fish segmentation results. To verify the effectiveness of the proposed model, we trained and evaluated it using a publicly available joint underwater video dataset and a separate DeepFish public dataset. Compared with the advanced underwater fish segmentation model, the proposed model has greater robustness to low-quality background optical flow in the DeepFish dataset, with the mean pixel accuracy (mPA) and mean intersection over union (mIoU) values reaching 98.77% and 97.65%, respectively. On the joint dataset, the mPA and mIoU of the proposed model are 92.61% and 90.12%, respectively, which are 0.72% and 1.21% higher than those of the advanced underwater video object segmentation model MSGNet. The results indicate that the proposed model can adaptively select the input and accurately segment fish in complex underwater scenes, which provides an effective solution for investigating fishery resources.

Enhancing Long-Term Robustness of Inter-Space Laser Links in Space Gravitational Wave Detection: An Adaptive Weight Optimization Method for Multi-Attitude Sensors Data Fusion

The stable and high-precision acquisition of attitude data is crucial for sustaining the long-term robustness of laser links to detect gravitational waves in space. We introduce an effective method that utilizes an adaptive weight optimization approach for the fusion of attitude data obtained from charge-coupled device (CCD) spot-positioning-based attitude measurements, differential power sensing (DPS), and differential wavefront sensing (DWS). This approach aims to obtain more robust and lower-noise-level attitude data. A system is designed based on the Michelson interferometer for link simulations; validation experiments are also conducted. The experimental results demonstrate that the fused data exhibit higher robustness. Even in the case of a single sensor failure, valid attitude data can still be obtained. Additionally, the fused data have lower noise levels, with root mean square errors of 9.5%, 37.4%, and 93.4% for the single CCD, DPS, and DWS noise errors, respectively.

A modified extended Kalman filter based fusion of auto-correlation data for robot SLAM estimation

Abstract. With the popularization of robot technology, various technological developments related to robots have gradually entered a stage of rapid development. Especially path planning technology, which has been a very important and worthwhile part of robot motion and behavior since the concept of robots emerged. In this technology, various obstacles, environmental changes, and efficiency factors usually need to be considered. Whether in the past, present, or future, robot behavior pursues greater precision, reliability, adaptability to the environment, and even anthropomorphism. In order to achieve these goals, the idea of data fusion has emerged in the development of robot data processing. Therefore, the research direction is to use Kalman filtering for noise filtering and data correction to improve the accuracy of fused data. By using a feasible and strongly correlated data fusion method, combined with an improved extended Kalman filter, various data generated by robots can be fused in multiple dimensions and have correlations, reducing the negative impact of data differences and improving the quality and reliability of data fusion.

A Review of Data-Driven Intelligent Monitoring for Geological Drilling Processes

The exploration and development of resources and energy are fundamental to human survival and development, and geological drilling is a key method for deep resource and energy exploration. Intelligent monitoring technology can achieve anomaly detection, fault diagnosis, and fault prediction in the drilling process, which is crucial for ensuring production safety and improving drilling efficiency. The drilling process is characterized by complex geological conditions, variable working conditions, and low information value density, which pose a series of difficulties and challenges for intelligent monitoring. This paper reviews the research progress of the data-driven intelligent monitoring of geological drilling processes, focusing on the above difficulties and challenges. It mainly includes multivariate statistics, machine learning, and multi-model fusion. Multivariate statistical methods can effectively handle and analyze complex geological drilling data, while machine learning methods can efficiently extract key patterns and trends from a large amount of geological drilling data. Multi-model fusion methods, by combining the advantages of the first two methods, enhance the ability to handle complex multivariable and nonlinear problems. This review shows that existing research still faces problems such as limited data processing capabilities and insufficient model generalization capabilities. Improving the efficiency of data processing and the generalization capability of models may be the main research directions in the future.

SIFusion: Lightweight infrared and visible image fusion based on semantic injection.

The objective of image fusion is to integrate complementary features from source images to better cater to the needs of human and machine vision. However, existing image fusion algorithms predominantly focus on enhancing the visual appeal of the fused image for human perception, often neglecting their impact on subsequent high-level visual tasks, particularly the processing of semantic information. Moreover, these fusion methods that incorporate downstream tasks tend to be overly complex and computationally intensive, which is not conducive to practical applications. To address these issues, a lightweight infrared and visible light image fusion method known as SIFusion, which is based on semantic injection, is proposed in this paper. This method employs a semantic-aware branch to extract semantic feature information, and then integrates these features into the fused features through a Semantic Injection Module (SIM) to meet the semantic requirements of high-level visual tasks. Furthermore, to simplify the complexity of the fusion network, this method introduces an Edge Convolution Module (ECB) based on structural reparameterization technology to enhance the representational capacity of the encoder and decoder. Extensive experimental comparisons demonstrate that the proposed method performs excellently in terms of visual appeal and advanced semantics, providing satisfactory fusion results for subsequent high-level visual tasks even in challenging scenarios.

UPGCN: User Perception-Guided Graph Convolutional Network for Multimodal Recommendation

To tackle the challenges of cold start and data sparsity in recommendation systems, an increasing number of researchers are integrating item features, resulting in the emergence of multimodal recommendation systems. Although graph convolutional network-based approaches have achieved significant success, they still face two limitations: (1) Users have different preferences for various types of features, but existing methods often treat these preferences equally or fail to specifically address this issue. (2) They do not effectively distinguish the similarity between different modality item features, overlook the unique characteristics of each type, and fail to fully exploit their complementarity. To solve these issues, we propose the user perception-guided graph convolutional network for multimodal recommendation (UPGCN). This model consists of two main parts: the user perception-guided representation enhancement module (UPEM) and the multimodal two-step enhanced fusion method, which are designed to capture user preferences for different modalities to enhance user representation. At the same time, by distinguishing the similarity between different modalities, the model filters out noise and fully leverages their complementarity to achieve more accurate item representations. We performed comprehensive experiments on the proposed model, and the results indicate that it outperforms other baseline models in recommendation performance, strongly demonstrating its effectiveness.

Day2Dark: Pseudo-Supervised Activity Recognition Beyond Silent Daylight

AbstractThis paper strives to recognize activities in the dark, as well as in the day. We first establish that state-of-the-art activity recognizers are effective during the day, but not trustworthy in the dark. The main causes are the limited availability of labeled dark videos to learn from, as well as the distribution shift towards the lower color contrast at test-time. To compensate for the lack of labeled dark videos, we introduce a pseudo-supervised learning scheme, which utilizes easy to obtain unlabeled and task-irrelevant dark videos to improve an activity recognizer in low light. As the lower color contrast results in visual information loss, we further propose to incorporate the complementary activity information within audio, which is invariant to illumination. Since the usefulness of audio and visual features differs depending on the amount of illumination, we introduce our ‘darkness-adaptive’ audio-visual recognizer. Experiments on EPIC-Kitchens, Kinetics-Sound, and Charades demonstrate our proposals are superior to image enhancement, domain adaptation and alternative audio-visual fusion methods, and can even improve robustness to local darkness caused by occlusions. Project page: https://xiaobai1217.github.io/Day2Dark/.

YOLOX-S-TKECB: A Holstein Cow Identification Detection Algorithm

Accurate identification of individual cow identity is a prerequisite for the construction of digital farms and serves as the basis for optimized feeding, disease prevention and control, breed improvement, and product quality traceability. Currently, cow identification faces challenges such as poor recognition accuracy, large data volumes, weak model generalization ability, and low recognition speed. Therefore, this paper proposes a cow identification method based on YOLOX-S-TKECB. (1) Based on the characteristics of Holstein cows and their breeding practices, we constructed a real-time acquisition and preprocessing platform for two-dimensional Holstein cow images and built a cow identification model based on YOLOX-S-TKECB. (2) Transfer learning was introduced to improve the convergence speed and generalization ability of the cow identification model. (3) The CBAM attention mechanism module was added to enhance the model’s ability to extract features from cow torso patterns. (4) The alignment between the apriori frame and the target size was improved by optimizing the clustering algorithm and the multi-scale feature fusion method, thereby enhancing the performance of object detection at different scales. The experimental results demonstrate that, compared to the traditional YOLOX-S model, the improved model exhibits a 15.31% increase in mean average precision (mAP) and a 32-frame boost in frames per second (FPS). This validates the feasibility and effectiveness of the proposed YOLOX-S-TKECB-based cow identification algorithm, providing valuable technical support for the application of dairy cow identification in farms.

Investigation of hybrid modeling and its transferability in building load prediction used for district heating systems

In the district heating systems, the historical operation data of the buildings in those areas would be partially or entirely missing. The traditional data-driven model is hard to predict the ground truth results because the historical data is not available for model training. However, utilizing the physics-based methods for load calculation takes a long time to process and encounters low accuracy issues. This paper investigates several hybrid models that integrate the data-driven model and the physics-based models with different fusion methods. The physics-based models calculate envelope load and infiltration load, based on Fourier's law and the grand canonical ensemble theory, respectively. After undergoing load processing, features fusion, and residual connection, the best advanced hybrid models generate 21.35%, 16.35%, and 12.73% better prediction results compared with the data-driven model. Moreover, the advanced hybride models also perform strong transferability across all the data quantity groups. In terms of practical application, the advanced hybrid models could be deployed with effective generalization in limited data scenarios and robust transfer capabilities. The selected best model constructed by hybrid modeling displays the highest performance and saves the total training costs with strong transferability.

Cattle identification based on multiple feature decision layer fusion

In farming scenarios, cattle identification has become a key issue for the development of precision farming. In precision livestock farming, single-feature recognition methods are prone to misjudgment in complex scenarios involving multiple cattle obscuring each other during drinking and feeding. This paper proposes a decision-level identification method based on the multi-feature fusion of cattle faces, muzzle patterns, and ear tags. The method utilizes the SOLO algorithm to segment images and employs the FaceNet and PP-OCRv4 networks to extract features for the cattle’s faces, muzzle patterns, and ear tags. These features are compared with the Ground truth, from which the Top 3 features are extracted. The corresponding cattle IDs of these features are then processed using One-Hot encoding to serve as the final input for the decision layer, and various ensemble strategies are used to optimize the model. The results show that using the multimodal decision fusion method makes the recognition accuracy reach 95.74%, 1.4% higher than the traditional optimal unimodal recognition accuracy. The verification rate reaches 94.72%, 10.65% higher than the traditional optimal unimodal recognition verification rate. The research results demonstrate that the multi-feature fusion recognition method has significant advantages in drinking and feeding farm environments, providing an efficient and reliable solution for precise identification and management of cattle in farms and significantly improving recognition accuracy and stability.

Image Dehazing Enhancement Strategy Based on Polarization Detection of Space Targets

In view of the fact that the technology of polarization detection performs better at identifying targets through clouds and fog, the recognition ability of the space target detection system under haze conditions will be improved by applying the technology. However, due to the low ambient brightness and limited target radiation information during space target detection, the polarization information of space target is seriously lost, and the advantages of polarization detection technology in identifying targets through clouds and fog cannot be effectively exerted under the condition of haze detection. In order to solve the above problem, a dehazing enhancement strategy specifically applied to polarization images of space targets is proposed. Firstly, a hybrid multi-channel interpolation method based on regional correlation analysis is proposed to improve the calculation accuracy of polarization information during preprocessing. Secondly, an image processing method based on full polarization information inversion is proposed to obtain the degree of polarization of the image after inversion and the intensity of the image after dehazing. Finally, the image fusion method based on discrete cosine transform is used to obtain the dehazing polarization fusion enhancement image. The effectiveness of the proposed image processing strategy is verified by carrying out simulated and real space target detection experiments. Compared with other methods, by using the proposed image processing strategy, the quality of the polarization images of space targets obtained under the haze condition is significantly improved. Our research results have important practical implications for promoting the wide application of polarization detection technology in the field of space target detection.

Weighted Fusion Method of Marine Gravity Field Model Based on Water Depth Segmentation

Among the marine gravity field models derived from satellite altimetry, the Scripps Institution of Oceanography (SIO) series and Denmark Technical University (DTU) series models are the most representative and are often used to integrate global gravity field models, which were inverted by the deflection of vertical method and sea surface height method, respectively. The fusion method based on the offshore distance used in the EGM2008 model is just model stitching, which cannot realize the true fusion of the two types of marine gravity field models. In the paper, a new fusion method based on water depth segmentation is proposed, which established the Precision–Depth relationship of each model in each water depth segment in the investigated area, then constructed the FUSION model by weighted fusion based on the precision predicted from the Precision–Depth relationship at each grid in the whole region. The application in the South China Sea shows that the FUSION model built by the new fusion method has better accuracy than SIO28 and DTU17, especially in shallow water and offshore areas. Within 20 km offshore, the RMS of the FUSION model is 5.10 mGal, which is 8% and 4% better than original models, respectively. Within 100 m of shallow water, the accuracy of the FUSION model is 4.01 mGal, which is 14% and 12% higher than the original models, respectively. A further analysis shows that the fusion model is in better agreement with the seabed topography than original models. The new fusion method can blend the effective information of original models to provide a higher-precision marine gravity field.

SDR2Tr‐GAN: A Novel Medical Image Fusion Pipeline Based on GAN With SDR2 Module and Transformer Optimization Strategy

ABSTRACTIn clinical practice, radiologists diagnose brain tumors with the help of different magnetic resonance imaging (MRI) sequences and judge the type and grade of brain tumors. It is hard to realize the brain tumor computer‐aided diagnosis system only with a single MRI sequence. However, the existing multiple MRI sequence fusion methods have limitations in the enhancement of tumor details. To improve fusion details of multi‐modality MRI images, a novel conditional generative adversarial fusion network based on three discriminators and a Staggered Dense Residual2 (SDR2) module, named SDR2Tr‐GAN, was proposed in this paper. In the SDR2Tr‐GAN network pipeline, the generator consists of an encoder, decoder, and fusion strategy that can enhance the feature representation. SDR2 module is developed with Res2Net into the encoder to extract multi‐scale features. In addition, a Multi‐Head Spatial/Channel Attention Transformer, as a fusion strategy to strengthen the long‐range dependencies of global context information, is integrated into our pipeline. A Mask‐based constraint as a novel fusion optimization mechanism was designed, focusing on enhancing salient feature details. The Mask‐based constraint utilizes the segmentation mask obtained by the pre‐trained Unet and Ground Truth to optimize the training process. Meanwhile, MI and SSIM loss jointly improve the visual perception of images. Extensive experiments were conducted on the public BraTS2021 dataset. The visual and quantitative results demonstrate that the proposed method can simultaneously enhance both global image quality and local texture details in multi‐modality MRI images. Besides, our SDR2Tr‐GAN outperforms the other state‐of‐the‐art fusion methods regarding subjective and objective evaluation.