Fusion Method Research Articles

Local element segregation-induced cellular structures and dominant dislocation planar slip enable exceptional strength-ductility synergy in an additively-manufactured CoNiV multicomponent alloy with ageing treatment

We proposed an additively manufactured equiatomic CoNiV multicomponent alloy (MCA) using a conventional laser powder bed fusion (LPBF) method, and an exceptional strength-ductility synergy of the alloy was attained through a simple post-ageing treatment. Pronounced hierarchical microstructures were achieved in our printed alloys, including heterogeneous grain structures, and intragranular cellular structures composed of interior domain with limited dislocations and cell walls led by significant vanadium local segregation. Besides the outstanding mechanical properties at room temperature of 298 K, a giga-pascal yielding strength (> 1.1 GP) and over 40% uniform elongation were attained in the aged specimen deformed at a cryogenic temperature of 77 K, predominating the mechanical properties of many alloys reported in previous works. Such exceptional performance of the aged alloy can be mainly ascribed to considerable local chemical orders (LCOs), aggravated elemental fluctuation in the alloy matrix, and intensified vanadium segregation at walls of intragranular cellular structures which can strongly interact with dislocations. As a result, a planar slip array of dislocations with an extremely high density, namely large numbers of slip bands that can sustain and transfer high strains, dominates the deformation microstructures, thus efficiently strengthening and toughening the aged alloy, especially at a low temperature like 77 K. The above post-ageing strategy is readily and low-costly employed on additively manufactured MCAs with relatively high stacking fault energy (SFE) and proved as a feasible method to produce high-performance structural materials for extreme conditions.

Multistream Adaptive Attention-Enhanced Graph Convolutional Networks for Youth Fencing Footwork Training.

The popularity of fencing and intense sports competition has burdened adolescents with excessive training, harming their immature bodies. Traditional training methods fail to provide timely movement corrections and personalized plans, leading to ineffective exercises. This paper aims to use artificial intelligence technology to reduce ineffective exercises and alleviate the training burden. We propose an action recognition algorithm based on the characteristics of adolescent athletes. This algorithm uses multimodal input data to comprehensively extract action information. Each modality is processed by the same network structure, utilizing attention mechanisms and adaptive graph structures. A multibranch feature fusion method is used to determine the final action category. We gathered the fencing footwork data set 2.0. Our model achieved 93.3% accuracy, with the highest precision at 95.8% and the highest F1-Score at 94.5% across all categories. It effectively recognized actions of adolescents with different heights and speeds, outperforming traditional methods. Our artificial intelligence-based training solution improves training efficiency and reduces the training burden on adolescents.

Echocardiographic mitral valve segmentation model

Segmentation of mitral valve is not only important for clinical diagnosis, but also has far-reaching impact on prevention and prognosis of the disease by experts and doctors.In this paper, the multi-channel cross fusion transformer based U-Net network model (MCCT-UNet) is proposed according to the classical U-Net architecture. First, the jump connection part of MCCT-UNet is designed by using a multi-channel cross-fusion based attention mechanism module (MCCT) instead of the original jump connection, and this module fuses the feature maps from different scales in different stages of the encoder. Second, the optimization of the feature fusion method is proposed in the decoding stage by designing the cross-compression excitation sub-module (C-SENet) to replace the simple feature splicing, and the C-SENet is used to bridge the inconsistency of the semantic hierarchy by effectively combining the deeper information in the encoding stage with the shallower information. This two modules can establish a close connection between the encoder and decoder by exploring multi-scale global contextual information to solve the semantic divide problem, thus it significantly improves the segmentation performance of the network. The experimental results show that the improvement is effective, and the MCCT-UNet model outperforms the other 9 network models. Specifically, the MCCT-UNet achieved a Dice coefficient of 0.8734, an IoU of 0.7854, and an accuracy of 0.9977, demonstrating significant improvements over the compared models.

High-accuracy bathymetric method fusing ICESAT-2 datasets and the two-media photogrammetry model

Improving the accuracy of nearshore bathymetric measurements is essential for understanding coastal environments, resource management, and navigation. The Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) is the first laser satellite that uses the photon-counting technique. The ICESat-2 is equipped with the Advanced Topographic Laser Altimeter System (ATLAS), which enables higher-accuracy measurements of water, ice, and land elevation on Earth. Two-media photogrammetric bathymetry is a type of nearshore bathymetric technology that uses the geometrical characteristics of light rays. With this technique, the accuracy and reliability mainly depend on eliminating systematic errors and ensuring accurate spatial photogrammetric positioning relative to the object being measured. To improve the bathymetric accuracy of two-media photogrammetry, we integrated high-accuracy elevation data from photon datasets as constraining and control parameters. The improved method effectively eliminated systematic errors in two-media photogrammetry during the established joint-block adjustment model. To improve its accuracy and reliability, we employed multispectral WorldView-2 stereo images in our experiments. Furthermore, the bathymetric results were validated and assessed using in situ and photon data. The experimental results show that the highest accuracy achieved with the bathymetric measurements in our study area was a root mean square error (RMSE) of 0.96 m and a mean absolute error of 0.57 m. Using the proposed fusion method, the bathymetric accuracy (as measured using the RMSE) was 1 m higher than that of two-media photogrammetry without the photon datasets.

Activation of Ms 6.9 Milin Earthquake on Sedongpu Disaster Chain, China with Multi-Temporal Optical Images

In recent years, disaster chains caused by glacier movements have occurred frequently in the lower Yarlung Tsangpo River in southwest China. However, it is still unclear whether earthquakes significantly contribute to glacier movements and disaster chains. In addition, it is difficult to measure the high-frequency and large gradient displacement time series with optical remote sensing images due to cloud coverage. To this end, we take the Sedongpu disaster chain as an example, where the Milin earthquake, with an epicenter 11 km away, occurred on 18 November 2017. Firstly, to deal with the cloud coverage problem for single optical remote sensing analysis, we employed multiple platform optical images and conducted a cross-platform correlation technique to invert the two-dimensional displacement rate and the cumulative displacement time series of the Sedongpu glacier. To reveal the correlation between earthquakes and disaster chains, we divided the optical images into three classes according to the Milin earthquake event. Lastly, to increase the accuracy and reliability, we propose two strategies for displacement monitoring, that is, a four-quadrant block registration strategy and a multi-window fusion strategy. Results show that the RMSE reduction percentage of the proposed registration method reaches 80%, and the fusion method can retrieve the large magnitude displacements and complete displacement field. Secondly, the Milin earthquake accelerated the Sedongpu glacier movement, where the pre-seismic velocities were less than 0.5 m/day, the co-seismic velocities increased to 1 to 6 m/day, and the post-seismic velocities decreased to 0.5 to 3 m/day. Lastly, the earthquake had a triggering effect around 33 days on the Sedongpu disaster chain event on 21 December 2017. The failure pattern can be summarized as ice and rock collapse in the source area, large magnitude glacier displacement in the moraine area, and a large volume of sediment in the deposition area, causing a river blockage.

Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks

In order to solve the problem that existing visible and infrared image fusion methods rely only on the original local or global information representation, which has the problem of edge blurring and non-protrusion of salient targets, this paper proposes a layered fusion method based on channel attention mechanism and improved Generative Adversarial Network (HFCA_GAN). Firstly, the infrared image and visible image are decomposed into a base layer and fine layer, respectively, by a guiding filter. Secondly, the visible light base layer is fused with the infrared image base layer by histogram mapping enhancement to improve the contour effect. Thirdly, the improved GAN algorithm is used to fuse the infrared and visible image refinement layer, and the depth transferable module and guided fusion network are added to enrich the detailed information of the fused image. Finally, the multilayer convolutional fusion network with channel attention mechanism is used to correlate the local information of the layered fusion image, and the final fusion image containing contour gradient information and useful details is obtained. TNO and RoadSence datasets are selected for training and testing. The results show that the proposed algorithm retains the global structure features of multilayer images and has obvious advantages in fusion performance, model generalization and computational efficiency.

Leveraging Multi-Modality and Enhanced Temporal Networks for Robust Violence Detection

In this paper, we present a novel model that enhances performance by extending the dual-modality TEVAD model—originally leveraging visual and textual information—into a multi-modal framework that integrates visual, audio, and textual data. Additionally, we refine the multi-scale temporal network (MTN) to improve feature extraction across multiple temporal scales between video snippets. Using the XD-Violence dataset, which includes audio data for violence detection, we conduct experiments to evaluate various feature fusion methods. The proposed model achieves an average precision (AP) of 83.9%, surpassing the performance of single-modality approaches (visual: 73.9%, audio: 67.1%, textual: 29.9%) and dual-modality approaches (visual + audio: 78.8%, visual + textual: 78.5%). These findings demonstrate that the proposed model outperforms models based on the original MTN and reaffirm the efficacy of multi-modal approaches in enhancing violence detection compared to single- or dual-modality methods.

A Deep Unfolding Network for Multispectral and Hyperspectral Image Fusion

Multispectral and hyperspectral image fusion (MS/HS fusion) aims to generate a high-resolution hyperspectral (HRHS) image by fusing a high-resolution multispectral (HRMS) and a low-resolution hyperspectral (LRHS) images. The deep unfolding-based MS/HS fusion method is a representative deep learning paradigm due to its excellent performance and sufficient interpretability. However, existing deep unfolding-based MS/HS fusion methods only rely on a fixed linear degradation model, which focuses on modeling the relationships between HRHS and HRMS, as well as HRHS and LRHS. In this paper, we break free from this observation model framework and propose a new observation model. Firstly, the proposed observation model is built based on the convolutional sparse coding (CSC) technique, and then a proximal gradient algorithm is designed to solve this model. Secondly, we unfold the iterative algorithm into a deep network, dubbed as MHF-CSCNet, where the proximal operators are learned using convolutional neural networks. Finally, all trainable parameters can be automatically learned end-to-end from the training pairs. Experimental evaluations conducted on various benchmark datasets demonstrate the superiority of our method both quantitatively and qualitatively compared to other state-of-the-art methods.

Theories and Methods for Indoor Positioning Systems: A Comparative Analysis, Challenges, and Prospective Measures

In the era of the Internet of Things (IoT), the demand for accurate positioning services has become increasingly critical, as location-based services (LBSs) depend on users’ location data to deliver contextual functionalities. While the Global Positioning System (GPS) is widely regarded as the standard for outdoor localization due to its reliability and comprehensive coverage, its effectiveness in indoor positioning systems (IPSs) is limited by the inherent complexity of indoor environments. This paper examines the various measurement techniques and technological solutions that address the unique challenges posed by indoor environments. We specifically focus on three key aspects: (i) a comparative analysis of the different wireless technologies proposed for IPSs based on various methodologies, (ii) the challenges of IPSs, and (iii) forward-looking strategies for future research. In particular, we provide an in-depth evaluation of current IPSs, assessing them through multidimensional matrices that capture diverse architectural and design considerations, as well as evaluation metrics established in the literature. We further examine the challenges that impede the widespread deployment of IPSs and highlight the potential risk that these systems may not be recognized with a single, universally accepted standard method, unlike GPS for outdoor localization, which serves as the golden standard for positioning. Moreover, we outline several promising approaches that could address the existing challenges of IPSs. These include the application of transfer learning, feature engineering, data fusion, multisensory technologies, hybrid techniques, and ensemble learning methods, all of which hold the potential to significantly enhance the accuracy and reliability of IPSs. By leveraging these advanced methodologies, we aim to improve the overall performance of IPSs, thus paving the way for more robust and dependable LBSs in indoor environments.

Establishing effective learning bridge cross multi-scale feature maps for object detection and semantic segmentation

ABSTRACT In the field of image object detection and semantic segmentation, improving the accuracy of object identification and segmentation is a primary goal. To achieve this, leveraging the potential of multi-scale information through feature map refinement and fusion has been widely recognized. However, existing feature fusion methods either design more complex feature pyramid networks, replace existing detectors, or incrementally introduce feature fusion modules, overlooking the effective approach of enhancing spatial information in deep feature maps. We propose a novel pluggable feature fusion paradigm termed ‘Effective Learning Bridge’. Our research introduces an efficient and adaptive learning mechanism that builds learning bridges between feature maps at different scales within the feature pyramid, thereby enhancing the spatial information of objects in deep feature maps. This mechanism is specifically designed for multi-scale feature maps and can be seamlessly integrated into any network incorporating feature maps. By altering the model’s backpropagation path, we successfully improve learning efficiency, which in turn enhances the accuracy of object detection and segmentation. Our proposed paradigm and method were extensively evaluated through experiments on SIMD, HRSID, and WHDLD datasets and benchmark models. The results unequivocally demonstrate the effectiveness of our approach in significantly improving the accuracy of object detection and semantic segmentation, as well as the overall learning efficiency of the model.

Research progress on electronic health records multimodal data fusion based on deep learning

Currently, the development of deep learning-based multimodal learning is advancing rapidly, and is widely used in the field of artificial intelligence-generated content, such as image-text conversion and image-text generation. Electronic health records are digital information such as numbers, charts, and texts generated by medical staff using information systems in the process of medical activities. The multimodal fusion method of electronic health records based on deep learning can assist medical staff in the medical field to comprehensively analyze a large number of medical multimodal data generated in the process of diagnosis and treatment, thereby achieving accurate diagnosis and timely intervention for patients. In this article, we firstly introduce the methods and development trends of deep learning-based multimodal data fusion. Secondly, we summarize and compare the fusion of structured electronic medical records with other medical data such as images and texts, focusing on the clinical application types, sample sizes, and the fusion methods involved in the research. Through the analysis and summary of the literature, the deep learning methods for fusion of different medical modal data are as follows: first, selecting the appropriate pre-trained model according to the data modality for feature representation and post-fusion, and secondly, fusing based on the attention mechanism. Lastly, the difficulties encountered in multimodal medical data fusion and its developmental directions, including modeling methods, evaluation and application of models, are discussed. Through this review article, we expect to provide reference information for the establishment of models that can comprehensively utilize various modal medical data.

Advances in the diagnosis of prostate cancer based on image fusion

Image fusion currently plays an important role in the diagnosis of prostate cancer (PCa). Selecting and developing a good image fusion algorithm is the core task of achieving image fusion, which determines whether the fusion image obtained is of good quality and can meet the actual needs of clinical application. In recent years, it has become one of the research hotspots of medical image fusion. In order to make a comprehensive study on the methods of medical image fusion, this paper reviewed the relevant literature published at home and abroad in recent years. Image fusion technologies were classified, and image fusion algorithms were divided into traditional fusion algorithms and deep learning (DL) fusion algorithms. The principles and workflow of some algorithms were analyzed and compared, their advantages and disadvantages were summarized, and relevant medical image data sets were introduced. Finally, the future development trend of medical image fusion algorithm was prospected, and the development direction of medical image fusion technology for the diagnosis of prostate cancer and other major diseases was pointed out.

A Multi-Hierarchical Complementary Feature Interaction Network for Accelerated Multi-Modal MR Imaging

Magnetic resonance (MR) imaging is widely used in the clinical field due to its non-invasiveness, but the long scanning time is still a bottleneck for its popularization. Using the complementary information between multi-modal imaging to accelerate imaging provides a novel and effective MR fast imaging solution. However, previous technologies mostly use simple fusion methods and fail to fully utilize their potential sharable knowledge. In this study, we introduced a novel multi-hierarchical complementary feature interaction network (MHCFIN) to realize joint reconstruction of multi-modal MR images with undersampled data and thus accelerate multi-modal imaging. Firstly, multiple attention mechanisms are integrated with a dual-branch encoder–decoder network to represent shared features and complementary features of different modalities. In the decoding stage, the multi-modal feature interaction module (MMFIM) acts as a bridge between the two branches, realizing complementary knowledge transfer between different modalities through cross-level fusion. The single-modal feature fusion module (SMFFM) carries out multi-scale feature representation and optimization of the single modality, preserving better anatomical details. Extensive experiments are conducted under different sampling patterns and acceleration factors. The results show that this proposed method achieves obvious improvement compared with existing state-of-the-art reconstruction methods in both visual quality and quantity.

Research progress of breast pathology image diagnosis based on deep learning

Breast cancer is a malignancy caused by the abnormal proliferation of breast epithelial cells, predominantly affecting female patients, and it is commonly diagnosed using histopathological images. Currently, deep learning techniques have made significant breakthroughs in medical image processing, outperforming traditional detection methods in breast cancer pathology classification tasks. This paper first reviewed the advances in applying deep learning to breast pathology images, focusing on three key areas: multi-scale feature extraction, cellular feature analysis, and classification. Next, it summarized the advantages of multimodal data fusion methods for breast pathology images. Finally, the study discussed the challenges and future prospects of deep learning in breast cancer pathology image diagnosis, providing important guidance for advancing the use of deep learning in breast diagnosis.

A highly naturalistic facial expression generation method with embedded vein features based on diffusion model

Abstract Facial expression generation technology has achieved notable progress in computer vision and artificial intelligence. However, challenges persist regarding background consistency, expression clarity, and detailed representation. Additionally, the instability of generative adversarial networks (GANs) during training affects both image quality and diversity. While diffusion models have demonstrated potential advantages over GANs, research on controllable expression generation remains limited. To address these challenges, this paper proposes a highly natural facial expression generation method based on denoising diffusion implicit models (DDIM) with embedded vein features. This approach avoids adversarial training by employing gradual diffusion to generate specific expressions, thereby enhancing both the diversity and authenticity of the images. Vein features are introduced and embedded within the generated expression images to protect the intellectual property of algorithm-generated digital resources. Firstly, image and expression text guide words are combined as conditional inputs to improve the authenticity and diversity of the generated images. Secondly, a classification coding network is introduced to guide expression generation, thus enhancing the accuracy and consistency of the produced expressions. Furthermore, this paper proposes a vein feature fusion method based on multi-directional local dynamic feature coding operator (MLDFO) and integrates DDIM with frequency-domain watermarking technology to achieve image intellectual property protection. Experimental results demonstrate the effectiveness of this method across several public datasets, including FFHQ, CelebA, FV-USM, and SDUMLA-HMT. Notably, in the CelebA dataset, the average expression recognition rate increased by 11.41%, with a 100.00% recognition rate for happy expressions. The generated expression images exhibit a high degree of authenticity and consistency, and the video conversion tests reveal a natural and smooth effect. These results confirm that this method not only advances facial expression generation technology but also significantly enhances the steganographic protection of images.

High-resolution mapping of significant wave heights in the Northeast Pacific and Northwest Atlantic using improved multi-source satellite altimetry fusion method

High-resolution mapping of significant wave heights in the Northeast Pacific and Northwest Atlantic using improved multi-source satellite altimetry fusion method

An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19.

Multimodal medical image fusion methods, which combine complementary information from many multi-modality medical images, are among the most important and practical approaches in numerous clinical applications. Various conventional image fusion techniques have been developed for multimodality image fusion. Complex procedures for weight map computing, fixed fusion strategy and lack of contextual understanding remain difficult in conventional and machine learning approaches, usually resulting in artefacts that degrade the image quality. This work proposes an efficient hybrid learning model for medical image fusion using pre-trained and non-pre-trained networks i.e. VGG-19 and SNN with stacking ensemble method. The model leveraging the unique capabilities of each architecture, can effectively preserve the detailed information with high visual quality, for numerous combinations of image modalities in image fusion challenges, notably improved contrast, increased resolution, and lower artefacts. Additionally, this ensemble model can be more robust in the fusion of various combinations of source images that are publicly available from Havard-Medical-Image-Fusion Datasets, GitHub. and Kaggle. Our proposed model performance is superior in terms of visual quality and performance metrics to that of the existing fusion methods in literature like PCA+DTCWT, NSCT, DWT, DTCWT+NSCT, GADCT, CNN and VGG-19.

Interplay Between Atmospheric Correction and Fusion Techniques Enhances the Quality of Remote Sensing Image Fusion

Remote sensing image fusion technology integrates observational data from multiple satellite platforms to leverage the complementary advantages of the different types of remote sensing images. High-quality fused remote sensing images provide detailed information on surface radiation, climate, and environmental conditions, thereby supporting governmental policies on environmental changes. Improving the quality and quantitative accuracy of fused images is a crucial trend in remote sensing image fusion research. This study investigates the impact of atmospheric correction and five widely applied fusion techniques on remote sensing image fusion. By constructing four fusion frameworks, it evaluates how the choice of fusion method, the implementation of atmospheric correction, the synchronization of atmospheric parameters, and the timing of atmospheric correction influence the outcomes of remote sensing image fusion. Aerial flights using remote sensors were conducted to acquire atmospheric parameter distribution images that are strictly synchronous with the remote sensing images. Comprehensive and systematic evaluations of the fused remote sensing images were performed. Experiments show that for the remote sensing images used, selecting the appropriate fusion method can improve the spatial detail evaluation metrics of the fused images by up to 2.739 times, with the smallest deviation from true reflectance reaching 35.02%. Incorporating synchronous atmospheric parameter distribution images can enhance the spatial detail evaluation metrics by up to 2.03 times, with the smallest deviation from true reflectance reaching 5.4%. This indicates that choosing an appropriate fusion method and performing imaging-based synchronous atmospheric correction before fusion can maximize the enhancement of spatial details and spectral quantification in fused images.

MDAR: A Multiscale Features-Based Network for Remotely Measuring Human Heart Rate Utilizing Dual-Branch Architecture and Alternating Frame Shifts in Facial Videos

Remote photoplethysmography (rPPG) refers to a non-contact technique that measures heart rate through analyzing the subtle signal changes of facial blood flow captured by video sensors. It is widely used in contactless medical monitoring, remote health management, and activity monitoring, providing a more convenient and non-invasive way to monitor heart health. However, factors such as ambient light variations, facial movements, and differences in light absorption and reflection pose challenges to deep learning-based methods. To solve these difficulties, we put forward a measurement network of heart rate based on multiscale features. In this study, we designed and implemented a dual-branch signal processing framework that combines static and dynamic features, proposing a novel and efficient method for feature fusion, enhancing the robustness and reliability of the signal. Furthermore, we proposed an alternate time-shift module to enhance the model’s temporal depth. To integrate the features extracted at different scales, we utilized a multiscale feature fusion method, enabling the model to accurately capture subtle changes in blood flow. We conducted cross-validation on three public datasets: UBFC-rPPG, PURE, and MMPD. The results demonstrate that MDAR not only ensures fast inference speed but also significantly improves performance. The two main indicators, MAE and MAPE, achieved improvements of at least 30.6% and 30.2%, respectively, surpassing state-of-the-art methods. These conclusions highlight the potential advantages of MDAR for practical applications.

Pushing the boundaries for fuel discovery with a multiview features fusion approach

AbstractGlobal warming poses a serious challenge to the human environment, prompting us to rapidly develop new environmentally friendly fuels. However, the time and cost required to determine the physical properties of fuels are constrained by the related industries. In this paper, we propose a multiview features fusion method based on neural networks. This method uses the eight graph neural networks models as the basis of the multichannel network coordinator to extract the compound's molecular feature; also the functional groups in the compound are embedded with molecule weight as functional groups feature, and finally, combining the molecular view with the functional groups view for the prediction of compound flash point (FP). We used a data set consisting of 2200 hydrocarbons and oxygenated compounds for model training and testing. The results show that the model performance is improved in both after feature fusion. Finally, the ablation experiments demonstrate that the use of this method is effective and provides a new idea for fast and accurate screening of fuels. The Attentive FP‐FG model was the most effective, with a mean absolute error of 4.395 K, root mean square error of 5.854 K, and R‐squared score (R2) of 0.986.