Fusion Approach Research Articles

Collaborative Learning With Heterogeneous Local Models: A Rule-Based Knowledge Fusion Approach

Collaborative Learning With Heterogeneous Local Models: A Rule-Based Knowledge Fusion Approach

Selective Identification of Hazardous Gases Using Flexible, Room-Temperature Operable Sensor Array Based on Reduced Graphene Oxide and Metal Oxide Nanoparticles via Machine Learning.

Selective detection and monitoring of hazardous gases with similar properties are highly desirable to ensure human safety. The development of flexible and room-temperature (RT) operable chemiresistive gas sensors provides an excellent opportunity to create wearable devices for detecting hazardous gases surrounding us. However, chemiresistive gas sensors typically suffer from poor selectivity and zero-cross selectivity toward similar types of gases. Herein, a flexible, RT operable chemiresistive gas sensors array is designed, featuring reduced graphene oxide (rGO) and rGO decorated with zinc oxide (ZnO), titanium dioxide (TiO2), and tin dioxide (SnO2) nanoparticles (NPs) on a flexible polyimide (PI) substrate. The sensor array consists of four different sensing layers capable of the selective identification of various hazardous gases such as NO2, NO, and SO2 using machine learning (ML). The gas sensor array exhibits a stable response even when mechanically deformed or exposed to high humidity (up to 60%). Each gas sensor, due to the different metal oxide NPs, shows unique responses in terms of sensitivity, responsiveness, response time, and recovery time to different gases. Consequently, the sensor array generates distinct response patterns that effectively differentiate between the target gases. By leveraging these distinctive recovery patterns and employing a data fusion approach in ML, specific concentrations of target gases can be distinguished. Using ML with fused array sensing data, the training and test accuracies achieved were 98.20 and 97.70%, respectively. This innovative combination of sensor arrays and ML offers significant potential for selective gas detection in environmental monitoring and personal safety applications.

Characterizing the spatial variability of marine soil properties with site-specific sparse data using a Bayesian data fusion approach

Characterizing the spatial variability of marine soil properties with site-specific sparse data using a Bayesian data fusion approach

Real-time localization with probabilistic maps and unscented kalman filtering: a dynamic sensor fusion approach

This paper presents a robust position estimation technique for autonomous vehicles navigating urban and highway environments. It employs a probabilistic framework, integrating Monte Carlo simulation and Bayesian filtering via particle filters to represent position estimates beyond Gaussian distributions. To improve accuracy, Unscented Kalman Filtering (UKF) is used to fuse radar and lidar data, facilitating the detection of static, pole-like objects. These objects are matched with landmarks from a detailed reference map generated through 3D lidar scans. The proposed method addresses both lateral and longitudinal localisation challenges, ensuring precise vehicle positioning. Comprehensive simulation tests validate the system's effectiveness, demonstrating reliable performance across various driving conditions.

Coordinating public and government responses to air pollution exposure: A multi-source data fusion approach

Aligning public demand with government supply of clean air aids in efficient air pollution control and enhancement of public happiness. However, comparative empirical analyses of public and government attention to air quality changes are still sparse due to data and methodological constraints. Here, we adopt multi-source data fusion approaches to assess the impacts of air pollution exposure on public and government attention. Specifically, remote and social sensing data, alongside keywords extracted from textual data, are utilized to quantify air pollution exposure and corresponding public and government attention levels in 273 Chinese cities from 2011 to 2019, and a two-stage least squares regression model is employed to tackle reverse causality issues underlying the exposure-response relationship. Our findings reveal that, on average, a unit increase in PM2.5 levels would result in a 17.7% growth in public attention and a 12.7% rise in government attention, respectively, suggesting that demand-driven public attention tends to be more sensitive to air quality changes than policy-driven government attention. Results for the spatial-temporal heterogeneity further demonstrate that public attention varies across time and space, whereas government attention remains relatively consistent. Additionally, we have identified 116 cities exhibiting disparities between the public and government responses to air quality changes, calling for environmental policy refinements to better serve the needs of residents. This study emphasizes the necessity of public engagement in environmental governance and offers rich policy implications for air pollution control in China.

Using data fusion with multiple imputation to correct for misclassification in self-reported exposure: a case-control study of cannabis use and homicide victimization

BackgroundCannabis use has been causally linked to violent behaviors in experimental and case studies, but its association with homicide victimization has not been rigorously assessed through epidemiologic research.MethodsWe performed a case-control analysis using two national data systems. Cases were homicide victims from the National Violent Death Reporting System (NVDRS), and controls were participants from the National Survey on Drug Use and Health (NSDUH). While the NVDRS contained toxicological testing data on cannabis use, the NSDUH only collected self-reported data, and thus the potential misclassification in the self-reported data needed to be corrected. We took a data fusion approach by concatenating the NSDUH with a third data system, the National Roadside Survey of Alcohol and Drug Use by Drivers (NRS), which collected toxicological testing and self-reported data on cannabis use for drivers. The data fusion approach provided multiple imputations (MIs) of toxicological testing results on cannabis use for the participants in the NSDUH, which were then used in the case-control analysis. Bootstrap was used to obtain valid statistical inference.ResultsThe analyses revealed that cannabis use was associated with 3.55-fold (95% CI: 2.75–4.35) increased odds of homicide victimization. Alcohol use, being Black, male, aged 21–34 years, and having less than a high school education were also significantly associated with increased odds of homicide victimization.ConclusionsCannabis use is a major risk factor for homicide victimization. The data fusion with MI method is useful in integrative data analysis for harmonizing measures between different data sources.

Development of the numerical differentiation method for approximating pitch acceleration using sensor fusion approach

In this paper, a novel algorithm is proposed to accurately estimate pitch acceleration that is crucial for moment coefficient estimation of the mathematical model of aircraft and control design in the presence of measurement noise. The angular velocity of the body as well as the Euler angles provided by the navigation system are used to interpolate the attitude trajectories using an algorithm based on the Hermite-spline polynomial. By differentiating the resultant trajectory function, the angular acceleration can be estimated accurately. This paper also analyzes a well-known method-Poplavski method based on polynomial regression, developed by the Russian scientist B.K. Poplavski to estimate derivatives. The simulation results obtained from the novel algorithm are compared with those obtained using the Poplavski method. The results verified that the novel algorithm that uses both pitch angle and angular velocity provides better accuracy in estimating pitch acceleration than the Poplavski method does, regardless of the sampling rate, which is very important in numerical differentiation and the noise level.

Fire Recognition Method Based on PSO-BP Neural Network and ResNet50

The rapid development of modern society and continuous urbanization have resulted in a proliferation of functional buildings, which offer significant convenience to individuals, but pose significant fire hazards as well. How to detect the fire at the early stage is always the focus of research. This paper proposes a multi-information source fusion fire recognition method based on particle swarm optimization (PSO)-backpropagation (BP) neural networks and ResNet50. The PSO algorithm is applied to optimize the initial parameters of a BP neural network model, while data from three sensors — temperature, humidity and smoke — are integrated, through iterative training of the system, accurate recognition of sensor data can be achieved. Additionally, a method is proposed for the recognition of infrared fire images using ResNet50 and transfer learning. By improving the ResNet50 network model and migrating the ResNet50 pre-trained network weight, infrared fire image recognition accuracy is further enhanced. Then the sensor information recognition results and image information recognition results are input into the fuzzy system for fusion reasoning again, and the final decision is output according to the set fuzzy rules. Experimental findings demonstrate that the multi-information source fusion approach utilizing the PSO-BP neural network and ResNet50 significantly enhances the accuracy and response time of fire recognition, and achieves a remarkable recognition effect.

Pushing the boundaries for fuel discovery with a multiview features fusion approach

AbstractGlobal warming poses a serious challenge to the human environment, prompting us to rapidly develop new environmentally friendly fuels. However, the time and cost required to determine the physical properties of fuels are constrained by the related industries. In this paper, we propose a multiview features fusion method based on neural networks. This method uses the eight graph neural networks models as the basis of the multichannel network coordinator to extract the compound's molecular feature; also the functional groups in the compound are embedded with molecule weight as functional groups feature, and finally, combining the molecular view with the functional groups view for the prediction of compound flash point (FP). We used a data set consisting of 2200 hydrocarbons and oxygenated compounds for model training and testing. The results show that the model performance is improved in both after feature fusion. Finally, the ablation experiments demonstrate that the use of this method is effective and provides a new idea for fast and accurate screening of fuels. The Attentive FP‐FG model was the most effective, with a mean absolute error of 4.395 K, root mean square error of 5.854 K, and R‐squared score (R2) of 0.986.

Distributed Multi-Sensor Fusion for Multi-Group/Extended Target Tracking with Different Limited Fields of View

The identification of sensing group targets and extended targets is of paramount importance in the context of vehicle tracking and early warning detection. As the scope of target monitoring and tracking extends, conventional single-sensor-based tracking techniques are proving to be inadequate in meeting the practical demands of the field. Consequently, multi-sensor fusion tracking technology has emerged as a viable alternative. However, the use of multiple sensors is constrained by their limited fields of view (FOVs), which leads to issues such as the loss of target detection and the introduction of false targets after fusion. Hence, this study proposes a combination of weighted geometric averaging (WGA) and weighted arithmetic averaging (WAA) methods to solve distributed multi-group/extended target tracking with different fields of view. Specifically, a local Poisson multi-Bernoulli mixture (PMBM) filter was first used on individual sensors. Subsequently, we combined a sequential fusion technique and the proposed fusion approach to fuse the PMBM filter densities. The efficacy and superiority of this approach were demonstrated through simulations.

KOALA: A Modular Dual-Arm Robot for Automated Precision Pruning Equipped with Cross-Functionality Sensor Fusion

Landscape maintenance is essential for ensuring agricultural productivity, promoting sustainable land use, and preserving soil and ecosystem health. Pruning is a labor-intensive task among landscaping applications that often involves repetitive pruning operations. To address these limitations, this paper presents the development of a dual-arm holonomic robot (called the KOALA robot) for precision plant pruning. The robot utilizes a cross-functionality sensor fusion approach, combining light detection and ranging (LiDAR) sensor and depth camera data for plant recognition and isolating the data points that require pruning. The You Only Look Once v8 (YOLOv8) object detection model powers the plant detection algorithm, achieving a 98.5% pruning plant detection rate and a 95% pruning accuracy using camera, depth sensor, and LiDAR data. The fused data allows the robot to identify the target boxwood plants, assess the density of the pruning area, and optimize the pruning path. The robot operates at a pruning speed of 10–50 cm/s and has a maximum robot travel speed of 0.5 m/s, with the ability to perform up to 4 h of pruning. The robot’s base can lift 400 kg, ensuring stability and versatility for multiple applications. The findings demonstrate the robot’s potential to significantly enhance efficiency, reduce labor requirements, and improve landscape maintenance precision compared to those of traditional manual methods. This paves the way for further advancements in automating repetitive tasks within landscaping applications.

Learning infrared degradations for coherent visible image fusion in the undecimated dual-tree complex wavelet domain

Traditional infrared (IR) and visible (VIS) image fusion methods demand identical resolution levels for source images, which can be problematic due to the inherent low-resolution nature of IR imagery. In this paper, we introduce an innovative image fusion approach that harmonizes resolution across IR-VIS source images, leading to the generation of fused images with higher resolution. We employ a convolutional neural network model to recover the real-time image degradations in IR data, with a particular focus on super-resolution through the multi degradation resolution enhancement network (MDREN). We adopt undecimated dual-tree complex wavelet transform (UDT-CWT) in our fusion process due to its near shift invariance and better directionality capabilities. This results in coherent information of the fused images with minimized noise and loss. Experiments employing five image quality assessment measures are used to compare the proposed method to nine state-of-the-art approaches and show its efficacy.

Impact of metadata in multimodal classification of bone tumours

The accurate classification of bone tumours is crucial for guiding clinical decisions regarding treatment and follow-up. However, differentiating between various tumour types is challenging due to the rarity of certain entities, high intra-class variability, and limited training data in clinical practice. This study proposes a multimodal deep learning model that integrates clinical metadata and X-ray imaging to improve the classification of primary bone tumours. The dataset comprises 1,785 radiographs from 804 patients collected between 2000 and 2020, including metadata such as age, affected bone site, tumour position, and gender. Ten tumour types were selected, with histopathology or tumour board decisions serving as the reference standard.MethodsOur model is based on the NesT image classification model and a multilayer perceptron with a joint fusion architecture. Descriptive statistics included incidence and percentage ratios for discrete parameters, and mean, standard deviation, median, and interquartile range for continuous parameters.ResultsThe mean age of the patients was 33.62 ± 18.60 years, with 54.73% being male. Our multimodal deep learning model achieved 69.7% accuracy in classifying primary bone tumours, outperforming the Vision Transformer model by five percentage points. SHAP values indicated that age had the most substantial influence among the considered metadata.ConclusionThe joint fusion approach developed in this study, integrating clinical metadata and imaging data, outperformed state-of-the-art models in classifying primary bone tumours. The use of SHAP values provided insights into the impact of different metadata on the model’s performance, highlighting the significant role of age. This approach has potential implications for improving diagnostic accuracy and understanding the influence of clinical factors in tumour classification.

MFB: A Generalized Multimodal Fusion Approach for Bitcoin Price Prediction Using Time-Lagged Sentiment and Indicator Features

Bitcoin’s volatile nature has made its price prediction a sought-after mathematical model in the FinTech industry. Existing studies, however, need to look into the critical aspect of time-lagged sentiment in Bitcoin price forecasting. This omission is significant because time-lagged sentiment captures delayed market reactions that are not immediately apparent in price movements. Moreover, the correlation between time-lagged sentiment and technical indicators and the limitations of individual machine learning and deep learning models necessitates a comprehensive approach for accurate and reliable Bitcoin price predictions. This paper introduces the multimodal fusion Bitcoin (MFB), an innovative generalized multimodal fusion approach that effectively integrates BiLSTM and BiGRU layers for complex feature extraction. The model employs the BorutaShap algorithm for feature selection and utilizes attention mechanisms and spatial dropout for optimization and generalization. MFB’s training and validation use news and tweet data combined with Bitcoin technical indicators to explore the impact of time-lagged sentiment on price movements, leading to more accurate and timely market predictions. The MFB performs superior Bitcoin prediction performance, achieving 97.63% accuracy and an MAE of 0.0065. Experiments highlight MFB’s capability to outperform existing models, offering significant insights for investors in making informed decisions. MFB’s innovative methodology, particularly in next-hour Bitcoin price forecasting, marks an advancement in financial forecasting. By capturing the nuanced dynamics of market sentiment and its delayed effects, MFB is a pioneering multimodal fusion approach in the FinTech domain, revolutionizing Bitcoin price prediction.

A novel anti-CTLA-4 nanobody-IL12 fusion protein in combination with a dendritic cell/tumour fusion cell vaccine enhances the antitumour activity of CD8+ T cells in solid tumours.

We previously developed a nanobody targeting CTLA-4 and demonstrated that it can boost antitumour T-cell responses in vitro; however, the resulting responses after the injection of T cells into cancer models are usually weak and transient. Here, we explored whether fusing our nanobody to IL-12 would enable it to induce stronger, longer-lasting T-cell immune responses after exposure to immature dendritic cell and tumour cell fusions. The fusion protein enhanced the response of CD8+ T cells to tumour antigens in vitro and led to stronger, more persistent immune responses after the T cells were injected into mice bearing different types of xenografts. Our in vitro and in vivo results suggest the anticancer potential of our nanobody-interleukin fusion system and support the clinical application of this fusion approach for various nanobodies.

Brain Tumor Detection through Image Fusion Using Cross Guided Filter and Convolutional Neural Network

This Data fusion has become a significant issue in diagnostic imaging, particularly in medical applications like radiation and guided image surgery. Medical image fusion aims to enhance the precision of tumor diagnosis, by preserving the salient information and characteristics of the original images in the fused image. It has been shown that guided filters are capable of maintaining edges well. In this paper, we propose a novel cross-guided filter-based fusion approach for multimodal medical images utilizing convolutional neural networks. The cross-guided filter is used in the proposed algorithm to extract the detailed features from the source images. Convolutional neural networks are used to generate the feature weights of source images derived from the detail layers. The weighted average rule is used to merge the source images based on these weights. We used thirty distinct types of medical images from diverse sources to compare the effectiveness of the proposed strategy to that of existing methods, both numerically and visually. The experimental findings demonstrated that, in terms of both objective evaluation and qualitative image quality, the suggested system performs better than other standard methods already in use. The quantitative results show that compared to existing methods under consideration for comparison, the proposed algorithm improves mutual information by 25%, image entropy by 9.5%, spatial frequency by 21%, standard deviation by 18.1%, structural similarity index by 30%, and edge strength of the fused image by 39%.

Photomanipulatable colloidal clusters from the aggregation of azo molecular glass spheres.

Colloidal clusters with well-controlled shapes have attracted extensive interest in the fields of materials, chemistry, physics, and biology. This communication reports the controllable fabrication of photoresponsive colloidal clusters with a wide range of adjustable sizes and complex architectures through an approach of microsphere formation and fusion. The clusters of colloidal spheres were obtained via adding ethanol dropwise into a tetrahydrofuran solution of an isosorbide-based azo compound (IAC-4). In the process, the colloidal spheres with soft and sticky shells were first formed in the dispersion. After stirring at an appropriate rate and time, clusters composed of controlled numbers of colloidal spheres were obtained. With increasing stirring time, the colloidal spheres in the clusters underwent fusion transforming into a range of structures with particular architectures. The structure formation, evolution and control were investigated by scanning electron microscopy (SEM) and dynamic light scattering (DLS). Under linearly polarized light irradiation, colloidal spheres in the clusters in the solid state were observed to be stretched along the direction of electric-field oscillation and these clusters were thus transformed into complex particles with unique morphologies. This exploration can lead to a new methodology to effectively fabricate colloidal clusters with complex architectures and shed new light on colloidal packing and organization under the driving forces of extrinsic energy input.

Sibling Discrimination Using Linear Fusion on Deep Learning Face Recognition Models

Facial recognition technology has revolutionised human identification, providing a non-invasive alternative to traditional biometric methods like signatures and voice recognition. The integration of deep learning has significantly enhanced the accuracy and adaptability of these systems, now widely used in criminal identification, access control, and security. Initial research focused on recognising full-frontal facial features, but recent advancements have tackled the challenge of identifying partially visible faces, a scenario that often reduces recognition accuracy. This study aims to identify siblings based on facial features, particularly in cases where only partial features like eyes, nose, or mouth are visible. Utilising advanced deep learning models such as VGG19, VGG16, VGGFace, and FaceNet, the research introduces a framework to differentiate between sibling images effectively. To boost discrimination accuracy, the framework employs a linear fusion technique that merges insights from all the models used. The methodology involves preprocessing image pairs, extracting embeddings with pre-trained models, and integrating information through linear fusion. Evaluation metrics, including confusion matrix analysis, assess the framework's robustness and precision. Custom datasets of cropped sibling facial areas form the experimental basis, testing the models under various conditions like different facial poses and cropped regions. Model selection emphasises accuracy and extensive training on large datasets to ensure reliable performance in distinguishing subtle facial differences. Experimental results show that combining multiple models' outputs using linear fusion improves the accuracy and realism of sibling discrimination based on facial features. Findings indicate a minimum accuracy of 96% across different facial regions. Although this is slightly lower than the accuracy achieved by a single model like VGG16 with full-frontal poses, the fusion approach provides a more realistic outcome by incorporating insights from all four models. This underscores the potential of advanced deep learning techniques in enhancing facial recognition systems for practical applications.

An Adaptive SCG-ECG Multimodal Gating Framework for Cardiac CTA.

Cardiovascular disease (CVD) is the leading cause of death worldwide. Coronary artery disease (CAD), a prevalent form of CVD, is typically assessed using catheter coronary angiography (CCA), an invasive, costly procedure with associated risks. While cardiac computed tomography angiography (CTA) presents a less invasive alternative, it suffers from limited temporal resolution, often resulting in motion artifacts that degrade diagnostic quality. Traditional ECG-based gating methods for CTA inadequately capture cardiac mechanical motion. To address this, we propose a novel multimodal approach that enhances CTA imaging by predicting cardiac quiescent periods using seismocardiogram (SCG) and ECG data, integrated through a weighted fusion (WF) approach and artificial neural networks (ANNs). We developed a regression-based ANN framework (r-ANN WF) designed to improve prediction accuracy and reduce computational complexity, which was compared with a classification-based framework (c-ANN WF), ECG gating, and US data. Our results demonstrate that the r-ANN WF approach improved overall diastolic and systolic cardiac quiescence prediction accuracy by 52.6% compared to ECG-based predictions, using ultrasound (US) as the ground truth, with an average prediction time of 4.83 ms. Comparative evaluations based on reconstructed CTA images show that both r-ANN WF and c-ANN WF offer diagnostic quality comparable to US-based gating, underscoring their clinical potential. Additionally, the lower computational complexity of r-ANN WF makes it suitable for real-time applications. This approach could enhance CTA's diagnostic quality, offering a more accurate and efficient method for CVD diagnosis and management.

Trans-Pars Interarticularis Approach for Lumbar Interbody Fusion: An Efficient, Straightforward, and Minimally Invasive Surgery for Lumbar Spondylolisthesis and Stenosis.

Lumbar interbody fusion is a commonly applied surgical treatment for spondylolisthesis. For this procedure, various minimally invasive (MIS) approaches have been developed, including posterior lumbar interbody fusion, transforaminal lumbar interbody fusion (TLIF), oblique lumbar interbody fusion, and anterior lumbar interbody fusion. In this study, we characterized the features of an MIS trans-pars interarticularis lumbar interbody fusion (TPLIF) and compared its surgical outcomes with those of MIS-TLIF. This study included 89 and 44 patients who had undergone MIS-TPLIF and MIS-TLIF, respectively, between September 2016 and December 2022. The following clinical outcomes were analyzed: operative time, blood loss, and hospitalization duration. The average operative time, blood loss, and hospitalization duration for the MIS-TPLIF and MIS-TLIF groups were, respectively, 98.28 and 191.15 minutes, 41.97 and 101.85 mL, and 5.8 and 6.9 days. The MIS-TPLIF approach for lumbar spondylolisthesis or other degenerative diseases involves the use of the commonly available and cost-effective instrument Taylor retractor, thus enabling posterior lumbar interbody fusion to be performed with minimal invasion. This approach also confers the benefits of a short learning curve and an intuitive approach. Our results suggest that although MIS-TPLIF is noninferior to MIS-TLIF, it is easier to learn and perform than MIS-TLIF.