Data Fusion Techniques Research Articles

Tracking health-related quality of life dynamics: Advances in ambulatory assessment methods

Assessing quality of life in clinical practice and research via patient-reported outcomes is an emerging practice enabling the assessment of individual well-being across different populations and conditions, such as immune-mediated and inflammatory disorders. Standardized questionnaires have traditionally been used for this purpose, but they have several limitations inherent to one-time snapshots from recall-based self-reports. Ambulatory measures may provide an alternative approach to assessing quality of life that allows capturing an individual’s states and behaviours in the context of daily life. In this overview article, we define ambulatory measures, including ecological momentary self-reports, wearable sensors, biochemical sampling, and smartphone sensing, that may be used to estimate quality of life, and present examples of studies that have used these measures. Additionally, we highlight the benefits of integrating multi-method ambulatory measures and overview methods for combining and analyzing them, including data fusion techniques and machine learning. While there are still challenges and limitations to overcome, the integration and application of ambulatory measures in medical research and clinical practice have the potential to provide a more comprehensive and accurate assessment of the quality of (everyday) life.

Enhanced HAR using Dynamic STGAT

Action recognition has seen significant advancements with the integration of spatio-temporal representations, particularly leveraging skeleton-based models and cross-modal data fusion techniques. However, existing approaches face challenges in capturing long- range dependencies within the human body skeleton and effectively balancing features from diverse modalities. To address these limitations, a novel framework, the Dynamic Spatio-Temporal Graph Attention Transformer (D-STGAT), is proposed, which seamlessly integrates the strengths of dynamic graph attention mechanisms and transformer architectures for enhanced action recognition. The framework builds upon recent innovations in graph attention networks (GAT) and transformer models. First, the Spatial-Temporal Dynamic Graph Attention Network (ST-DGAT) is introduced, extending traditional GAT by incorporating a dynamic attention mechanism to capture spatial- temporal patterns within skeleton sequences. By reordering the weighted vector operations in GAT, the approach achieves a global approximate attention function, significantly enhancing its expressivity and capturing long-distance dependencies more effectively than static attention mechanisms. Furthermore, to address the challenges of cross-modal feature representation and fusion, the spatio-temporal Cross Attention Transformer (ST-CAT) is introduced. This model efficiently integrates spatio-temporal information from both video frames and skeleton sequences by employing a combination of full spatio-temporal attention (FAttn), zigzag spatio-temporal attention (ZAttn), and binary spatio-temporal attention (BAttn) modules. Through the proper arrangement of these modules within the transformer encoder and decoder, ST-CAT learns a multi-feature representation that effectively captures the intricate spatiotemporal dynamics inherent in action recognition tasks. Experimental results on the Penn- Action, NTU-RGB+D 60, and 120 datasets showcase the efficacy of the approach, yielding promising performance improvements over previous state-of-the-art methods. In summary, the proposed D-STGAT and ST-CAT frameworks offer novel solutions for action recognition tasks by leveraging dynamic graph attention mechanisms and transformer architectures to effectively capture and fuse spatiotemporal features from diverse modalities, leading to superior performance compared to existing approaches.

Machine Learning in Petrology: State-of-the-Art and Future Perspectives

Abstract This article reports on the state-of-the-art and future perspectives of machine learning (ML) in petrology. To achieve this goal, it first introduces the basics of ML, including definitions, core concepts, and applications. Then, it starts reviewing the state-of-the-art of ML in petrology. Established applications mainly concern the so-called data-driven discovery and involve specific tasks like clustering, dimensionality reduction, classification, and regression. Among them, clustering and dimensionality reduction have been demonstrated to be valuable for decoding the chemical record stored in igneous and metamorphic phases and to enhance data visualization, respectively. Classification and regression tasks find applications, for example, in petrotectonic discrimination and geo-thermobarometry, respectively. The main core of the manuscript consists of depicting emerging trends and the future directions of ML in petrological investigations. I propose a future scenario where ML methods will progressively integrate and support established petrological methods in automating time-consuming and repetitive tasks, improving current models, and boosting discovery. In this framework, promising applications include (1) the acquisition of new multimodal petrologic data; (2) the development of data fusion techniques, physics-informed ML models, and ML-supported numerical simulations; and (3) the continuous exploration of the ML potential in petrology. To boost the contribution of ML in petrology, our main challenges are: (1) to improve the ability of ML models to capture the complexity of petrologic processes, (2) progressively link ML algorithms with the physical and thermodynamic nature of the investigated problems, and (3) to start a collaborative effort among researchers coming from different disciplines, both in research and teaching.

Research progress in water quality prediction based on deep learning technology: a review.

Water, an invaluable and non-renewable resource, plays an indispensable role in human survival and societal development. Accurate forecasting of water quality involves early identification of future pollutant concentrations and water quality indices, enabling evidence-based decision-making and targeted environmental interventions. The emergence of advanced computational technologies, particularly deep learning, has garnered considerable interest among researchers for applications in water quality prediction because of its robust data analytics capabilities. This article comprehensively reviews the deployment of deep learning methodologies in water quality forecasting, encompassing single-model and mixed-model approaches. Additionally, we delineate optimization strategies, data fusion techniques, and other factors influencing the efficacy of deep learning-based water quality prediction models, because understanding and mastering these factors are crucial for accurate water quality prediction. Although challenges such as data scarcity, long-term prediction accuracy, and limited deployments of large-scale models persist, future research aims to address these limitations by refining prediction algorithms, leveraging high-dimensional datasets, evaluating model performance, and broadening large-scale model application. These efforts contribute to precise water resource management and environmental conservation.

Data Fusion of Wing Pressure Distributions Using Scalable Gaussian Processes

Aircraft design requires a large volume of aerodynamic data to characterize various flight conditions throughout the aircraft’s flight envelope, and the data are typically obtained through wind tunnel testing and numerical analysis. Data acquisition can be costly and inevitably entails multiple sources of uncertainty. Data fusion techniques aim to bring together the strengths and mitigate the limitations of data from various information sources. A multifidelity data fusion framework employing Gaussian process regression is adopted herein and applied to the surface pressure data of a large aircraft wing model. The modeling approach is non-hierarchical in that there is no established hierarchy of accuracy among the information sources. Scalability issues arising from the large volume of data required for the study of pressure distributions are overcome by using an approximate Gaussian process regression based on stochastic variational inference, enabling the data fusion framework to be applied effectively to industry-relevant analysis and design challenges. The approach is demonstrated for a high-dimensional data set generated through wind tunnel testing in an industrial setting and numerical analysis.

Assessing the 2023 Canadian wildfire smoke impact in Northeastern US: Air quality, exposure and environmental justice

The Canadian wildfires in June 2023 significantly impacted the northeastern United States, particularly in terms of worsened air pollution and environmental justice concerns. While advancements have been made in low-cost sensor deployments and satellite observations of atmospheric composition, integrating dynamic human mobility with wildfire PM2.5 exposure to fully understand the environmental justice implications remains underinvestigated. This study aims to enhance the accuracy of estimating ground-level fine particulate matter (PM2.5) concentrations by fusing chemical transport model outputs with empirical observations, estimating exposures using human mobility data, and evaluating the impact of environmental justice. Employing a novel data fusion technique, the study combines the Weather Research and Forecasting model with Chemistry (WRF-Chem) outputs and surface PM2.5 measurements, providing a more accurate estimation of PM2.5 distribution. The study addresses the gap in traditional exposure assessments by incorporating human mobility data and further investigates the spatial correlation of PM2.5 levels with various environmental and demographic factors from the US Environmental Protection Agency (EPA) Environmental Justice Screening and Mapping Tool (EJScreen). Results reveal that despite reduced mobility during high PM2.5 levels from wildfire smoke, exposure for both residents and individuals on the move remains high. Regions already burdened with high environmental pollution levels face amplified PM2.5 effects from wildfire smoke. Furthermore, we observed mixed correlations between PM2.5 concentrations and various demographic and socioeconomic factors, indicating complex exposure patterns across communities. Urban areas, in particular, experience persistent high exposure, while significant correlations in rural areas with EJScreen factors highlight the unique vulnerabilities of these populations to smoke exposure. These results advocate for a comprehensive approach to environmental health that leverages advanced models, integrates human mobility data, and addresses socio-demographic disparities, contributing to the development of equitable strategies against the growing threat of wildfires.

Optimizing Parkinson's Disease Diagnosis with Multimodal Data Fusion Techniques

Parkinson's disease (PD) is a central nervous system neurodegenerative illness. Its symptoms include poor motor skills, speech, cognition, and memory. The condition is incurable, although evidence shows that early identification and therapy reduce symptoms. A lack of medical facilities and personnel hinders PD identification. PD is a common chronic degenerative neurological dyskinesia that threatens the elderly. Multi-modal data fusion may reveal more about PD pathophysiology. This study aims to contribute to the evaluation of PD by introducing a novel multimodal deep-learning technique for distinguishing individuals with PD from those without PD. This study utilizes resting functional magnetic resonance imaging (rfMRI) and gene data obtained from the Parkinson's Progression Markers Initiative (PPMI) and Alzheimer's Disease Neuroimaging Initiative (ADNI) databases. The primary objective is to predict the specific pathological brain regions and identify the risk genes associated with PD. The authors want to learn more about the genetic components and underlying procedures by analyzing these datasets. Contributing to the development and progression of PD. In this study, we present our findings that demonstrate the superior recital of our proposed multimodal method compared to both unimodal approaches and other existing multimodal methods. Our evaluation is based on an extensive dataset consisting of real patients. Specifically, our proposed method stacked deep learning classifiers (SDLC) achieves an impressive F1-score of 0.99 and an accuracy of 99.4%, surpassing the performance of both unimodal approaches and other multimodal methods. These results highlight the efficiency and potential of our method in enhancing the accuracy and reliability of patient data analysis. In this study, we demonstrate that our proposed method consistently surpasses alternative approaches in terms of performance, as indicated by a higher average increase in F1-score. This finding highlights the advantage of training on multiple modalities, even when a particular modality is absent during inference.

Human and Small Animal Detection Using Multiple Millimeter-Wave Radars and Data Fusion: Enabling Safe Applications.

Millimeter-wave (mmWave) radars attain high resolution without compromising privacy while being unaffected by environmental factors such as rain, dust, and fog. This study explores the challenges of using mmWave radars for the simultaneous detection of people and small animals, a critical concern in applications like indoor wireless energy transfer systems. This work proposes innovative methodologies for enhancing detection accuracy and overcoming the inherent difficulties posed by differences in target size and volume. In particular, we explore two distinct positioning scenarios that involve up to four mmWave radars in an indoor environment to detect and track both humans and small animals. We compare the outcomes achieved through the implementation of three distinct data-fusion methods. It was shown that using a single radar without the application of a tracking algorithm resulted in a sensitivity of 46.1%. However, this sensitivity significantly increased to 97.10% upon utilizing four radars using with the optimal fusion method and tracking. This improvement highlights the effectiveness of employing multiple radars together with data fusion techniques, significantly enhancing sensitivity and reliability in target detection.

Development of a novel machine learning-based approach for brain function assessment and integrated software solution

Industry 5.0 emphasizes human-centered solutions. However, bridging the gap between human conditions and engineering systems remains a challenge in this era. Promoting the development of cybernetics applications in the field of rehabilitation is imperative. In the healthcare sector, the evaluation of rehabilitation products’ ergonomics and the dynamic assessment of the human body are of utmost importance. Amid the burgeoning prominence of data-driven product design concepts, ergonomics methods have emerged as indispensable evaluative tools. Nevertheless, the integration of data from multiple sources remains a contemporary challenge; one must extract meaningful information from diverse data sets while enhancing data analysis and fusion techniques. In this study, we collect multimodal neurobiological signals using electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) to devise a robust multimodal rehabilitation data fusion method based on graph convolutional network (GCN), particularly focusing on the motor imagery (MI) task. To achieve a more holistic evaluation, we incorporate granger causality (GC) and brain region adjacency matrix as supplementary features, integrating electrophysiological and hemodynamic perspectives. This integration aims to enhance information complementarity and foster a comprehensive understanding of the neural processes involved in MI. Our results demonstrate the superiority of the multimodal fusion method: it achieves higher average accuracy and improved stability, as indicated by the reduced standard deviation of accuracy. This enhanced performance suggests its potential regarding broader research applications. Building upon our successful findings, we establish an intelligent algorithmic rehabilitation platform based on multimodal neural data. This platform not only holds significant clinical value, facilitating precision and personalized medicine, but also propels the advancement of medical practices, paving the way for a more tailored and effective approach to patient care. Our research makes progresses in the areas of rehabilitation modeling, equipment design, intelligence, and interdisciplinary collaboration through the in-depth study and extension of cybernetic principles and techniques.

ENHANCING TRAFFIC SAFETY: A COMPREHENSIVE APPROACH THROUGH REAL-TIME DATA AND INTELLIGENT TRANSPORTATION SYSTEMS

Today, traffic accidents are still a difficult and urgent problem for many countries around the world. Traffic accidents on highways are often more serious than accidents on urban roads. Therefore, disseminating emergency information and creating immediate connections with road users is key to rescuing passengers and reducing congestion. Thus, this study applies data fusion and data mining techniques to analyze travel time and valuable information about traffic accidents based on the real-time data collected from On-Board Unit installed in vehicles. The results show that this important information is the vital database to analyze traffic conditions and safety factors, thereby developing a smart traffic information platform. This result enables traffic managers to provide real-time traffic information or forecasts of congestion and traffic accidents to road users. This helps limit congestion and serious accidents on the Highway.

A fuzzy track-to-track association algorithm with dynamic time warping for trajectory-level vehicle detection

Multi-source track-to-track association (TTTA), which identifies trajectories from multiple sensors or data sources of the same dynamic vehicle, is an important data fusion technique widely applied to vehicle detection in the fields of road, marine, and aviation transportation. However, issues such as time asynchrony, heterogeneous sampling intervals, and random sensing errors have posed considerable challenges to the accuracy and robustness of TTTA. Aiming to address these issues in an integrated manner, this paper proposes a TTTA algorithm that comprehensively calculates the similarity between trajectories using multiple trajectory features through dynamic time warping and Cauchy distribution degree of membership function. Multiple experimental datasets were generated by randomly sampling real AIS trajectory data into two trajectory data sources and adding random errors. The average association accuracy of all scenarios and error levels of the proposed method reached 97.33%, far higher than other benchmark methods. Experimental results demonstrated the advantage of the proposed algorithm in various TTTA scenarios, especially its robustness in intricate trajectory situations. The results also indicated that more features can maintain the stability of associations in the presence of larger random errors, and dynamic time warping can improve association accuracy in intricate scenarios. This study provides a practical solution for the problem of time asynchrony, heterogeneous sampling intervals, and random errors in multi-source trajectory data fusion, showcasing promising applications across diverse domains.

CM-GAN: A Cross-Modal Generative Adversarial Network for Imputing Completely Missing Data in Digital Industry.

Multimodal data fusion analysis is essential to model the uncertainty of environment awareness in digital industry. However, due to communication failure and cyberattack, the sampled time-series data often have the issue of data missing. In some extreme cases, part of units are unobservable for a long time, which results in complete data missing (CDM). To impute missing data, many models have been proposed. However, they cannot address the CDM issue, because no observation data of the unobservable units can be obtained in this case. Thus, to address the CDM issue, a novel cross-modal generative adversarial network (CM-GAN) is proposed in this article. It combines the cross-modal data fusion technique and the deep adversarial generation technique to construct a cross-modal data generator. This generator can generate long-term time-series data from widely existing spatio-temporal modal data in modern industrial system, and then impute missing value by replacing them with generated data. To test the performance of CM-GAN, extensive experiments are conducted on photovoltaic (PV) power output dataset. Compared with other baseline models, the performance of CM-GAN is generally better and reaches the state-of-the-art level. Moreover, sufficient ablation studies are conducted to present the contribution of the cross-modal data fusion technique and show the reasonability of parameter settings of CM-GAN. Apart from this, some prediction experiments are also conducted. The results show that the PV data recovered by CM-GAN can provide more predictability information for improving the prediction accuracy of deep learning model.

Condition monitoring of Industrial machine’s transmission elements – MSDF feature extraction approach

ABSTRACT Predictive maintenance is a modern Industry 4.0 strategy in which machines are continuously monitored to detect flaws and prevent breakdowns before they occur. A single sensor focuses on a single variable while neglecting larger information components, resulting in poor data quality and an increased risk of issues in critical rotating equipment. Multi-sensory configuration technology has been developed to collect huge amounts of data from a machine in order to enhance monitoring capabilities in terms of precision, resolution, efficiency, resilience, and trustworthiness of the overall system. The goal of this work is to provide an integrated perspective on machine monitoring using the Multi Sensor Data Fusion (MSDF) technique. On four fault bearings, a case study contrasts the results of single and multiple sensors. A feature-level data fusion method is used, in which computations using time-domain vibration signature data are utilised to build a fusional vector, which is then classified using SVM and analysed with Gaussian kernels. The experimental results suggest that the proposed Gaussian kernel with SVM technique outperforms single sensor data interpretation in terms of classification accuracy and generalisation capability. It is an efficient way for finding defects in rotating machinery in excessively noisy environments.

Time-resolved ion energy measurements using a retarding potential analyzer for electric propulsion applications.

To completely characterize the evolving state of a plasma, diagnostic tools that enable measurements of the time-resolved behavior are required. In this study, a gridded ion source with superimposed oscillations was utilized to verify the functionality of a high-speed retarding potential analyzer (HSRPA), at frequencies equivalent to the low frequency oscillations occurring in Hall effect thrusters (HETs). The verification of this device provides an effective alternative to existing diagnostics for measuring time-resolved ion energies. Retarding potential analyzers (RPAs) have established themselves as a fundamental diagnostic in the field of electric propulsion (EP), enabling the measurement of ion energy distributions within the plumes of EP thrusters. The work presented here has demonstrated the capability of a standard RPA in conjunction with high-speed circuitry and data fusion techniques to produce time-resolved ion energy distribution functions (IEDFs) at higher frequencies and beam potentials than have previously been investigated. Tested frequencies ranged between 20 and 80kHz with 10V peak-to peak oscillations at a mean beam potential of 570V. In addition, measurements were conducted with several waveforms, functioning as the superimposed oscillation, including a sine wave, triangle wave, and noisy sine wave. Data from the HSRPA were successfully reconstructed into time series utilizing two data fusion techniques: the empirical transfer function method and shadow manifold interpolation. Time-resolved IEDFs were produced at all frequency set points and waveforms. This investigation has demonstrated the HSRPA effectiveness at producing time-resolved measurements under conditions similar to those occurring in HETs.

The Multisensor Data Fusion Method Based on Improved Fuzzy Evidence Theory in the Coal Mine Environment

An enhanced evidence theory-based multisensor data fusion technique is presented to address the problem of poor data fusion caused by an unknown interference in the fully automated mining face multisensor system of a coal mine. Initially, the set of all measurement values is considered as the identification framework, and the principles of fuzzy mathematics are applied to introduce the membership function. This leads to the proposal of a novel method for calculating mutual support among multiple sensors. Furthermore, the basic belief assignment (BBA) in evidence theory is determined by measuring the confidence distance between sensors. Subsequently, a divergence measure is employed to assess the level of conflict and difference between BBA functions, which serves as an indicator of their credibility. The credibility of BBA functions is further adjusted by calculating their information volume using Shannon entropy. This adjustment aims to increase the weight of BBA functions that exhibit less conflict with other BBA functions. Ultimately, the fusion result is obtained through an evidence combination rule based on a conflict allocation. The numerical experimental results demonstrate that the proposed approach achieves higher accuracy, better robustness, and generality compared to the existing methods.

QoE Estimation of WebRTC-based Audio-visual Conversations from Facial and Speech Features

The utilization of user’s facial- and speech-related features for the estimation of the Quality of Experience (QoE) of multimedia services is still underinvestigated despite its potential. Currently, only the use of either facial or speech features individually has been proposed, and relevant limited experiments have been performed. To advance in this respect, in this study, we focused on WebRTC-based videoconferencing, where it is often possible to capture both the facial expressions and vocal speech characteristics of the users. First, we performed thorough statistical analysis to identify the most significant facial- and speech-related features for QoE estimation, which we extracted from the participants’ audio-video data collected during a subjective assessment. Second, we trained individual QoE estimation machine learning-based models on the separated facial and speech datasets. Finally, we employed data fusion techniques to combine the facial and speech datasets into a single dataset to enhance the QoE estimation performance due to the integrated knowledge provided by the fusion of facial and speech features. The obtained results demonstrate that the data fusion technique based on the Improved Centered Kernel Alignment (ICKA) allows for reaching a mean QoE estimation accuracy of 0.93, whereas the values of 0.78 and 0.86 are reached when using only facial or speech features, respectively.

Understanding electric vehicle ownership using data fusion and spatial modeling

The global shift toward electric vehicles (EVs) for climate sustainability lacks comprehensive insights into the impact of the built environment on EV ownership, especially in varying spatial contexts. This study, focusing on New York State, integrates data fusion techniques across diverse datasets to examine the influence of socioeconomic and built environmental factors on EV ownership. The utilization of spatial regression models reveals consistent coefficient values, highlighting the robustness of the results, with the Spatial Lag model better at capturing spatial autocorrelation. Results underscore the significance of charging stations within a 10-mile radius, indicative of a preference for convenient charging options influencing EV ownership decisions. Factors like higher education levels, lower rental populations, and concentrations of older population align with increased EV ownership. Utilizing publicly available data offers a more accessible avenue for understanding EV ownership across regions, complementing traditional survey approaches.

Modelling parking behaviour of commercial vehicles: a scoping review

ABSTRACT Parking in dense urban areas is a major challenge for last mile logistics. Parking shortage and policies that do not address commercial vehicles’ needs often lead these vehicles to park illegally. This paper conducts a scoping literature review on the parking behaviours of commercial freight and service vehicles, methods used to model these behaviours, and factors that determine their outcomes. Thirty-four studies are included in the review. It is found that commercial vehicles’ parking behaviours mainly comprise parking location and type choices including illegal parking, parking duration, and parking cruising. Methods used to model these behaviours primarily include discrete-choice modelling, regression analysis, survival analysis and simulation. We identify key knowledge gaps and provide insights on research opportunities in modelling more complex parking decisions, investigating parking cruising of commercial vehicles, evaluating the implications of freight demand management, and developing data fusion techniques.

Multi-sensor data fusion framework for energy optimization in smart homes

Advancements in Internet of Energy (IoE) technologies drive the development of several energy efficient frameworks for better energy optimization, economic savings, safety, and security in smart homes. However, certain challenges such as real-time operational data for each micro-moment, proper application of data fusion techniques, and end-to-end computing and deployment architecture prevent the establishment of an effective energy-efficient framework to provide personalized energy-saving recommendations. This work presents energy management for smart spaces (EMSS), the proposed energy efficiency framework implemented in an edge-cloud computing platform that fuses data from heterogeneous data sources (environmental sensors, camera, plug data, etc.) at appropriate data fusion levels and process them to generate actionable, explainable, personalized, and persuasive recommendations at the right moment. The user response to the generated recommendations triggers the actuators to perform respective energy-saving actions and provide more personalized future recommendations. Further, SMARTHome - a data generation framework based on configurable scenario files and a set of software codes was proposed to generate synthetic data with respect to different building types and micro-moments. The functionalities of the EMSS (device and user registration), user dashboard, analytics, and energy-saving recommendations were made accessible to the user through web and mobile applications. The validation analysis of the EMSS was performed by (i) comparative analysis of the machine learning and deep learning algorithms used by the decision engine to generate energy-saving recommendations and (ii) benchmarking of EMSS based on the taxonomy of data fusion-based energy efficiency frameworks for smart homes.

Utilizing Fuzzy Logic in Precision Agriculture: Techniques for Disease Detection and Management

Utilizing Fuzzy Logic in Precision Agriculture: Techniques for Disease Detection and Management