Feature Level Data Fusion Research Articles

Rapid and accurate quantification of trypsin activity using integrated infrared and ultraviolet spectroscopy with data fusion techniques

Proteases play a crucial role in industrial enzyme formulations, with activity fluctuations significantly impacting product quality and yield. Therefore, developing a method for precise and rapid detection of protease activity is paramount. This study aimed to develop a rapid and accurate method for quantifying trypsin activity using integrated infrared (IR) and ultraviolet (UV) spectroscopy combined with data fusion techniques. The developed method evaluates the enzymatic activity of trypsin under varying conditions, including temperature, pH, and ionic strength. By comparing different data fusion methods, the study identifies the optimal model for accurate enzyme activity prediction. The results demonstrated significant improvements in predictive performance using the feature-level data fusion approach. Additionally, substituting the spectral data of the samples in the validation sets into the best prediction model resulted in a minimal residual difference between predicted and true values, further verifying the model's accuracy and reliability. This innovative approach offers a practical solution for the efficient and precise quantification of enzyme activity, with broad applications in industrial processes.

Comparison of data fusion strategies for automated prostate lesion detection using mpMRI correlated with whole mount histology

BackgroundIn this work, we compare input level, feature level and decision level data fusion techniques for automatic detection of clinically significant prostate lesions (csPCa).MethodsMultiple deep learning CNN architectures were developed using the Unet as the baseline. The CNNs use both multiparametric MRI images (T2W, ADC, and High b-value) and quantitative clinical data (prostate specific antigen (PSA), PSA density (PSAD), prostate gland volume & gross tumor volume (GTV)), and only mp-MRI images (n = 118), as input. In addition, co-registered ground truth data from whole mount histopathology images (n = 22) were used as a test set for evaluation.ResultsThe CNNs achieved for early/intermediate / late level fusion a precision of 0.41/0.51/0.61, recall value of 0.18/0.22/0.25, an average precision of 0.13 / 0.19 / 0.27, and F scores of 0.55/0.67/ 0.76. Dice Sorensen Coefficient (DSC) was used to evaluate the influence of combining mpMRI with parametric clinical data for the detection of csPCa. We compared the DSC between the predictions of CNN’s trained with mpMRI and parametric clinical and the CNN’s trained with only mpMRI images as input with the ground truth. We obtained a DSC of data 0.30/0.34/0.36 and 0.26/0.33/0.34 respectively. Additionally, we evaluated the influence of each mpMRI input channel for the task of csPCa detection and obtained a DSC of 0.14 / 0.25 / 0.28.ConclusionThe results show that the decision level fusion network performs better for the task of prostate lesion detection. Combining mpMRI data with quantitative clinical data does not show significant differences between these networks (p = 0.26/0.62/0.85). The results show that CNNs trained with all mpMRI data outperform CNNs with less input channels which is consistent with current clinical protocols where the same input is used for PI-RADS lesion scoring.Trial registrationThe trial was registered retrospectively at the German Register for Clinical Studies (DRKS) under proposal number Nr. 476/14 & 476/19.

Research on classification methods for rubber based on terahertz time-domain spectroscopy with data fusion strategy

Rubber, an essential industrial product, plays a crucial role in industry and human life. However, some manufacturers of rubber use inferior, recycled or low-cost rubber materials to replace high-quality rubber products. Therefore, in order to ensure industrial production, protect natural environment and ensure personal safety, it is of great economic and social significance to enable rapid and non-destructive qualitative identification of rubber products. In this paper, eight different types of black rubber, including Nitrile Butadiene Rubber (NBR), Choloroprene Rubber (CR) and Ethylen Propylen Diene Monomer (EPDM), were selected as research objects. Terahertz Time-Domain Spectroscopy (THz-TDS) technology was utilized to get the absorption spectra, refractive index spectra and time-domain spectra of these samples for exploring the differences between different types of rubber in terahertz spectra. To reduce the data volume, three band selection algorithms were employed to extract features from the absorption spectra and refractive index spectra, and temporal features of time-domain spectra were extracted for subsequent data processing. In the selection of classification model algorithms, K-Nearest Neighbor (KNN), Random Forest (RF) and Supervised Kohonen Neural Networks (S-Kohonen) were employed to recognize these spectral data. To fully exploit the latent information within these data and considering the possibility of the loss of partial information in single-type signal, a feature-level data fusion method was applied to different optical parameter signals to obtain more information about terahertz spectra during the process of data modeling. The results of the research indicated that the model with the highest prediction accuracy was the KNN model, with an accuracy of 97.37%, when refractive index spectra and time-domain spectra were fused as inputs, the F1-score reached 99.64. The performance of data fusion was noticeably better than that of a single-type signal. This study validates the feasibility of using terahertz time-domain spectroscopy for qualitative identification of different types of rubber.

Condition monitoring of Industrial machine’s transmission elements – MSDF feature extraction approach

ABSTRACT Predictive maintenance is a modern Industry 4.0 strategy in which machines are continuously monitored to detect flaws and prevent breakdowns before they occur. A single sensor focuses on a single variable while neglecting larger information components, resulting in poor data quality and an increased risk of issues in critical rotating equipment. Multi-sensory configuration technology has been developed to collect huge amounts of data from a machine in order to enhance monitoring capabilities in terms of precision, resolution, efficiency, resilience, and trustworthiness of the overall system. The goal of this work is to provide an integrated perspective on machine monitoring using the Multi Sensor Data Fusion (MSDF) technique. On four fault bearings, a case study contrasts the results of single and multiple sensors. A feature-level data fusion method is used, in which computations using time-domain vibration signature data are utilised to build a fusional vector, which is then classified using SVM and analysed with Gaussian kernels. The experimental results suggest that the proposed Gaussian kernel with SVM technique outperforms single sensor data interpretation in terms of classification accuracy and generalisation capability. It is an efficient way for finding defects in rotating machinery in excessively noisy environments.

Research on Multi-Source Heterogeneous Big Data Fusion Method Based on Feature Level

With the development of research on multi-modal data fusion and its combination with online data management, the application of multi-modal big data fusion in information management systems is more and more extensive. How to integrate multi-modal big data effectively is the key technology to building an efficient information management system. In this paper, based on the combination of a multi-support vector machine and convolution neural network, the feature-level data fusion of multi-source heterogeneous big data is implemented, and it is applied to the real data set to test the relevant model. Experimental results show that this method can not only realize heterogeneous integration of big data, but also has high accuracy and reliability.

A Comparison of Different Data Fusion Strategies’ Effects on Maize Leaf Area Index Prediction Using Multisource Data from Unmanned Aerial Vehicles (UAVs)

The leaf area index (LAI) is an important indicator for crop growth monitoring. This study aims to analyze the effects of different data fusion strategies on the performance of LAI prediction models, using multisource images from unmanned aerial vehicles (UAVs). For this purpose, maize field experiments were conducted to obtain plants with different growth status. LAI and corresponding multispectral (MS) and RGB images were collected at different maize growth stages. Based on these data, different model design scenarios, including single-source image scenarios, pixel-level multisource data fusion scenarios, and feature-level multisource data fusion scenarios, were created. Then, stepwise multiple linear regression (SMLR) was used to design LAI prediction models. The performance of models were compared and the results showed that (i) combining spectral and texture features to predict LAI performs better than using only spectral or texture information; (ii) compared with using single-source images, using a multisource data fusion strategy can improve the performance of the model to predict LAI; and (iii) among the different multisource data fusion strategies, the feature-level data fusion strategy performed better than the pixel-level fusion strategy in the LAI prediction models. Thus, a feature-level data fusion strategy is recommended for the creation of maize LAI prediction models using multisource UAV images.

Bolt tightness monitoring using multiple reconstructed narrowband Lamb waves combined with piezoelectric ultrasonic transducer

Narrowband tone burst excitation with a carefully selected frequency is generally utilized to generate Lamb waves for mode purity, dispersion diminution and signal interpretability. Narrowband excitation exhibits saturation phenomenon and even non-monotonic trends with the development of bolt looseness. In this research, a piezoelectric ultrasonic transducer with high signal-to-noise ratio is designed and fabricated for bolt tightness monitoring. The chirp signal is utilized to drive the actuator to generate broadband Lamb waves in the connected plate, and a transfer function-based signal reconstruction algorithm is innovatively proposed to extract narrowband tone burst responses with different center frequencies from the broadband chirp response for bolt tightness monitoring. The leaked wave energy-based bolt tightness indexes (TIs) are calculated based on the extracted multiple tone burst responses and a feature-level data fusion strategy is proposed to combine the TIs for exploiting the merits of different-frequency inspecting waves for different bolt tightness conditions. The fused TI not only presents a monotonic tendency with the increasing of bolt tightness, but also tackles the low sensitivity of narrowband Lamb waves for embryo bolt looseness. The proposed method contributes a novel and stable scheme for bolt tightness monitoring and opens a new perspective for damage evaluation of structures.

Use of Machine Learning with Fused Spectral Data for Prediction of Product Sensory Characteristics: The Case of Grape to Wine

Generations of sensors have been developed for predicting food sensory profiles to circumvent the use of a human sensory panel, but a technology that can rapidly predict a suite of sensory attributes from one spectral measurement remains unavailable. Using spectra from grape extracts, this novel study aimed to address this challenge by exploring the use of a machine learning algorithm, extreme gradient boosting (XGBoost), to predict twenty-two wine sensory attribute scores from five sensory stimuli: aroma, colour, taste, flavour, and mouthfeel. Two datasets were obtained from absorbance-transmission and fluorescence excitation-emission matrix (A-TEEM) spectroscopy with different fusion methods: variable-level data fusion of absorbance and fluorescence spectral fingerprints, and feature-level data fusion of A-TEEM and CIELAB datasets. The results for externally validated models showed slightly better performance using only A-TEEM data, predicting five out of twenty-two wine sensory attributes with R2 values above 0.7 and fifteen with R2 values above 0.5. Considering the complex biotransformation involved in processing grapes to wine, the ability to predict sensory properties based on underlying chemical composition in this way suggests that the approach could be more broadly applicable to the agri-food sector and other transformed foodstuffs to predict a product's sensory characteristics from raw material spectral attributes.

A feature-level degradation measurement method for composite health index construction and trend prediction modeling

Accurate measurement of degradation levels and evolution trend for mechanical equipment has great significance to achieve the goal of condition-based maintenance (CBM) and increase the economic benefit. In this paper, a feature-level degradation measurement method is proposed for composite health index (CHI) construction and trend prediction, systematically blending the multi-source signals analysis theory and deep learning technology. First, a CHI is innovatively constructed based on a novel hybrid stacked auto-encoder (HSAE) and self-organizing map (SOM) network to comprehensively evaluate the degradation levels, which achieves the feature-level data fusion for degradation measurement. Second, considering the significant characteristics of strong nonlinear and random fluctuations for the generated CHI sequence, a hybrid algorithm, combining the variational mode decomposition (VMD) and a dynamic step size-based fruit fly optimization algorithm (DSSFOA), is applied for effectively extracting the multiscale frequency components from raw sequence and capturing the inherent laws of health degradation from different scales. Specifically, by designing a hierarchical search structure with dynamic step sizes in DSSFOA, the parameters of VMD can be optimized with high efficiency and accuracy. Finally, a scale-adaptive attention (SAA)-based bidirectional long short-term memory (BiLSTM) network and the corresponding dual-stage training strategy are developed to adaptively estimate the attention weights of different components and further predict the degradation trend in future. Extensive experiments on the C-MAPSS turbofan engine dataset and the PSU dataset validate the more superior performance of the proposed method on both of health index construction and degradation trend prediction than other existing methods.

Exploring human activity recognition using feature level fusion of inertial and electromyography data.

Wearables are objective tools for human activity recognition (HAR). Advances in wearables enable synchronized multi-sensing within a single device. This has resulted in studies investigating the use of single or multiple wearable sensor modalities for HAR. Some studies use inertial data, others use surface electromyography (sEMG) from multiple muscles and different post-processing approaches. Yet, questions remain about accuracies relating to e.g., multi-modal approaches, and sEMG post-processing. Here, we explored how inertial and sEMG could be efficiently combined with machine learning and used with post-processing methods for better HAR. This study aims recognition of four basic daily life activities; walking, standing, stair ascent and descent. Firstly, we created a new feature vector based on the domain knowledge gained from previous mobility studies. Then, a feature level data fusion approach was used to combine inertial and sEMG data. Finally, two supervised learning classifiers (Support Vector Machine, SVM, and the k-Nearest Neighbors, kNN) were tested with 5-fold cross-validation. Results show the use of inertial data with sEMG increased overall accuracy by 3.5% (SVM) and 6.3% (kNN). Extracting features from linear envelopes instead of bandpass filtered sEMG improves overall HAR accuracy in both classifiers. Clinical Relevance- Post-processing on sEMG signals can improve the performance of multimodal HAR.

A convolutional neural network-based multi-sensor fusion approach for in-situ quality monitoring of selective laser melting

Selective laser melting (SLM) is an emerging and popular metal additive manufacturing (AM) technique to fabricate advanced metal components with complex geometries. However, its broad adoption in industry is still hampered by poor process repeatability and low part consistency. To overcome this limitation, there are growing efforts towards in-situ monitoring and control technologies for quality assessment of parts. In this paper, a convolutional neural network (CNN)-based multi-sensor fusion approach is proposed to integrate layer-wise images, acoustic emission signals, and photodiode signals for in-situ quality monitoring of SLM. An off-axial in-situ monitoring system equipped with three types of sensors, namely, a digital camera, a microphone, and a photodiode is first developed for signatures acquisition. Porosity and density measurements are then carried out to label the layer-based sensing data and measure the quality. Thereafter, an image processing method and a signal-to-image strategy are proposed to merge the multi-source heterogeneous sensor data. Finally, three CNN-based multi-sensor fusion models are developed from data-level fusion, feature-level fusion, and decision-level fusion respectively and compared in quality identification. Data-level fusion model is established through channel fusion, features are extracted and fused in the feature-level fusion model, and the decision-level fusion model is developed by fusing the classification results from individual models. Results showed that the proposed CNN-based multi-sensor fusion approach can significantly improve the classification accuracy compared to the three individual sensor-based models. Furthermore, the feature-level data fusion model exhibits the best classification performance among the three data fusion models.

A novel method for steel bar all-stage pitting corrosion monitoring using the feature-level fusion of ultrasonic direct waves and coda waves

Corrosion monitoring of steel bars has drawn extensive attention in recent decades. Conventional ultrasonic method, utilizing direct waves to detect damage, is adequate for severe pitting corrosion but suffers from low sensitivity to incipient pitting corrosion. Coda wave technique, a very sensitive method to subtle changes in medium using later arrival wave packets, is innovatively introduced to monitor pitting corrosion of steel bars, especially in the early stages. The decorrelation coefficient (DC) values are calculated to quantify the variations of both direct waves and coda waves. To overcome the limitations of coda waves for severe pitting corrosion and remedy the low sensitivity of direct waves for incipient pitting corrosion, a feature-level data fusion strategy is proposed to integrate the two probing waves to monitor all-stage pitting corrosion of steel bars. The combination of direct waves and coda waves could exploit the complementary merits in various pitting corrosion configurations. The proposed feature-level fusion strategy of ultrasonic coda waves and direct waves intercepted from the same recorded signals opens a new perspective in all-stage pitting corrosion monitoring of steel bars and contributes a novel scheme for whole-process damage evaluation of structures.

Deep Convolutional Neural Network Framework for Diagnostics of Planetary Gearboxes Under Dynamic Loading With Feature-Level Data Fusion

Abstract Planetary gearboxes are susceptible to premature failures due to cyclic random loadings and extreme operating conditions. Fault diagnostics strategies are crucial to increase operational safety and reduce economic costs. This led to the research question is: Can a deep convolutional neural network (DCNN) with data fusion improve diagnostics of a planetary gearbox using simulated data? To answer this question, a DCNN framework was proposed to diagnose planetary gearbox with crack using simulated time and the frequency response. A finite element model was developed to generate a time-varying mesh stiffness response for gear tooth meshing at different crack levels. The mesh stiffness was expanded in terms of the Fourier series to generate values at any rotational speed and time interval. The generated mesh stiffness response was used on a dynamic model to generate the time and frequency response of the system. An additional data set was generated using feature-level data fusion. The two datasets were fed to the DCNN model to diagnose the crack faults and results were compared. It was shown that the feature-level data fusion method is very robust in diagnosing crack faults with good accuracy rates even with the presence of a high level of noise.

Deep Learning for Feature-Level Data Fusion: Higher Resolution Reconstruction of Historical Landsat Archive

Long-term record of fine spatial resolution remote sensing datasets is critical for monitoring and understanding global environmental change, especially with regard to fine scale processes. However, existing freely available global land surface observations are limited by medium to coarse resolutions (e.g., 30 m Landsat) or short time spans (e.g., five years for 10 m Sentinel-2). Here we developed a feature-level data fusion framework using a generative adversarial network (GAN), a deep learning technique, to leverage the overlapping Landsat and Sentinel-2 observations during 2016–2019, and reconstruct 10 m Sentinel-2 like imagery from 30 m historical Landsat archives. Our tests with both simulated data and actual Landsat/Sentinel-2 imagery showed that the GAN-based fusion method could accurately reconstruct synthetic Landsat data at an effective resolution very close to that of the real Sentinel-2 observations. We applied the GAN-based model to two dynamic systems: (1) land over dynamics including phenology change, cropping rotation, and water inundation; and (2) human landscape changes such as airport construction, coastal expansion, and urbanization, via historical reconstruction of 10 m Landsat observations from 1985 to 2018. The resulting comparison further validated the robustness and efficiency of our proposed framework. Our pilot study demonstrated the promise of transforming 30 m historical Landsat data into a 10 m Sentinel-2-like archive with advanced data fusion. This will enhance Landsat and Sentinel-2 data science, facilitate higher resolution land cover and land use monitoring, and global change research.

Comprehensive fault diagnostics of wind turbine gearbox through adaptive condition monitoring scheme

The current work reports a multi-level classification to envisage the location, type/category and severity level of local defects at different stages of speed in a wind turbine gearbox with minimal human intervention. Experiments are conducted by subjecting a three-stage gearbox to fluctuating speeds with multiple sensors recording the real-time information generated. Wavelet coefficients are employed to extract the statistical features from the raw signatures decomposed through wavelet transform. A decision tree algorithm is used to identify features of significance and an integrated multi-variable feature data set is devised based on feature-level data fusion. The intended multi-level classification on the integrated feature data set is accomplished with the help of machine-learning algorithms. The results reveal that the adaptive neuro-fuzzy inference system (ANFIS) performs the intended four-level classification on the wind turbine gearbox with a classification accuracy of 92%. Thus, the integration of multi-sensor information in conjunction with ANFIS as a classification algorithm, owing to its efficiency in predicting every possible detail about the health/condition of the different gearbox components, demonstrates its potential to be used as an adaptive condition monitoring as it.

Rapid determination of potential aflatoxigenic fungi contamination on peanut kernels during storage by data fusion of HS-GC-IMS and fluorescence spectroscopy

Abstract This study described the rapid determination of potential aflatoxigenic fungi contamination on peanut kernels based on headspace-gas chromatography-ion mobility spectrometry (HS-GC-IMS) coupled to fluorescence spectroscopy. Data-level and feature-level fusion strategies were introduced to integrate HS-GC-IMS and fluorescence spectra, aiming at improving the performances of identification and prediction models. The application of feature-level data fusion using first 10 PCs coupled with orthogonal partial least squares discriminant analysis (OPLS-DA) offered more accurate characterization (96.7 %) for aflatoxigenic and non-aflatoxigenic fungal infection on peanut samples. Regression models were established for predicting colony counts of peanuts infected with aflatoxigenic fungi based on independent and fused signals by partial least squares regression (PLSR). Feature-level data fusion using first 10 PCs achieved the best performances in colony counts predictions for A. flavus (R2 = 0.950) and A. parasiticus (R2 = 0.971). These results demonstrated that the combination of HS-GC-IMS and fluorescence spectra might offer the feasibility for early detection of potential aflatoxigenic risk in peanuts.

Optimal estimation of low-rank factors via feature level data fusion of multiplex signal systems

The design of fusion engines is a subject of great importance in a variety of fields. In this paper, we focus on the problem of linear fusion at the feature level for multiple signal matrices with noises, with the features being extremal eigenvectors. When given multiple similarity matrices, the objective is to find an estimate of the latent signal eigenspace. The concentration result for the inner product of features from different matrix samples is developed, utilizing the random matrix theory. Based on of the theoretical results, we proposed an efficient algorithm, EigFuse, to solve the constrained data-driven optimization problem with different level of noises. Our method is of high efficiency by comparing it with state-of-the-art baseline approaches with multiple noise levels. Comprehensive experiments on several synthetic as well as real-life networks demonstrate our method's superior performance.

Predictive modelling of surface roughness in fused deposition modelling using data fusion

To realise high quality, additively manufactured parts, real-time process monitoring and advanced predictive modelling tools are crucial for accelerating quality assurance in additive manufacturing. While previous research has demonstrated the effectiveness of physics- and model-based diagnosis and prognosis for additive manufacturing, very little research has been reported on real-time monitoring and predictive modelling of the surface roughness of additively manufactured parts. This paper presents a data fusion approach to predicting surface roughness in fused deposition modelling (FDM) processes. The predictive models are trained using random forests (RFs), support vector regression (SVR), ridge regression (RR), and least absolute shrinkage and selection operator (LASSO). A real-time monitoring system is developed to monitor the health condition of a FDM machine in real-time using multiple sensors. RFs, SVR, RR, and LASSO are demonstrated on the condition monitoring data collected from these sensors. To integrate the data sources, a feature-level data fusion method is introduced. Experimental results have shown that the predictive models trained by the machine learning algorithms are capable of predicting the surface roughness of additively manufacturing parts with very high accuracy. The prediction accuracy can be further improved using the data fusion method.

Reliability improvement of nonlinear ultrasonic modulation based fatigue crack detection using feature-level data fusion

In this study, the reliability of nonlinear ultrasonic modulation based fatigue crack detection is improved using a feature-level data fusion approach. When two ultrasonic inputs at two distinct frequencies are applied to a specimen with a fatigue crack, modulation components at the summation and difference of these two input frequencies appear. First, the spectral amplitudes of the modulation components and their spectral correlations are defined as individual features. Then, a 2D feature space is constructed by combining these two features, and the presence of a fatigue crack is identified in the feature space. The effectiveness of the proposed fatigue crack detection technique is experimentally validated through cyclic loading tests of aluminum plates, full-scale steel girders and a rotating shaft component. Subsequently, the improved reliability of the proposed technique is quantitatively investigated using receiver operating characteristic analysis. The uniqueness of this study lies in (1) improvement of nonlinear ultrasonic modulation based fatigue crack detection reliability using feature-level data fusion, (2) reference-free fatigue crack diagnosis without using the baseline data obtained from the intact condition of the structure, (3) application to full-scale steel girders and shaft component, and (4) quantitative investigation of the improved reliability using receiver operating characteristic analysis.

Algorithm for damage detection in wind turbine blades using a hybrid dense sensor network with feature level data fusion

Damage detection in wind turbine blades requires the ability to distinguish local faults over a global area. The implementation of dense sensor networks provides a solution to this local-global monitoring challenge. Here the authors propose a hybrid dense sensor network consisting of capacitive-based thin-film sensors for monitoring the additive strain over large areas and fiber Bragg grating sensors for enforcing boundary conditions. This hybrid dense sensor network is leveraged to derive a data-driven damage detection and localization method for wind turbine blades. In the proposed method, the blade's complex geometry is divided into less geometrically complex sections. Orthogonal strain maps are reconstructed from the sectioned hybrid dense sensor network by assuming different bidirectional shape functions and are solved using the least squares estimator. The error between the estimated strain maps and measured strains is extracted to define damage detection features that are dependent on the selected shape functions. This technique fuses sensor data into a single damage detection feature, providing a simple and robust method for inspecting large numbers of sensors without the need for complex model driven approaches. Numerical simulations demonstrate the proposed method's capability to distinguish healthy sections from possibly damaged sections on simplified 2D geometries.