Kernel Canonical Correlation Analysis Research Articles

Multimodal Emotion Recognition Based on Facial Expressions, Speech, and Body Gestures

The research of multimodal emotion recognition based on facial expressions, speech, and body gestures is crucial for oncoming intelligent human–computer interfaces. However, it is a very difficult task and has seldom been researched in this combination in the past years. Based on the GEMEP and Polish databases, this contribution focuses on trimodal emotion recognition from facial expressions, speech, and body gestures, including feature extraction, feature fusion, and multimodal classification of the three modalities. In particular, for feature fusion, two novel algorithms including supervised least squares multiset kernel canonical correlation analysis (SLSMKCCA) and sparse supervised least squares multiset kernel canonical correlation analysis (SSLSMKCCA) are presented, respectively, to carry out efficient facial expression, speech, and body gesture feature fusion. Different from the traditional multiset kernel canonical correlation analysis (MKCCA) algorithms, our SLSKMCCA algorithm is a supervised version and is based on the least squares form. The SSLSKMCCA algorithm is implemented by the combination of SLSMKCCA and a sparse item (L1 Norm). Moreover, two effective solving algorithms for SLSMKCCA and SSLSMKCCA are presented in addition, which use the alternated least squares and augmented Lagrangian multiplier methods, respectively. The extensive experimental results on the popular public GEMEP and Polish databases show that the recognition rate of multimodal emotion recognition is superior to bimodal and monomodal emotion recognition on average, and our presented SLSMKCCA and SSLSMKCCA fusion methods both obtain very high recognition rates, especially for the SSLSMKCCA fusion method.

Distributed monitoring of nonlinear plant-wide processes based on GA-regularized kernel canonical correlation analysis

Fault detection and diagnosis is important for ensuring process safety and is gaining increasing attention in the system safety field. A regularized kernel canonical correlation analysis (RKCCA) approach is proposed for monitoring nonlinear plantwide processes. For each local unit, genetic algorithm (GA)-based regularization is performed to determine the communication variables from neighboring units, which preserves the maximum correlations and eliminates the irrelevant variables. Then variables from a local unit and the communication variables are mapped into high-dimensional feature spaces, and the feature space of the local unit is decomposed into three orthogonal subspaces, namely the residual subspace, the inner subspace, and the outer-related subspace. Monitoring statistics to identify both the process status and the characteristic of a detected fault are constructed. The proposed RKCCA-based monitoring method considers both the information of a local unit and the beneficial information of related units to facilitate fault detection, thereby exhibiting superior performance to some state-of-the-arts methods. Applications on the Tennessee Eastman benchmark process and an industrial tail gas treatment process demonstrate the superiority of RKCCA monitoring.

Data‐driven nonlinear state observation for controlled systems: A kernel method and its analysis

AbstractThis work proposes a data‐driven state observation algorithm for nonlinear dynamical systems, when the true state trajectory is not measurable and hence the states information needs to be reconstructed from input and output measurements. Such a reduction is formed by kernel canonical correlation analysis (KCCA), which (i) implicitly maps the available input–output data into a higher‐dimensional feature space, namely the reproducing kernel Hilbert space (RKHS); (ii) finds a projection of the past input–output data and a projection of the future input–output data with maximal correlation; and (iii) identifies the projected inputs and outputs, namely the canonical variates, as the observed states. We adopt a least squares support vector machine (LS‐SVM) formulation for KCCA, which imposes regularization on the vectors that specify the projections and is amenable to convex optimization. We prove theoretically that, based on the statistical consistency of KCCA, the observed states determined by the proposed state observer has a guaranteed correlativity with the actual states (when properly transformed). Furthermore, such observed states, when supplemented with the information of succeeding inputs, can be used to predict the succeeding outputs with guaranteed upper bound on the prediction error. Case studies are performed on two numerical examples and an exothermic continuously stirred tank reactor (CSTR).

Data oversampling and feature selection for class imbalanced datasets

Introduction: Significant advancements and modifications have been implemented in data classification (DC) in the past few decades. Due to their infinite quantity and imbalance, data becomes challenging for classification. The biggest concern in DM (Data Mining) is Class Imbalance (CI). To avoid these issues in recent work proposed map reduce based data parallelization of class imbalanced datasets. Methods: A novel Over Sampling (OS) technique called Minority Oversampling in Kernel Canonical Correlation Adaptive Subspaces (MOKCCAS) has been suggested with the objective to minimize data loss throughout (FSP) Feature Space Projections. This technique takes advantage of the constant Feature Extraction (FE) capability of a version of the ASSOM (Adaptive Subspace Self-Organizing Maps) that is derived from Kernel Canonical Correlation Analysis (KCCA). And in classification, Feature Selection (FS) plays an important role because the acquired dataset might contain large volume of samples, utilizing all features of samples from the dataset for classification will decrease the classifier performance. And then data parallelization will be done by using map reduce framework to solve this computation requirement problem.Result: Then proposes a feature selection model using Mutated whale optimization (MWO) methods and produces features and reduces the time consumption. Finally proposed class balancing model will be tested using uniform distribution based enhanced adaptive neuro fuzzy inference system (UDANFIS). Test outcomes validate the efficiency of the suggested technique by precision, recall, accuracy and Error Rate (ER).Conclusion: The study subsequently suggests a novel OS approach called MOKCCAS to lessen the loss of data throughout feature space projection.

Audio Scanning Network: Bridging Time and Frequency Domains for Audio Classification

With the rapid growth of audio data, there's a pressing need for automatic audio classification. As a type of time-series data, audio exhibits waveform fluctuations in both the time and frequency domains that evolve over time, with similar instances sharing consistent patterns. This study introduces the Audio Scanning Network (ASNet), designed to leverage abundant information for achieving stable and effective audio classification. ASNet captures real-time changes in audio waveforms across both time and frequency domains through reservoir computing, supported by Reservoir Kernel Canonical Correlation Analysis (RKCCA) to explore correlations between time-domain and frequency-domain waveform fluctuations. This innovative approach empowers ASNet to comprehensively capture the changes and inherent correlations within the audio waveform, and without the need for time-consuming iterative training. Instead of converting audio into spectrograms, ASNet directly utilizes audio feature sequences to uncover associations between time and frequency fluctuations. Experiments on environmental sound and music genre classification tasks demonstrate ASNet's comparable performance to state-of-the-art methods.

Specific Emitter Identification through Multi-Domain Mixed Kernel Canonical Correlation Analysis

Radar specific emitter identification (SEI) involves extracting distinct fingerprints from radar signals to precisely attribute them to corresponding radar transmitters. In view of the limited characterization of fingerprint information by single-domain features, this paper proposes the utilization of multi-domain mixed kernel canonical correlation analysis for radar SEI. Initially, leveraging the complementarity across diverse feature domains, fingerprint features are extracted from four distinct domains including: envelope feature, spectrum feature, short-time Fourier transform and ambiguity function. Subsequently, kernel canonical correlation analysis is employed to amalgamate the correlation characteristics inherent in multi-domain data. Considering the insufficient of a single kernel function with only interpolation or extrapolation ability, we adopt mixed kernel to improve the projection ability of the kernel function. Experimental results substantiate that the proposed feature fusion approach maximizes the complementarity of multiple features while reducing feature dimensionality. The method achieves an accuracy of up to 95% in experiments, thereby enhancing the efficacy of radar SEI.

Weed density evaluation using KCCA-CFBLS based on fusion of visual and tactile features in special paddy field environment

Accurately evaluating weed density in paddy fields is crucial for achieving high-quality control of rice weeds, as it enables precise application of herbicides or matching of mechanical weeding intensity. However, visual methods alone may not provide ideal classification of weed density in specialized paddy field environments due to complex backgrounds, environmental interference, and visual occlusion. In this study, we propose an innovative method that combines vision and tactile features to achieve accurate weed density evaluation. We first obtained tactile data containing weed density information using a self-made tactile sensor, while simultaneously capturing corresponding visual images. We then utilized the kernel canonical correlation analysis (KCCA) algorithm to maximize the correlation of discriminating features extracted from the tactile data and visual images, and obtained fusion eigenvectors that characterize the weed density. These eigenvectors were used as input for a broad learning system (BLS), in which the random feature mapping was replaced by a cascade feature node to construct a KCCA-based cascade feature broad learning system (KCCA-CFBLS). Our method achieved accurate evaluation of weed density in low, medium, and high weed conditions using a novel weed dataset from a specialized paddy field environment. We analyzed the accuracy and running time, and found that KCCA-CFBLS outperformed methods based on SVM and YOLOv5-Lite, achieving 11.56% and 7.56% higher accuracy, respectively, with less time consumption. Experimental results demonstrated significant advantages of our method in terms of accuracy and real-time performance compared to visual methods, providing a decision-making basis for the implementation of intelligent chemical and mechanical weeding in specialized paddy field environments.

Statistical matching using kernel canonical correlation analysis and super-organizing map

The potential to study and improve different aspects of our lives is ever growing thanks to the abundance of data available in today’s modern society. Scientists and researchers often need to analyze data from different sources; the observations, which only share a subset of the variables, cannot always be paired to detect common individuals. This is the case, for example, when the information required to study a certain phenomenon is coming from different sample surveys. Statistical matching is a common practice to combine these data sets. In this paper, we investigate and extend to statistical matching two methods based on Kernel Canonical Correlation Analysis (KCCA) and Super-Organizing Map (Super-OM). These methods are designed to deal with various variable types, sample weights and incompatibilities among categorical variables. In the first case, we use KCCA, a non-linear extension of CCA, to create canonical variables that we can compare in the two data sets. In the second case, Super-OM uses organizing maps to create subgroups of individuals who share the same characteristics. We use the 2017 Belgian Statistics on Income and Living Conditions (SILC) and we compare the performance of the proposed statistical matching methods by means of a cross-validation technique, as if the data were available from two separate sources. The results indicate that our proposed methods are superior to existing methods because they preserve the distribution of generated variables while also providing good predictions. Existing methods typically only achieve one or the other. These new techniques open the door to improving statistical matching in other applications such as medicine, economics, …

Exploring changes in brain function in IBD patients using SPCCA: a study of simultaneous EEG-fMRI.

Research on functional changes in the brain of inflammatory bowel disease (IBD) patients is emerging around the world, which brings new perspectives to medical research. In this paper, the methods of canonical correlation analysis (CCA), kernel canonical correlation analysis (KCCA), and sparsity preserving canonical correlation analysis (SPCCA) were applied to the fusion of simultaneous EEG-fMRI data from 25 IBD patients and 15 healthy individuals. The CCA, KCCA and SPCCA fusion methods were used for data processing to compare the results obtained by the three methods. The results clearly show that there is a significant difference in the activation intensity between IBD and healthy control (HC), not only in the frontal lobe (p < 0.01) and temporal lobe (p < 0.01) regions, but also in the posterior cingulate gyrus (p < 0.01), gyrus rectus (p < 0.01), and amygdala (p < 0.01) regions, which are usually neglected. The mean difference in the SPCCA activation intensity was 60.1. However, the mean difference in activation intensity was only 36.9 and 49.8 by using CCA and KCCA. In addition, the correlation of the relevant components selected during the SPCCA calculation was high, with correlation components of up to 0.955; alternatively, the correlations obtained from CCA and KCCA calculations were only 0.917 and 0.926, respectively. It can be seen that SPCCA is indeed superior to CCA and KCCA in processing high-dimensional multimodal data. This work reveals the process of analyzing the brain activation state in IBD disease, provides a further perspective for the study of brain function, and opens up a new avenue for studying the SPCCA method and the change in the intensity of brain activation in IBD disease.

Spatio-temporal information analytics based performance-driven industrial process monitoring framework with cloud–edge-device collaboration

Smart manufacturing in the context of Industry 4.0 presents new characteristics, such as large-scale, complex sub-processes cooperation, and integrated information management, which makes comprehensive performance-driven process monitoring difficult to be achieved in practice. Cloud–edge-device collaboration brings new perspectives for modern industries. This paper presents a performance-driven process monitoring framework in cloud–edge-device collaborative architecture, including a spatio-temporal information analytics based method and its application to a hot strip mill process (HSMP). Firstly, an integrated performance-driven process monitoring framework is proposed, where different monitoring tasks are deployed in the device, edge, and cloud layers, respectively. Secondly, a new performance-related spatio-temporal information analytics based fault detection and localization method is developed. In detail, temporal kernel canonical correlation analysis (CCA) is employed to extract the canonical correlation features with nonlinearities in the temporal domains, and spatial CCA is designed to form the spatial features. Based on the spatio-temporal features, the equipment operation condition monitoring model is developed in the device layer for real-time fault localization, and the process monitoring model is deployed in the edge for fault detection. Furthermore, to give a high-level evaluation of the operating status of the process, a qualitative trend analysis based operation risks evaluation model is deployed in the cloud. Different from the traditional architectures, the data processing capability of each layer is fully utilized in the proposed one, and the function is relevant to the enterprise management system. It provides a topological basis with clear priorities for subsequent process regulation and optimization. Finally, an actual HSMP is introduced to testify the performance of the framework. From the results, the fault detection rates are improved in comparison with traditional methods, and the fault localization results are consistent with the actual situations.

Transfer Learning based Low Shot Classifier for Software Defect Prediction

Background: The rapid growth and increasing complexity of software applications are causing challenges in maintaining software quality within constraints of time and resources. This challenge led to the emergence of a new field of study known as Software Defect Prediction (SDP), which focuses on predicting future defect in advance, thereby reducing costs and improving productivity in software industry. Objective: This study aimed to address data distribution disparities when applying transfer learning in multi-project scenarios, and to mitigate performance issues resulting from data scarcity in SDP. Methods: The proposed approach, namely Transfer Learning based Low Shot Classifier (TLLSC), combined transfer learning and low shot learning approaches to create an SDP model. This model was designed for application in both new projects and those with minimal historical defect data. Results: Experiments were conducted using standard datasets from projects within the National Aeronautics and Space Administration (NASA) and Software Research Laboratory (SOFTLAB) repository. TLLSC showed an average increase in F1-Measure of 31.22%, 27.66%, and 27.54% for project AR3, AR4, and AR5, respectively. These results surpassed those from Transfer Component Analysis (TCA+), Canonical Correlation Analysis (CCA+), and Kernel Canonical Correlation Analysis plus (KCCA+). Conclusion: The results of the comparison between TLLSC and state-of-the-art algorithms, namely TCA+, CCA+, and KCCA+ from the existing literature consistently showed that TLLSC performed better in terms of F1-Measure. Keywords: Just-in-time, Defect Prediction, Deep Learning, Transfer Learning, Low Shot Learning

ORCCA: Optimal Randomized Canonical Correlation Analysis.

Random features approach has been widely used for kernel approximation in large-scale machine learning. A number of recent studies have explored data-dependent sampling of features, modifying the stochastic oracle from which random features are sampled. While proposed techniques in this realm improve the approximation, their suitability is often verified on a single learning task. In this article, we propose a task-specific scoring rule for selecting random features, which can be employed for different applications with some adjustments. We restrict our attention to canonical correlation analysis (CCA) and provide a novel, principled guide for finding the score function maximizing the canonical correlations. We prove that this method, called optimal randomized CCA (ORCCA), can outperform (in expectation) the corresponding kernel CCA with a default kernel. Numerical experiments verify that ORCCA is significantly superior to other approximation techniques in the CCA task.

Parametric Model for Coaxial Cavity Filter with Combined KCCA and MLSSVR

Aiming at the problems of poor data effectiveness, low modeling accuracy, and weak generalization in the tuning process of microwave cavity filters, a parametric model for coaxial cavity filter using kernel canonical correlation analysis (KCCA) and multioutput least squares support vector regression (MLSSVR) is proposed in this study. First, the low-dimensional tuning data is mapped to the high-dimensional feature space by kernel canonical correlation analysis, and the nonlinear feature vectors are fused by the kernel function; second, the multioutput least squares support vector regression algorithm is used for parametric modeling to solve the problems of low accuracy and poor prediction performance; third, the support vector of the parameter model is optimized by the differential evolution whale algorithm (DWA) to improve the convergence and generalization ability of the model in actual tuning. Finally, the tuning experiments of two cavity filters with different topologies are carried out. The experimental results show that the proposed method has an obvious improvement in generalization performance and prediction accuracy compared with the traditional methods.

KPCA-CCA-Based Quality-Related Fault Detection and Diagnosis Method for Nonlinear Process Monitoring

This work concerns the issue of quality-related fault detection and diagnosis (QrFDD) for nonlinear process monitoring. A kernel principal component analysis (KPCA) based canonical correlation analysis (CCA) model is proposed in this paper. First, KPCA is utilized to extract the kernel principal components (KPCs) of original variables data to eliminate nonlinear coupling among the variables. Then, the KPCs and output are used for CCA modeling, which not only avoids the complex decomposition of kernel CCA but also maintains high interpretability. Afterwards, under the premise of Gaussian kernel, a proportional relationship between process variables sample and kernel sample is introduced, on the basis of which, the linear regression model between process and quality variables is established. Based on the coefficient matrix of the regression model, a nonlinear QrFDD method is finally implemented which has both the data processing capability of nonlinear methods and the form of linear methods. Therefore, it significantly outperforms existing kernel-based CCA methods in terms of algorithmic complexity and interpretability, which demonstrated by the simulation results of the Tennessee-Eastman (TE) chemical process.

Learning Sparse Kernel CCA With Graph Priors for Nonlinear Process Monitoring

Process monitoring (PM) is important for improving product quality and ensuring plant safety in industrial systems. Recently, canonical correlation analysis (CCA)-based PM has shown excellent performance. The core idea is to first seek the relationship between two sets of process variables via CCA, then construct a residual generator and determine test statistics, and finally achieve the best monitoring. However, the global and local structure of process variables is not fully exploited. In this article, we propose a new nonlinear PM approach called joint sparse constrained kernel CCA and graph learning (JSKCCA-GL) by incorporating joint sparse kernel CCA (JSKCCA) and graph priors. Technically, imposing the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{{2},{0}}$ </tex-math></inline-formula> -norm joint sparse constraints can capture the global row-wise structure, and embedding the graph Laplacian priors can preserve the local structured variable correlation information. Furthermore, an efficient alternating minimization algorithm is developed by integrating least squares and iterative hard thresholding (IHT). The superiority of the proposed JSKCCA-GL is extensively verified on the benchmark Tennessee Eastman process (TEP) and a practical cylinder-piston assembly (CPA) of marine diesel engines. In particular, the fault detection rate (FDR) values for unknown-type faults in the TEP have been improved by around 16.10% on average.

K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human–Robot Interaction

In this article, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -meansclustering-based Kernel canonical correlation analysis algorithm is proposed for multimodal emotion recognition in human–robot interaction (HRI). The multimodal features (gray pixels; time and frequency domain) extracted from facial expression and speech are fused based on Kernel canonical correlation analysis. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means clustering is used to select features from multiple modalities and reduce dimensionality. The proposed approach can improve the heterogenicity among different modalities and make multiple modalities complementary to promote multimodal emotion recognition. Experiments on two datasets, namely SAVEE and eNTERFACE‘05, are conducted to evaluate the accuracy of the proposed method. The results show that the proposed method produces good recognition rates that are higher than the ones produced by the methods without <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -means clustering; more specifically, they are 2.77% higher in SAVEE and 4.7% higher in eNTERFACE‘05.

State Estimation for Closed-Loop LPV System Identification via Kernel Methods⋆

This paper proposes a state-estimation method for closed-loop identification of linear-parameter-varying (LPV) systems by extending an approach for linear-time-invariant (LTI) systems and applying kernel canonical correlation analysis (KCCA). The proposed method estimates the state term from the one-step-ahead prediction via a kernel approach. The incomplete Cholesky decomposition (ICD) is introduced to reduce the complexity of the KCCA. A simple numerical simulation shows the effectiveness of the proposed approach.

Radio frequency fingerprinting feature fusion based on multi-domain discriminant kernel canonical correlation analysis

Specific emitter identification (SEI) refers to the precise identification of a specific transmitter through the extraction of hardware fingerprint information from the received signal. The majority of related research focuses on the definition and extraction of new fingerprint features, with less emphasis on the comprehensive use of existing features. Because the characteristics of different analysis domains are complementary to the description of the radio frequency fingerprint, this paper proposes a multi-domain feature fusion strategy for SEI based on multi-domain discriminant kernel canonical correlation analysis (MDKCCA), which fully exploits the complementarity between the features of different domains and combines feature tag information. MDKCCA affords multi-domain feature dimensionality reduction and fusion in a high-dimensional space.The algorithm's performance is validated on four different types of real-world data sets using eight common fingerprint features in four feature analysis domains. The results show that this method eliminates the need for manual feature optimization and can significantly reduce the dimensionality of fusion features. The recognition rate on various targets exceeds 95%, which is higher than the best single feature. It is also superior to the simple feature synthesis method based on direct cascade or PCA dimensionality reduction transformation, the neural network-based feature synthesis method, and the feature fusion method based on discriminant canonical correlation methods.

A kernel canonical correlation analysis approach for removing environmental and operational variations for structural damage identification

Vibration-based damage detection relies on the observation of changes in damage-sensitive dynamic features. However, a major problem is that dynamic features are sensitive not only to structural damage but also to environmental and operational variations (EOVs), such as temperature, humidity, and operational loading. In addition, the influence of EOVs on damage-sensitive features is often nonlinear, which limits the application of many linear methods in the removal of environmental effects. To remove the nonlinear effects of EOVs on dynamic features, an improved method based on kernel canonical correlation analysis (KCCA) is proposed in this study. Using this method, the monitored data were divided into two groups. The two sets of data were then mapped into a higher-dimensional space through the kernel trick to determine their implicit linear relationship. Subsequently, two variables that share the co-occurrence information of EOV effects were computed using canonical correlation analysis (CCA), and a stationary residual insensitive to EOVs was obtained. Furthermore, the proposed approach was examined using a simulated 7-DOF example and then applied to real monitored data from the Z24 bridge, demonstrating that nonlinear EOV effects can be successfully removed and damage can be accurately identified.

Research on a Machine Learning-Based Method for Assessing the Safety State of Historic Buildings.

Historic and protected buildings are increasingly valued due to their valuable historical and cultural value. The assessment of the safety state of historic buildings has received more attention. Emerging machine learning algorithms, with their excellent computational performance, provide new ideas and new means to solve practical problems in various fields. Therefore, this paper proposes a method for assessing the safety state of historic buildings based on machine learning techniques. Firstly, based on the analysis of the characteristics of historical buildings and common security problems, the application of wireless sensor networks to the security monitoring of historical buildings is proposed in order to improve the automation of monitoring. Then, in order to improve the accuracy of the assessment, a combination of kernel canonical correlation analysis (KCCA) and support vector machine (SVM) is used to establish the security monitoring model. The experimental results show that by choosing a suitable KCCA function, the redundant features of the data can be reduced while the comprehensiveness of the building structure identification features can be retained, thus effectively improving the prediction accuracy of the SVM. The KCCA-SVM model can accurately predict the physical quantities such as relative structural displacement of historical buildings with good reliability.