Canonical Correlation Analysis Framework Research Articles

Functional generalized canonical correlation analysis for studying multiple longitudinal variables.

In this paper, we introduce functional generalized canonical correlation analysis, a new framework for exploring associations between multiple random processes observed jointly. The framework is based on the multiblock regularized generalized canonical correlation analysis framework. It is robust to sparsely and irregularly observed data, making it applicable in many settings. We establish the monotonic property of the solving procedure and introduce a Bayesian approach for estimating canonical components. We propose an extension of the framework that allows the integration of a univariate or multivariate response into the analysis, paving the way for predictive applications. We evaluate the method's efficiency in simulation studies and present a use case on a longitudinal dataset.

Multi-scale superpixel-based nearest subspace classifier for mucilage detection from hyperspectral PRISMA data

Hyperspectral imaging has a critical role in observing the Earth's surface, providing images with rich spectral information. As a result, it has become an essential tool for monitoring and addressing environmental issues such as pollution, water balance, and climate change. Among these challenges, marine pollution poses a significant ecological threat, as evidenced by the recent mucilage incident in Turkey in April 2021. Mucilage, a dense mucus-like substance resulting from the overgrowth of marine plankton and microorganisms, detrimentally affects marine ecosystems when its levels surge. Traditional mucilage detection involves labour-intensive sampling and laboratory analysis, rendering it impractical for vast marine environments. Remote sensing techniques offer a promising alternative for automated mucilage detection from satellite imagery. In this study, we propose a multi-scale superpixel-based classification approach for mucilage detection, using the PRISMA satellite's hyperspectral mucilage datasets captured over the Sea of Marmara. Our method performs classification by analyzing the closeness between the subspace spanned by the samples in the superpixel and the subspace spanned by the within-class training samples. We formulate the solution to this approach via a Canonical Correlation Analysis framework and show that our proposed method outperforms its competitors, especially when the training data is limited. Our comprehensive experiments include assessing the transferability of classification methods on datasets collected from different locations and at different times. We compared the performance of our proposed method with state-of-the-art spectral-spatial classification methods, including deep learning-based random patches network (RPNet), structural-kernel collaborative representation (SKCR), and window based nearest subspace classifier (WBNSC). The results show that the proposed method outperforms existing techniques in most cases. The datasets and source code are publicly available on GitHub at https://github.com/kgtoker/MSBNSC, enabling more comprehensive access and collaboration in environmental monitoring efforts.

Rebooting kernel CCA method for nonlinear quality-relevant fault detection in process industries

Process monitoring is essential and important strategy for ensuring process safety and product quality. However, due to the nonlinear characteristics and multiple working conditions in process industries, the traditional process monitoring method cannot be effectively applied. Therefore, we propose a novel process monitoring framework, termed as mixture enhanced kernel canonical correlation analysis framework (M-NAKCCA). The innovations and advantages of M-NAKCCA are as follows: 1). The traditional CCA method is re-boosted as a new method, M-NAKCCA, to better nonlinear fault detection. Also, a matter-element model (MEm) is assimilated into M-NAKCCA to make the information more refined. 2). To overcome the curse of dimensionality that usually occurs in the high-dimensional dataset, M-NAKCCA uses the Nyström approximation technology to compress the kernel matrix. Moreover, the T2 control chart is reconstructed and the corresponding control upper limit is re-configured to improve the method sensitivity and to better the fault detection performance. 3). The proposed M-NAKCCA framework is firstly used to monitor a wastewater treatment plant (WWTP) and chemical plant with diverse process behaviors. The experimental results showed that the M-NAKCCA framework achieved the best performance for both of case studies.

Hyper-graph based sparse canonical correlation analysis for the diagnosis of Alzheimer’s disease from multi-dimensional genomic data

The effective and accurate diagnosis of Alzheimer's disease (AD), especially in the early stage (i.e., mild cognitive impairment (MCI)) remains a big challenge in AD research. So far, multiple biomarkers have been associated with AD diagnosis and progression. However, most of the existing research only utilized single modality data for diagnostic biomarker identification, which did not take the advantages of multi-modal data that provide comprehensive and complementary information at multiple levels into consideration. In this paper, we integrate multi-modal genomic data from postmortem AD brains (i.e., mRNA, miRNA and epigenomic data) and propose a hyper-graph based sparse canonical correlation analysis (HGSCCA) method to extract the most correlated multi-modal biomarkers associated with AD and MCI. Specifically, our model utilizes the sparse canonical correlation analysis framework (SCCA), which aims at finding the best linear projections for each input modality so that the strongest correlation within the selected features of multi-dimensional genomic data can be captured. In addition, with the consideration of high-order relationships among different subjects, we also introduce a hyper-graph-based regularization term that will lead to the selection of more discriminative biomarkers. To evaluate the effectiveness of the proposed method, we conduct the experiments on the well-known AD cohort study, The Religious Orders Study and Memory and Aging Project (ROSMAP) dataset, and the results show that our method can not only identify meaningful biomarkers for the diagnosis AD disease, but also achieve superior classification performance than the comparing methods.

Integrative Analysis of Pathological Images and Multi-Dimensional Genomic Data for Early-Stage Cancer Prognosis.

The integrative analysis of histopathological images and genomic data has received increasing attention for studying the complex mechanisms of driving cancers. However, most image-genomic studies have been restricted to combining histopathological images with the single modality of genomic data (e.g., mRNA transcription or genetic mutation), and thus neglect the fact that the molecular architecture of cancer is manifested at multiple levels, including genetic, epigenetic, transcriptional, and post-transcriptional events. To address this issue, we propose a novel ordinal multi-modal feature selection (OMMFS) framework that can simultaneously identify important features from both pathological images and multi-modal genomic data (i.e., mRNA transcription, copy number variation, and DNA methylation data) for the prognosis of cancer patients. Our model is based on a generalized sparse canonical correlation analysis framework, by which we also take advantage of the ordinal survival information among different patients for survival outcome prediction. We evaluate our method on three early-stage cancer datasets derived from The Cancer Genome Atlas (TCGA) project, and the experimental results demonstrated that both the selected image and multi-modal genomic markers are strongly correlated with survival enabling effective stratification of patients with distinct survival than the comparing methods, which is often difficult for early-stage cancer patients.

Interest Level Estimation Based on Tensor Completion via Feature Integration for Partially Paired User’s Behavior and Videos

A novel method for interest level estimation based on tensor completion via feature integration for partially paired users' behavior and videos is presented in this paper. The proposed method defines a novel canonical correlation analysis (CCA) framework that is suitable for interest level estimation, which is a hybrid version of semi-supervised CCA (SemiCCA) and supervised locality preserving CCA (SLPCCA) called semi-supervised locality preserving CCA (S2LPCCA). For partially paired users' behavior and videos in actual shops and on the Internet, new integrated features that maximize the correlation between partially paired samples by the principal component analysis (PCA)-mixed CCA framework are calculated. Then videos that users have not watched can be used for the estimation of users' interest levels. Furthermore, local structures of partially paired samples in the same class are preserved for accurate estimation of interest levels. Tensor completion, which can be applied to three contexts, videos, users and “canonical features and interest levels,” is used for estimation of interest levels. Consequently, the proposed method realizes accurate estimation of users' interest levels based on S2LPCCA and the tensor completion from partially paired training features of users' behavior and videos. Experimental results obtained by applying the proposed method to actual data show the effectiveness of the proposed method.

Constrained ERM Learning of Canonical Correlation Analysis: A Least Squares Perspective.

Canonical correlation analysis (CCA) is a useful tool in detecting the latent relationship between two sets of multivariate variables. In theoretical analysis of CCA, a regularization technique is utilized to investigate the consistency of its analysis. This letter addresses the consistency property of CCA from a least squares view. We construct a constrained empirical risk minimization framework of CCA and apply a two-stage randomized Kaczmarz method to solve it. In the first stage, we remove the noise, and in the second stage, we compute the canonical weight vectors. Rigorous theoretical consistency is addressed. The statistical consistency of this novel scenario is extended to the kernel version of it. Moreover, experiments on both synthetic and real-world data sets demonstrate the effectiveness and efficiency of the proposed algorithms.

Robust and sparse canonical correlation analysis based L 2, p ‐norm

The objective function of canonical correlation analysis (CCA) is equivalent to minimising an L 2-norm distance of the paired data. Owing to the characteristic of L 2-norm, CCA is highly sensitive to noise and irrelevant features. To alleviate such problem, this study incorporates robust feature extraction and group sparse feature selection into the framework of CCA, and proposes a feature fusion method named robust and sparse CCA (RSCCA). In RSCCA, L 2,p -norm is adopted as the distance measurement of paired data, which can alleviate the effect of noise and irrelevant features and achieve robust performance. The experimental results show that our method outperforms CCA and its variants for feature fusion.

A Novel Generalized Fuzzy Canonical Correlation Analysis Framework for Feature Fusion and Recognition

In this paper, a novel CCA-based dimensionality reduction method called generalized fuzzy canonical correlation analysis (GFCCA) is proposed. GFCCA combines the generalized canonical correlation analysis and fuzzy set theory. GFCCA redefines the fuzzy between-class and within-class scatter matrices that relate directly to the samples distribution information. For nonlinear separated problems, we extend the kernel extension of GFCCA with positive definite kernels and indefinite kernels. Experiments on real-world data sets are performed to test and evaluate the effectiveness of the proposed algorithms.

Graphlet-based Characterization of Directed Networks.

We are flooded with large-scale, dynamic, directed, networked data. Analyses requiring exact comparisons between networks are computationally intractable, so new methodologies are sought. To analyse directed networks, we extend graphlets (small induced sub-graphs) and their degrees to directed data. Using these directed graphlets, we generalise state-of-the-art network distance measures (RGF, GDDA and GCD) to directed networks and show their superiority for comparing directed networks. Also, we extend the canonical correlation analysis framework that enables uncovering the relationships between the wiring patterns around nodes in a directed network and their expert annotations. On directed World Trade Networks (WTNs), our methodology allows uncovering the core-broker-periphery structure of the WTN, predicting the economic attributes of a country, such as its gross domestic product, from its wiring patterns in the WTN for up-to ten years in the future. It does so by enabling us to track the dynamics of a country’s positioning in the WTN over years. On directed metabolic networks, our framework yields insights into preservation of enzyme function from the network wiring patterns rather than from sequence data. Overall, our methodology enables advanced analyses of directed networked data from any area of science, allowing domain-specific interpretation of a directed network’s topology.

Cost Sensitive Semi-Supervised Canonical Correlation Analysis for Multi-view Dimensionality Reduction

To deal with the cost sensitive and semi-supervised learning problems in Multi-view Dimensionality Reduction (MDR), we propose a Cost Sensitive Semi-Supervised Canonical Correlation Analysis $$(\hbox {CS}^{3}\hbox {CCA}). \hbox {CS}^{3}\hbox {CCA}$$(CS3CCA).CS3CCA first uses the $$L_2$$L2 norm approach to obtain the soft label for each unlabeled data, and then embed the misclassification cost into the framework of Canonical Correlation Analysis (CCA). Compared with existing CCA based methods, $$\hbox {CS}^{3}\hbox {CCA}$$CS3CCA has the following advantages: (1) It uses the $$L_2$$L2 norm approach to infer the soft label for unlabeled data, which is computationally efficient and effective, especially for cost sensitive face recognition. (2) The objective function of $$\hbox {CS}^{3}\hbox {CCA}$$CS3CCA not only maximizes the soft cost sensitive within-class correlations and minimizes the soft cost sensitive between-class correlations in the inter-view, but also considers the class imbalance problem simultaneously. With the discriminant projections learned by $$\hbox {CS}^{3}\hbox {CCA}$$CS3CCA, we employ it for cost sensitive face recognition. The experimental results on four well-known face data sets, including AR, Extended Yale B, PIE and ORL, demonstrate the effectiveness of $$\hbox {CS}^{3}\hbox {CCA}$$CS3CCA.

Online Distributed Sparsity-Aware Canonical Correlation Analysis

The problem of identifying informative sensors that acquire measurements about multiple sources and clustering them according to their source content is considered. Toward this end, a novel canonical correlation analysis (CCA) framework equipped with sparsity-inducing norm-one regularization is introduced to identify correlated sensor measurements and identify informative groups of sensors. It is established that the novel framework is capable to cluster sensors, based on their source content, correctly (with probability one) even in nonlinear settings and when sources do not overlap. Block coordinate techniques are employed to derive a centralized algorithm that minimizes the sparsity-aware CCA framework. The latter framework is reformulated as a separable optimization program which is tackled in a distributed fashion via the alternating direction method of multipliers. A computationally efficient online distributed algorithm is further derived that is capable to process sensor data online. Extensive numerical tests corroborate that the novel techniques outperform existing alternatives.

Distributed information-based clustering of heterogeneous sensor data

Heterogeneous sensing systems, consisting of sensors with different sensing capabilities, offer flexibility and provide multiple views of the sensed field by acquiring different types of measurements. The acquired sensor measurements are affected by different and unknown in number phenomena/sources of interest. To this end, a novel canonical correlation analysis (CCA) framework equipped with norm-one and norm-two regularization terms is designed to cluster the sensor data based on their information content. Block coordinate descent (BCD) is combined with the alternating direction method of multipliers (ADMM) framework to derive a centralized algorithm tackling the novel regularized CCA framework. Further, splitting of the regularized CCA into localized minimization subtasks across sensors enables distributed clustering of heterogeneous data based on their information content. Numerical tests demonstrate that the novel framework can achieve higher probability of correct clustering than existing alternatives.

A unified multiset canonical correlation analysis framework based on graph embedding for multiple feature extraction

Abstract Multiset canonical correlation analysis (MCCA) can simultaneously reduce the dimensionality of multimodal data. Thus, MCCA is very much suitable and powerful for multiple feature extraction. However, most existing MCCA-related methods are unsupervised algorithms, which are not very effective for pattern classification tasks. In order to improve discriminative power for handling multimodal data, we, in this paper, propose a unified multiset canonical correlation analysis framework based on graph embedding for dimensionality reduction (GbMCC-DR). Under GbMCC-DR framework, three novel supervised multiple feature extraction methods, i.e., GbMCC-LDA, GbMCC-LDE, and GbMCC-MFA are presented by incorporating several well-known graphs. These three methods consider not only geometric structure of multimodal data but also separability of different classes. Moreover, theoretical analysis further shows that, in some specific circumstances, several existing MCCA-related algorithms can be unified into GbMCC-DR framework. Therefore, this proposed framework has good expansibility and generalization. The experimental results on both synthetic data and several popular real-world datasets demonstrate that three proposed algorithms achieve better recognition performance than existing related algorithms, which is also the evidence for effectiveness of GbMCC-DR framework.

An Efficient Optimization Algorithm for Structured Sparse CCA, with Applications to eQTL Mapping

In this paper we develop an efficient optimization algorithm for solving canonical correlation analysis (CCA) with complex structured-sparsity-inducing penalties, including overlapping-group-lasso penalty and network-based fusion penalty. We apply the proposed algorithm to an important genome-wide association study problem, eQTL mapping. We show that, with the efficient optimization algorithm, one can easily incorporate rich structural information among genes into the sparse CCA framework, which improves the interpretability of the results obtained. Our optimization algorithm is based on a general excessive gap optimization framework and can scale up to millions of variables. We demonstrate the effectiveness of our algorithm on both simulated and real eQTL datasets.

A Novel Approach for Target Detection and Classification Using Canonical Correlation Analysis

We present a novel detection approach, detection with canonical correlation (DCC), for target detection without prior information on the interference. We use the maximum canonical correlations between the target set and the observation data set as the detection statistic, and the coefficients of the canonical vector are used to determine the indices of components from a given target library, thus enabling both detection and classification of the target components that might be present in the mixture. We derive an approximate distribution of the maximum canonical correlation when targets are present. For applications where the contributions of components are non-negative, non-negativity constraints are incorporated into the canonical correlation analysis framework and a recursive algorithm is derived to obtain the solution. We demonstrate the effectiveness of DCC and its nonnegative variant by applying them on detection of surface-deposited chemical agents in Raman spectroscopy.

A novel multiset integrated canonical correlation analysis framework and its application in feature fusion

Multiset canonical correlation analysis (MCCA) is difficult to effectively express the integrated correlation among multiple feature vectors in feature fusion. Thus, this paper firstly presents a novel multiset integrated canonical correlation analysis (MICCA) framework. The MICCA establishes a discriminant correlation criterion function of multi-group variables based on generalized correlation coefficient. The criterion function can clearly depict the integrated correlation among multiple feature vectors. Then the paper presents a multiple feature fusion theory and algorithm using the MICCA method. The detailed process of the algorithm is as follows: firstly, extract multiple feature vectors from the same patterns by using different feature extraction methods; then extract multiset integrated canonical correlation features using MICCA; finally form effective discriminant feature vectors through two given feature fusion strategies for pattern classification. The multi-group feature fusion method based on MICCA not only achieves the aim of feature fusion, but also removes the redundancy between features. The experiment results on CENPARMI handwritten Arabic numerals and UCI multiple features database show that the MICCA method has better recognition rates and robustness than the fusion methods based on canonical correlation analysis (CCA) and MCCA.

Canonical Correlation Feature Selection for Sensors With Overlapping Bands: Theory and Application

The main focus of this paper is a rigorous development and validation of a novel canonical correlation feature- selection (CCFS) algorithm that is particularly well suited for spectral sensors with overlapping and noisy bands. The proposed approach combines a generalized canonical correlation analysis framework and a minimum mean-square-error criterion for the selection of feature subspaces. The latter induces ranking of the best linear combinations of the noisy overlapping bands and, in doing so, guarantees a minimal generalized distance between the centers of classes and their respective reconstructions in the space spanned by sensor bands. To demonstrate the efficacy and the scope of the proposed approach, two different applications are considered. The first one is separability and classification analysis of rock species using laboratory spectral data and a quantum-dot infrared photodetector (QDIP) sensor. The second application deals with supervised classification and spectral unmixing, and abundance estimation of hyperspectral imagery obtained from the Airborne Hyperspectral Imager sensor. Since QDIP bands exhibit significant spectral overlap, the first study validates the new algorithm in this important application context. The results demonstrate that proper postprocessing can facilitate the emergence of QDIP-based sensors as a promising technology for midwave- and longwave-infrared remote sensing and spectral imaging. In particular, the proposed CCFS algorithm makes it possible to exploit the unique advantage offered by QDIPs with a dot-in-a-well configuration, comprising their bias-dependent spectral response, which is attributable to the quantum Stark effect. The main objective of the second study is to assert that the scope of the new CCFS approach also extends to more traditional spectral sensors.

Distributed Estimation Using Reduced-Dimensionality Sensor Observations

We derive linear estimators of stationary random signals based on reduced-dimensionality observations collected at distributed sensors and communicated to a fusion center over wireless links. Dimensionality reduction compresses sensor data to meet low-power and bandwidth constraints, while linearity in compression and estimation are well motivated by the limited computing capabilities wireless sensor networks are envisioned to operate with, and by the desire to estimate random signals from observations with unknown probability density functions. In the absence of fading and fusion center noise (ideal links), we cast this intertwined compression-estimation problem in a canonical correlation analysis framework and derive closed-form mean-square error (MSE) optimal estimators along with coordinate descent suboptimal alternatives that guarantee convergence at least to a stationary point. Likewise, we develop estimators based on reduced-dimensionality sensor observations in the presence of fading and additive noise at the fusion center (nonideal links). Performance analysis and corroborating simulations demonstrate the merits of the novel distributed estimators relative to existing alternatives.

On rotational invariance in adaptive spatial filtering of fMRI data

Canonical correlation analysis (CCA) has previously been shown to work well for detecting neural activity in fMRI data. The reason is that CCA enables simultaneous temporal modeling and adaptive spatial filtering of the data. This article introduces a novel method for adaptive anisotropic filtering using the CCA framework and compares it to a previously proposed method. Isotropic adaptive filtering, which is only able to form isotropic filters of different sizes, is also presented and evaluated. It is shown that a new feature of the proposed method is invariance to the orientation of activated regions, and that the detection performance is superior to both that of the previous method and to isotropic filtering.