Discriminative Canonical Correlation Analysis Research Articles

Multi-Task Learning and Sparse Discriminant Canonical Correlation Analysis for Identification of Diagnosis-Specific Genotype-Phenotype Association.

The primary objective of imaging genetics research is to investigate the complex genotype-phenotype association for the disease under study. For example, to understand the impact of genetic variations over the brain functions and structure, the genotypic data such as single nucleotide polymorphism (SNP) is integrated with the phenotypic data such as imaging quantitative traits. The sparse models, based on canonical correlation analysis (CCA), are popular in this area to find the complex bi-multivariate genotype-phenotype association, as the number of features in genotypic and/or phenotypic data is significantly higher as compared to the number of samples. However, the sparse CCA based methods are, in general, unsupervised in nature, and fail to identify the diagnose-specific features those play an important role for the diagnosis and prognosis of the disease under study. In this regard, a new supervised model is proposed to study the complex genotype-phenotype association, by judiciously integrating the merits of CCA, linear discriminant analysis (LDA) and multi-task learning. The proposed model can identify the diagnose-specific as well as the diagnose-consistent features with significantly lower computational complexity. The performance of the proposed method, along with a comparison with the state-of-the-art methods, is evaluated on several synthetic data sets and one real imaging genetics data collected from Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. In the current study, the SNP as genetic data and resting state functional MRI ( fMRI) as imaging data are integrated to find the complex genotype-phenotype association. An important finding is that the proposed method has better correlation value, improved noise resistance and stability, and also has better feature selection ability. All the results illustrate the power and capability of the proposed method to find the diagnostic group-specific imaging genetic association, which may help to understand the neurodegenerative disorder in a more comprehensive way.

Multiview Regularized Discriminant Canonical Correlation Analysis: Sequential Extraction of Relevant Features From Multiblock Data.

One of the important issues associated with real-life high-dimensional data analysis is how to extract significant and relevant features from multiview data. The multiset canonical correlation analysis (MCCA) is a well-known statistical method for multiview data integration. It finds a linear subspace that maximizes the correlations among different views. However, the existing methods to find the multiset canonical variables are computationally very expensive, which restricts the application of the MCCA in real-life big data analysis. The covariance matrix of each high-dimensional view may also suffer from the singularity problem due to the limited number of samples. Moreover, the MCCA-based existing feature extraction algorithms are, in general, unsupervised in nature. In this regard, a new supervised feature extraction algorithm is proposed, which integrates multimodal multidimensional data sets by solving maximal correlation problem of the MCCA. A new block matrix representation is introduced to reduce the computational complexity for computing the canonical variables of the MCCA. The analytical formulation enables efficient computation of the multiset canonical variables under supervised ridge regression optimization technique. It deals with the "curse of dimensionality" problem associated with high-dimensional data and facilitates the sequential generation of relevant features with significantly lower computational cost. The effectiveness of the proposed multiblock data integration algorithm, along with a comparison with other existing methods, is demonstrated on several benchmark and real-life cancer data.

Creating Universum for class imbalance via locality and its application in multiview subspace learning

Since the existing methods for generating Universum fail in class imbalance cases, we propose a new method for creating Universum via locality and betweenness, named L-CIBU. Compared to existing methods for generating Universum, L-CIBU not only generates appropriate Universum in the case of class imbalance but also has higher efficiency. Given the existence of class imbalance in multiview learning, we introduce Universum to multiview subspace learning (MvSL) to further enhance its performance and L-CIBU happens to be able to overcome this problem. As far as we know, this is the first time to introduce Universum into MvSL with more than two views. Specifically, we introduce Universum created by L-CIBU into multiview discriminant analysis (MvDA) and propose Universum MvDA (UMvDA). By analyzing several randomly generated imbalanced datasets, it is intuitively demonstrated that L-CIBU is more effective than FIBU and CIBU in the case of class imbalance. The experimental results on two real datasets show that the time required for creating the Universum using L-CIBU is 25%∼95% less than that of FIBU and CIBU. Moreover, UMvDA achieves an accuracy improvement of approximately 1%∼6% compared to MvDA, Universum canonical correlation analysis (UCCA), and Universum discriminant canonical correlation analysis (UDCCA).

Quantum discriminative canonical correlation analysis

Quantum discriminative canonical correlation analysis

Multi-user motion recognition using sEMG via discriminative canonical correlation analysis and adaptive dimensionality reduction

The inability of new users to adapt quickly to the surface electromyography (sEMG) interface has greatly hindered the development of sEMG in the field of rehabilitation. This is due mainly to the large differences in sEMG signals produced by muscles when different people perform the same motion. To address this issue, a multi-user sEMG framework is proposed, using discriminative canonical correlation analysis and adaptive dimensionality reduction (ADR). The interface projects the feature sets for training users and new users into a low-dimensional uniform style space, overcoming the problem of individual differences in sEMG. The ADR method removes the redundant information in sEMG features and improves the accuracy of system motion recognition. The presented framework was validated on eight subjects with intact limbs, with an average recognition accuracy of 92.23% in 12 categories of upper-limb movements. In rehabilitation laboratory experiments, the average recognition rate reached 90.52%. The experimental results suggest that the framework offers a good solution to enable new rehabilitation users to adapt quickly to the sEMG interface.

Sparse additive discriminant canonical correlation analysis for multiple features fusion

Canonical correlation analysis (CCA) is an unsupervised representation learning technique to correlate multi-view data by learning a set of projection matrices. Being complementary with CCA, many discriminant methods are proposed to extract discriminative features of multi-view data by introducing the supervised class information. However, the learned projection matrices in these methods are mathematically constrained to be equal rank to the class number, and thus cannot represent the original data comprehensively. In this paper, we propose a general multi-view information fusion technique, named sparse additive discriminative canonical correlation analysis (SaDCCA). On one hand, SaDCCA is equipped with a strong degree of discrimination by defining a new affinity matrix that reflects the high-order characteristics of intra-class and the separability of inter-class. On the other hand, SaDCCA can exploit the correlation among multi-view data by maintaining the spirit of CCA. The discrimination among classes and the correlation among views are integrated in an additive manner. To obtain the sparse solutions, we first establish the relationship between the objective function and the underdetermined linear system equations, and then obtain the ℓ1-norm solution by accelerated Bregman iteration with matrix form. SaDCCA has no rank constraint on the projection matrices and is capable to provide accurate recognition performance. Experiments conducted on some publicly available datasets demonstrate the effectiveness of the proposed approach.

A Survey on Canonical Correlation Analysis

In recent years, the advances in data collection and statistical analysis promotes canonical correlation analysis (CCA) available for more advanced research. CCA is the main technique for two-set data dimensionality reduction such that the correlation between the pairwise variables in the common subspace is mutually maximized. Over 80-years of developments, a number of CCA models have been proposed according to different machine learning mechanisms. However, the field lacks an insightful review for the state-of-art developments. This survey targets to provide a well-organized overview for CCA and its extensions. Specifically, we first review the CCA theory from the perspective of both model formation and model optimization. The association between two popular solution methods, i.e., eigen value decomposition (EVD) and singular value decomposition (SVD), are discussed. Following that, we present a taxonomy of current progresses and classify them into seven groups: 1) multi-view CCA, 2) probabilistic CCA, 3) deep CCA, 4) kernel CCA, 5) discriminative CCA, 6) sparse CCA and 7) locality preserving CCA. For each group, we demonstrate two or three representative mathematical models, identifying their strengths and limitations. We summarize the representative applications and numerical results of these seven groups in real-world practices, collecting the data sets and open-sources for implementation. In the end, we provide several promising future research directions that can improve the current state of the art.

Automated Atrial Fibrillation Detection Based on Feature Fusion Using Discriminant Canonical Correlation Analysis

Atrial fibrillation (AF) is one of the most common cardiovascular diseases, with a high disability rate and mortality rate. The early detection and treatment of atrial fibrillation have great clinical significance. In this paper, a multiple feature fusion is proposed to screen out AF recordings from single lead short electrocardiogram (ECG) recordings. The proposed method uses discriminant canonical correlation analysis (DCCA) feature fusion. It fully takes intraclass correlation and interclass correlation into consideration and solves the problem of computation and information redundancy with simple series or parallel feature fusion. The DCCA integrates traditional features extracted by expert knowledge and deep learning features extracted by the residual network and gated recurrent unit network to improve the low accuracy of a single feature. Based on the Cardiology Challenge 2017 dataset, the experiments are designed to verify the effectiveness of the proposed algorithm. In the experiments, the F1 index can reach 88%. The accuracy, sensitivity, and specificity are 91.7%, 90.4%, and 93.2%, respectively.

Generalized canonical correlation analysis for labeled data

Multi-view learning is a method that is used to extract standard features from different information sources in various fields such as medical data analysis, computer vision, and web data analysis. Canonical correlation analysis (CCA), a dimensionality reduction method used for the data acquired in multi-view learning, can extract a low-dimensional space where the correlation between two multivariate data is high. However, CCA has the following problems. First, it is difficult to perform dimensionality reduction while taking advantage of the label information attached to the data. Second, CCA is an analysis method for two sets of data; it cannot be directly applied when we have three or more datasets. Discriminative canonical correlation analysis (DCCA) can be used to solve the first problem. It enables the dimensionality reduction of two datasets while reflecting the label information. Further, generalized canonical correlation analysis (GCCA) can be used to solve the second problem. It calculates canonical correlation variables, which are the products of parameters and data, for three or more datasets, and assumes that all datasets aggregate to a piece of shared information so that the parameters aggregate the information of each data. However, DCCA and GCCA do not simultaneously solve the problems of CCA. Therefore, in this study, we extend DCCA and propose a dimensionality reduction method using the label information for three or more datasets. We validate the usefulness of the proposed method through a simulation study.

A Dynamic Discriminative Canonical Correlation Analysis via Adaptive Weight Scheme

In multi-view learning, massive literature is devoted to exploring the intrinsic structure between cross-views. It is well known that canonical correlation analysis (CCA) is a conventional multi-view learning method, which considers the correlation between two views. However, it fails to utilize class information and is difficult to suit different issues to extract discriminative features. In this paper, we propose a novel cross-view discriminative feature learning method called dynamic discriminative canonical correlation analysis, which captures class information to yield discriminative features. More specifically, we develop an adaptive weight scheme of cross-view within-class and between-class scatters to make full use of distribution class information. In addition, an iterative algorithm with Cauchy inequalities and the Lagrange multiplier is proposed to handle the non-smooth objective function. Our method is applied to face recognition and multi-linguistic text classification tasks. Extensive experimental results reveal that the adaptive weight scheme plays a beneficial role and our method is an effective feature learning.

An integrated omics approach to investigate summer mortality of New Zealand Greenshell™ mussels.

Green-lipped mussels, commercially known as Greenshell™ mussels (Perna canaliculus Gmelin 1791), contribute > $300 million to New Zealand's aquaculture exports. However, mortalities during summer months and potential pathogenic outbreaks threaten the industry. Thermal stress mechanisms and immunological responses to pathogen infections need to be understood to develop health assessment strategies and early warning systems. P. canaliculus were collected during a mortality event at a commercial aquaculture farm in Firth of Thames, New Zealand. Gill tissues from six healthy and six unhealthy mussels were excised and processed for metabolomic (GC-MS) and label-free proteomic (LC-MS) profiling. Univariate analyses were conducted separately on each data layer, with data being integrated via sparse multiple discriminative canonical correlation analysis. Pathway enrichment analysis was used to probe coordinated changes in functionally related metabolite sets. Findings revealed disruptions of the tricarboxylic acid (TCA) cycle and fatty acid metabolism in unhealthy mussels. Metabolomics analyses also indicated oxidative stress in unhealthy mussels. Proteomics analyses identified under-expression of proteins associated with cytoskeleton structure and regulation of cilia/flagellum in gill tissues of unhealthy mussels. Integrated omics revealed a positive correlation between Annexin A4 and CCDC 150 and saturated fatty acids, as well as a negative correlation between 2-aminoadipic acid and multiple cytoskeletal proteins. Our study demonstrates the ability of using integrative omics to reveal metabolic perturbations and protein structural changes in the gill tissues of stressed P. canaliculus and provides new insight into metabolite and protein interactions associated with incidences of summer mortality in this species.

Moment-Guided Discriminative Manifold Correlation Learning on Ordinal Data

Canonical correlation analysis (CCA) is a typical and useful learning paradigm in big data analysis for capturing correlation across multiple views of the same objects. When dealing with data with additional ordinal information, traditional CCA suffers from poor performance due to ignoring the ordinal relationships within the data. Such data is becoming increasingly common, as either temporal or sequential information is often associated with the data collection process. To incorporate the ordinal information into the objective function of CCA, the so-called ordinal discriminative CCA has been presented in the literature. Although ordinal discriminative CCA can yield better ordinal regression results, its performance deteriorates when data is corrupted with noise and outliers, as it tends to smear the order information contained in class centers. To address this issue, in this article we construct a robust manifold-preserved ordinal discriminative correlation regression (rmODCR). The robustness is achieved by replacing the traditional ( l 2 -norm) class centers with l p -norm centers, where p is efficiently estimated according to the moments of the data distributions, as well as by incorporating the manifold distribution information of the data in the objective optimization. In addition, we further extend the robust manifold-preserved ordinal discriminative correlation regression to deep convolutional architectures. Extensive experimental evaluations have demonstrated the superiority of the proposed methods.

Discriminative Robust Head-Pose and Gaze Estimation Using Kernel-DMCCA Features Fusion

There remain outstanding challenges for improving accuracy of multi-feature information for head-pose and gaze estimation. The proposed framework employs discriminative analysis for head-pose and gaze estimation using kernel discriminative multiple canonical correlation analysis (K-DMCCA). The feature extraction component of the framework includes spatial indexing, statistical and geometrical elements. Head-pose and gaze estimation is constructed by feature aggregation and transforming features into a higher dimensional space using K-DMCCA for accurate estimation. The two main contributions are: Enhancing fusion performance through the use of kernel-based DMCCA, and by introducing an improved iris region descriptor based on quadtree. The overall approach is also inclusive of statistical and geometrical indexing that are calibration free (does not require any subsequent adjustment). We validate the robustness of the proposed framework across a wide variety of datasets, which consist of different modalities (RGB and Depth), constraints (wide range of head-poses, not only frontal), quality (accurately labelled for validation), occlusion (due to glasses, hair bang, facial hair) and illumination. Our method achieved an accurate head-pose and gaze estimation of 4.8∘ using Cave, 4.6∘ using MPII, 5.1∘ using ACS, 5.9∘ using EYEDIAP, 4.3∘ using OSLO and 4.6∘ using UULM datasets.

Multi-omics Data Integration for Identifying Osteoporosis Biomarkers and Their Biological Interaction and Causal Mechanisms.

SummaryOsteoporosis is characterized by low bone mineral density (BMD). The advancement of high-throughput technologies and integrative approaches provided an opportunity for deciphering the mechanisms underlying osteoporosis. Here, we generated genomic, transcriptomic, methylomic, and metabolomic datasets from 119 subjects with high (n = 61) and low (n = 58) BMDs. By adopting sparse multiple discriminative canonical correlation analysis, we identified an optimal multi-omics biomarker panel with 74 differentially expressed genes (DEGs), 75 differentially methylated CpG sites (DMCs), and 23 differential metabolic products (DMPs). By linking genetic data, we identified 199 targeted BMD-associated expression/methylation/metabolite quantitative trait loci (eQTLs/meQTLs/metaQTLs). The reconstructed networks/pathways showed extensive biomarker interactions, and a substantial proportion of these biomarkers were enriched in RANK/RANKL, MAPK/TGF-β, and WNT/β-catenin pathways and G-protein-coupled receptor, GTP-binding/GTPase, telomere/mitochondrial activities that are essential for bone metabolism. Five biomarkers (FADS2, ADRA2A, FMN1, RABL2A, SPRY1) revealed causal effects on BMD variation. Our study provided an innovative framework and insights into the pathogenesis of osteoporosis.

Features fusion of multichannel wrist pulse signal based on KL-MGDCCA and decision level combination

To sense the pulse in the representative positions of wrist is the basis of traditional Chinese pulse diagnosis. The pulse diagnosis has been obtaining more and more attentions for its non-invasive character and its convenience in analysis of health status. For objective analysis, various types of pulse features have been extracted from the pulse signal with the development of computerized pulse analysis. The effective utilization of the features of multichannel is the increasing and urgent need for the pulse analysis. A novel features fusion frame is proposed to reduce the redundant information for the homogeneous features, and eliminate the interference of heterogeneous features. For the same type of features extracted from the different channels, the proposed method uses Karhunen–Loeve multiple generalized discriminative canonical correlation analysis (KL-MGDCCA) to fuse them into one feature vector. A support vector machine (SVM) classifier is trained for each type of fused features. Then, the frame adopts decision level fusion approach to combine these classifiers for pulse signal classification to solve the problem of heterogeneous features fusion. Extensive experiments show that the proposed fusion frame can achieve the best performance on most indicators for multichannel pulse signal analysis. For the classification of Diabetes/Health, Nephropathy/Health and Diabetes/Nephropathy, the proposed method achieves the best F-score with the value of 75.25%, 79.02% and 56.54%, which outperforms state-of-the-art methods being compared.

Hybrid Dimensionality Reduction of Multi-sets Using Nature Inspired Algorithms and Discriminant Canonical Correlation Analysis for Automatic Sleep Stage Classification

Hybrid Dimensionality Reduction of Multi-sets Using Nature Inspired Algorithms and Discriminant Canonical Correlation Analysis for Automatic Sleep Stage Classification

Palmprint Identification Integrating Left and Right by Local Discriminant Canonical Correlation Analysis

Palmprint Identification Integrating Left and Right by Local Discriminant Canonical Correlation Analysis

Integrating leaf and flower by local discriminant CCA for plant species recognition

Plant species recognition using a single organ, such as flower and leaf, is not sufficiently reliable, because different species may have very similar flowers or leaves, while the same species may have rather different flowers or leaves. Combining leaves and flowers to recognize plant species can produce positive results. Based on multi-modal learning scheme, an automatic plant species recognition method is proposed by combining leaves and flowers of plant. In the method, a modified local discriminant canonical correlation analysis (MLDCCA) is designed by incorporating the idea of local discriminant embedding (LDE) into canonical correlation analysis (CCA). Firstly, two neighbor graphs are constructed based on the exploration of the manifold that the input data lie on. Then, two projection matrices for dimensionality reduction are obtained by making the within-class neighbor samples most correlated and between-class neighbor samples least correlated, and meanwhile keeping the correlation between leaves and flowers of the same species maximum. Finally, 1-nearest neighbor classifier with geodesic distance is used to recognize the plant species. MLDCCA is a powerful supervised multi-modal dimensional reduction method which can extract the discriminant features from two plant organs, meanwhile preserve the discriminant information and the data structure well. Experimental results on a real leaf and flower image dataset validate the effectiveness of the proposed method.

Sparse regularized discriminative canonical correlation analysis for multi-view semi-supervised learning

For multi-view data representation learning, recently the traditional unsupervised CCA method has been converted to supervised ways by introducing label information from samples. However, such supervised CCA variants require large numbers of labeled samples which hampers its practical application. In this paper, in order to mine the most discriminant information only from a few labeled samples, inspired by sparse representation we propose a novel sparse regularized discriminative CCA method to make use of the label information as much as possible. Through constructing sparse weighted matrices in multiple views, we incorporate the structure information into the original CCA framework to extract fused multi-view features which not only are the most correlated but also carry the important discriminative structure information. Our approach is evaluated on both handwritten dataset and face dataset. The experimental results and the comparisons with other related algorithms demonstrate its effectiveness and superiority.

Local Discriminant Canonical Correlation Analysis for Supervised PolSAR Image Classification

This letter proposes a novel multiview feature extraction method for supervised polarimetric synthetic aperture radar (PolSAR) image classification. PolSAR images can be characterized by multiview feature sets, such as polarimetric features and textural features. Canonical correlation analysis (CCA) is a well-known dimensionality reduction (DR) method to extract valuable information from multiview feature sets. However, it cannot exploit the discriminative information, which influences its performance of classification. Local discriminant embedding (LDE) is a supervised DR method, which can preserve the discriminative information and the local structure of the data well. However, it is a single-view learning method, which does not consider the relation between multiple view feature sets. Therefore, we propose local discriminant CCA by incorporating the idea of LDE into CCA. Specific to PolSAR images, a symmetric version of revised Wishart distance is used to construct the between-class and within-class neighboring graphs. Then, by maximizing the correlation of neighboring samples from the same class and minimizing the correlation of neighboring samples from different classes, we find two projection matrices to achieve feature extraction. Experimental results on the real PolSAR data sets demonstrate the effectiveness of the proposed method.