Tensor Canonical Correlation Analysis Research Articles

Trial Selection Tensor Canonical Correlation Analysis (TSTCCA) for Depression Recognition with Facial Expression and Pupil Diameter.

Facial expressions have been widely used for depression recognition because it is intuitive and convenient to access. Pupil diameter contains rich emotional information that is already reflected in facial video streams. However, the spatiotemporal correlation between pupillary changes and facial behavior changes induced by emotional stimuli has not been explored in existing studies. This paper presents a novel multimodal fusion algorithm - Trial Selection Tensor Canonical Correlation Analysis (TSTCCA) to optimize the feature space and build a more robust depression recognition model, which innovatively combines the spatiotemporal relevance and complementarity between facial expression and pupil diameter features. TSTCCA explores the interaction between trials and obtains an effective fusion representation of two modalities from a trial subset related to depression. The experimental results show that TSTCCA achieves the highest accuracy of 78.81% with the subset of 25 trials.

Learning High-Order Multi-View Representation by New Tensor Canonical Correlation Analysis

Canonical correlation analysis (CCA) has attracted great interest in multi-view representation. However, most of the CCA methods heavily rely on the matrix structure, which may neglect the prior geometric information in high-order data. To deal with the above issue, we first propose a novel tensor CCA formulation with orthogonality, called TCCA-O, based on the Tucker decomposition to preserve the orthogonality. Then, we incorporate a structured sparse regularization term into the TCCA-O, called TCCA-OS, to improve feature representation. In addition, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm to solve TCCA-OS and conduct numerical comparisons on four public datasets. The results validate the advantages of the proposed methods in terms of classification accuracy, parameter sensitivity, noise robustness, and model stability. In particular, TCCA-O and TCCA-OS improve the classification accuracy by at least 10.03% and 10.36%, respectively, over the state-of-the-art CCA methods on the Caltech101-7 dataset.

Multi-Group Tensor Canonical Correlation Analysis.

Tensor Canonical Correlation Analysis (TCCA) is a commonly employed statistical method utilized to examine linear associations between two sets of tensor datasets. However, the existing TCCA models fail to adequately address the heterogeneity present in real-world tensor data, such as brain imaging data collected from diverse groups characterized by factors like sex and race. Consequently, these models may yield biased outcomes. In order to surmount this constraint, we propose a novel approach called Multi-Group TCCA (MG-TCCA), which enables the joint analysis of multiple subgroups. By incorporating a dual sparsity structure and a block coordinate ascent algorithm, our MG-TCCA method effectively addresses heterogeneity and leverages information across different groups to identify consistent signals. This novel approach facilitates the quantification of shared and individual structures, reduces data dimensionality, and enables visual exploration. To empirically validate our approach, we conduct a study focused on investigating correlations between two brain positron emission tomography (PET) modalities (AV-45 and FDG) within an Alzheimer's disease (AD) cohort. Our results demonstrate that MG-TCCA surpasses traditional TCCA in identifying sex-specific cross-modality imaging correlations. This heightened performance of MG-TCCA provides valuable insights for the characterization of multimodal imaging biomarkers in AD.

RETRACTED: A Novel Day- ahead Electricity Price Forecasting Using multi-modal combined Integration via Stacked Pruning Sparse Denoising Auto Encoder

One of the best options for managing energy markets and power system performance optimization is precise Electricity Price Forecasting (EPF). In detecting the prices of energy, some factors are of stochastic behavior which has made an unwieldy task. In this study, carrying out Tensor Canonical Correlation Analysis (TCCA) is the first activity to choose the operative parameters in EPF issues and removing redundant factors with low correlation ranks. The second activity is making use of a Deep Neural Network (DNN) with a novel Stacked Pruning Sparse Denoising Autoencoder (SPSDAE) to decrease the noise of data sets with different sources individually. The third activity is that a novel method which is called the Maximum Separation Subspace (MASES) in Sufficient Dimension Reduction (SDR) with Categorical Response. This method is used to detect the outstanding properties of the input data and removing the uncertainty characteristics. The new multi-modal combined (MMC) method is the final step that needs to be taken to predict the day-ahead (one day in advance) cost of electricity. The proposed method is investigated based on the data that were collected from Australia The combination of various predicting models and suggested methods might develop the function of predicting possibility by supplying divergent types of compactness performance. The combined model comprised of three predicting possibilities called beta circulation fitting, scattered Bayesian learning, core compactness prediction. Predicting data respectively are used as a yardstick to compare the efficiency of forecasted results and consequently assess the proposed method. Results of this study confirmed a decrease of effectual error criterion and accuracy enhancement in EPF field as the advantageous effects of this proposed SDR structure.

Tensor Canonical Correlation Analysis With Convergence and Statistical Guarantees

In many applications, such as classification of images or videos, it is of interest to develop a framework for tensor data instead of an ad-hoc way of transforming data to vectors due to the computational and under-sampling issues. In this article, we study convergence and statistical properties of two-dimensional canonical correlation analysis under an assumption that data come from a probabilistic model. We show that carefully initialized the power method converges to the optimum and provide a finite sample bound. Then we extend this framework to tensor-valued data and propose the higher-order power method, which is commonly used in tensor decomposition, to extract the canonical directions. Our method can be used effectively in a large-scale data setting by solving the inner least squares problem with a stochastic gradient descent, and we justify convergence via the theory of Lojasiewicz’s inequalities without any assumption on data generating process and initialization. For practical applications, we further develop (a) an inexact updating scheme which allows us to use the state-of-the-art stochastic gradient descent algorithm, (b) an effective initialization scheme which alleviates the problem of local optimum in nonconvex optimization, and (c) a deflation procedure for extracting several canonical components. Empirical analyses on challenging data including gene expression and air pollution indexes in Taiwan, show the effectiveness and efficiency of the proposed methodology. Our results fill a missing, but crucial, part in the literature on tensor data. Supplementary materials for this article are available online.

Deep Tensor CCA for Multi-view Learning

We present Deep Tensor Canonical Correlation Analysis (DTCCA), a method to learn complex nonlinear transformations of multiple views (more than two) of data such that the resulting representations are linearly correlated in high order. The high-order correlation of given multiple views is modeled by covariance tensor, which is different from most CCA formulations relying solely on the pairwise correlations. Parameters of transformations of each view are jointly learned by maximizing the high-order canonical correlation. To solve the resulting problem, we reformulate it as the best sum of rank-1 approximation, which can be efficiently solved by existing tensor decomposition method. DTCCA is a nonlinear extension of tensor CCA (TCCA) via deep networks. Comparing with kernel TCCA, DTCCA not only can deal with arbitrary dimensions of the input data, but also does not need to maintain the training data for computing representations of any given data point. Hence, DTCCA as a unified model can efficiently overcome the scalable issue of TCCA for either high-dimensional multi-view data or a large amount of views, and it also naturally extends TCCA for learning nonlinear representation. Extensive experiments on four multi-view data sets demonstrate the effectiveness of the proposed method.

A Novel Multi-task Tensor Correlation Neural Network for Facial Attribute Prediction

Multi-task learning plays an important role in face multi-attribute prediction. At present, most researches excavate the shared information between attributes by sharing all convolutional layers. However, it is not appropriate to treat the low-level and high-level features of the face multi-attribute equally, because the high-level features are more biased toward the specific content of the category. In this article, a novel multi-attribute tensor correlation neural network (MTCN) is used to predict face attributes. MTCN shares all attribute features at the low-level layers, and then distinguishes each attribute feature at the high-level layers. To better excavate the correlations among high-level attribute features, each sub-network explores useful information from other networks to enhance its original information. Then a tensor canonical correlation analysis method is used to seek the correlations among the highest-level attributes, which enhances the original information of each attribute. After that, these features are mapped into a highly correlated space through the correlation matrix. Finally, we use sufficient experiments to verify the performance of MTCN on the CelebA and LFWA datasets and our MTCN achieves the best performance compared with the latest multi-attribute recognition algorithms under the same settings.

Tensor Canonical Correlation Analysis Networks for Multi-View Remote Sensing Scene Recognition

Convolutional neural network (CNN) has been proven an effective way to extract high-level features from remote sensing (RS) images automatically. Many variants of the CNN model have been proposed, including principal component analysis network (PCANet), canonical correlation analysis network (CCANet), multiple scale CCANet (MS-CCANet) and multiview CCANet (MCCANet). The PCANet is specialized for single view feature abstraction, while in many real-world practices, the RS data are frequently observed from many more views. Although CCANet, MS-CCANet and MCCANet can be applied to two or more view data, they consider only the pair-wise correlation by calculating a series of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">two-order</i> covariance matrices. However, the high-order consistence, which can only be explored by collectively and simultaneously examining all views, remains undiscovered. In this paper, we propose the tensor canonical correlation analysis network (TCCANet) to tackle this problem. Particularly, TCCANet learns filter banks by simultaneously maximizing arbitrary number of views with high-order-correlation and solves the optimization problem by decomposing a covariance tensor. After the convolutional stage, we utilize binarization and block-wise histogram strategies to generate the final feature. Furthermore, we also develop a Multiple Scale version of TCCANet, i.e., MS-TCCANet, to extract enriched representation of the RS data by incorporating all previous convolutional layers. Numerical experiment results on RSSCN7 and SAT-6 datasets demonstrate the advantages of TCCANet and MS-TCCANet for RS scene recognition.

Features-Enhanced Multi-Attribute Estimation with Convolutional Tensor Correlation Fusion Network

To achieve robust facial attribute estimation, a hierarchical prediction system referred to as tensor correlation fusion network (TCFN) is proposed for attribute estimation. The system includes feature extraction, correlation excavation among facial attribute features, score fusion, and multi-attribute prediction. Subnetworks (Age-Net, Gender-Net, Race-Net, and Smile-Net) are used to extract corresponding features while Main-Net extracts features not only from an input image but also from corresponding pooling layers of subnetworks. Dynamic tensor canonical correlation analysis (DTCCA) is proposed to explore the correlation of different targets’ features in the F7 layers. Then, for binary classifications of gender, race, and smile, corresponding robust decisions are achieved by fusing the results of subnetworks with those of TCFN while for age prediction, facial image into one of age groups, and then ELM regressor performs the final age estimation. Experimental results on benchmarks with multiple face attributes (MORPH-II, Adience Benchmark datasets, LAP-2016, and CelebA) show that the proposed approach has superior performance compared to state of the art.

Multi-view learning-based data proliferator for boosting classification using highly imbalanced classes

BackgroundMulti-view data representation learning explores the relationship between the views and provides rich complementary information that can improve computer-aided diagnosis. Specifically, existing machine learning methods devised to automate neurological disorder diagnosis using brain data provided new insights into how a particular disorder such as autism spectrum disorder (ASD) alters the brain construct. However, the performance of machine learning methods highly depends on the size of the training samples from both classes. In a real-world clinical setting, such medical data is very expensive and challenging to collect, might (i) suffer from several limitations such as imbalanced classes and (ii) have non-heterogeneous distribution when derived from multi-view brain representations. New methodTo the best of our knowledge, the problem of imbalanced and multi-view data classification remains unexplored in the field of network neuroscience. To fill this gap, we propose a Multi-View LEArning-based data Proliferator (MV-LEAP) that enables the classification of imbalanced multi-view representations. MV-LEAP comprises two key steps. First, a manifold learning-based proliferator, which enables to generate synthetic data for each view, is developed to handle imbalanced data. Second, a multi-view manifold data alignment leveraging tensor canonical correlation analysis is proposed to map all original and proliferated (i.e., synthesized) views into a shared subspace where their distributions are aligned for the target classification task. ResultsWe evaluated our method on imbalanced multi-view ASD vs. normal control (NC) connectomic datasets with imbalanced classes. ConclusionOverall, MV-LEAP achieved the best classification results in comparison with baseline data synthesis methods.

Tensor canonical correlation analysis.

Canonical correlation analysis (CCA) is a multivariate analysis technique for estimating a linear relationship between two sets of measurements. Modern acquisition technologies, for example, those arising in neuroimaging and remote sensing, produce data in the form of multidimensional arrays or tensors. Classic CCA is not appropriate for dealing with tensor data due to the multidimensional structure and ultrahigh dimensionality of such modern data. In this paper, we present tensor CCA (TCCA) to discover relationships between two tensors while simultaneously preserving multidimensional structure of the tensors and utilizing substantially fewer parameters. Furthermore, we show how to employ a parsimonious covariance structure to gain additional stability and efficiency. We delineate population and sample problems for each model and propose efficient estimation algorithms with global convergence guarantees. Also we describe a probabilistic model for TCCA that enables the generation of synthetic data with desired canonical variates and correlations. Simulation studies illustrate the performance of our methods.

Sparse tensor canonical correlation analysis for micro-expression recognition

A micro-expression is considered a fast facial movement that indicates genuine emotions and thus provides a cue for deception detection. Due to its promising applications in various fields, psychologists and computer scientists, particularly those focus on computer vision and pattern recognition, have shown interest and conducted research on this topic. However, micro-expression recognition accuracy is still low. To improve the accuracy of such recognition, in this study, micro-expression data and their corresponding Local Binary Pattern (LBP) (Ojala et al., 2002) [1] code data are fused by correlation analysis. Here, we propose Sparse Tensor Canonical Correlation Analysis (STCCA) for micro-expression characteristics. A sparse solution is obtained by the regularized low rank matrix approximation. Experiments are conducted on two micro-expression databases, CASME and CASME 2, and the results show that STCCA performs better than the Three-dimensional Canonical Correlation Analysis (3D-CCA) without sparse resolution. The experimental results also show that STCCA performs better than three-order Discriminant Tensor Subspace Analysis (DTSA3) with discriminant information, smaller projected dimensions and a larger training set sample size. The experiments also showed that Multi-linear Principal Component Analysis (MPCA) is not suitable for micro-expression recognition because the eigenvectors corresponding to smaller eigenvectors are discarded, and those eigenvectors include brief and subtle motion information.

Tensor completion via multi-shared-modes canonical correlation analysis

Low-rank tensor completion (LRTC) has been applied in many real-world problems. But most of the existing LRTC methods recover a tensor on a single dataset with the low-rank assumption, suffering from a low accuracy due to the complicated structures of higher-order data. To address this issue, we propose a novel tensor completion method for two correlated tensor datasets obtained from different sources. We first introduce the correlated tensors with multiple shared modes via tensor canonical correlation analysis (TCCA), and reveal the relationship between the transformation matrices of TCCA and the Tucker decomposition. Then we develop a Tucker-n decomposition method with n invariant modes to capture the latent structures of incomplete tensors, in which sufficient discriminative information for TCCA can be flexibly maintained by varying the number of invariant modes. Finally, we combine the Tucker-n decomposition method for LRTC with the correlation of TCCA as a regularizer to improve the completion performance, and derive relative error bounds for our LRTC approach to guarantee the recovery accuracy. Experimental results on synthetic and real data demonstrate the accuracy and efficiency of the proposed approach, as well as the benefit of multiple shared modes.

Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction

Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while in many real-world applications, the number of views is frequently many more. Although the ad hoc way of simultaneously exploring all possible pairs of features can numerically deal with multi-view data, it ignores the high order statistics (correlation information) which can only be discovered by simultaneously exploring all features. Therefore, in this work, we develop tensor CCA (TCCA) which straightforwardly yet naturally generalizes CCA to handle the data of an arbitrary number of views by analyzing the covariance tensor of the different views. TCCA aims to directly maximize the canonical correlation of multiple (more than two) views. Crucially, we prove that the multi-view canonical correlation maximization problem is equivalent to finding the best rank-1 approximation of the data covariance tensor, which can be solved efficiently using the well-known alternating least squares (ALS) algorithm. As a consequence, the high order correlation information contained in the different views is explored and thus a more reliable common subspace shared by all features can be obtained. In addition, a non-linear extension of TCCA is presented. Experiments on various challenge tasks, including large scale biometric structure prediction, internet advertisement classification and web image annotation, demonstrate the effectiveness of the proposed method.

RETRACTED ARTICLE: Sparse tensor CCA for color face recognition

This paper proposes a subspace learning method, named as sparse tensor canonical correlation analysis (ST-CCA), for color face recognition. A sample image is formalized as high-order tensors to preserve the inherent structure of the color face images. We utilize sparse canonical correlation analysis (SCCA) to choose gene. For each pair of tensors, SCCA generates the sparse loadings alternately, which is helpful for choosing significant variables to reduce dimensions and eliminate the redundancies of tensors. We use the elastic net as constraint condition to attack the collinearity problem by decorrelating and selecting the sufficient variables irrespective of the limited dimensions. Furthermore, ST-CCA gains stable recognition rates because the alternating least square solution converges. ST-CCA is convex with different initials of the projection matrices. Experimental results on AR face database and LFW face database show the superior performance of our method over the state-of-the-art ones.

Kernel analysis on Grassmann manifolds for action recognition

Modelling video sequences by subspaces has recently shown promise for recognising human actions. Subspaces are able to accommodate the effects of various image variations and can capture the dynamic properties of actions. Subspaces form a non-Euclidean and curved Riemannian manifold known as a Grassmann manifold. Inference on manifold spaces usually is achieved by embedding the manifolds in higher dimensional Euclidean spaces. In this paper, we instead propose to embed the Grassmann manifolds into reproducing kernel Hilbert spaces and then tackle the problem of discriminant analysis on such manifolds. To achieve efficient machinery, we propose graph-based local discriminant analysis that utilises within-class and between-class similarity graphs to characterise intra-class compactness and inter-class separability, respectively. Experiments on KTH, UCF Sports, and Ballet datasets show that the proposed approach obtains marked improvements in discrimination accuracy in comparison to several state-of-the-art methods, such as the kernel version of affine hull image-set distance, tensor canonical correlation analysis, spatial-temporal words and hierarchy of discriminative space-time neighbourhood features.