Multiset Canonical Correlation Analysis Research Articles

DNA Methylation and Resting Brain Function Mediate the Association between Childhood Urbanicity and Better Speed of Processing.

Urbanicity has been suggested to affect cognition, but the underlying mechanism remains unknown. We examined whether epigenetic modification (DNA methylation, DNAm), and brain white matter fiber integrity (fractional anisotropy, FA) or local spontaneous brain function activity (regional homogeneity, ReHo) play roles in the association between childhood urbanicity and cognition based on 497 healthy Chinese adults. We found significant correlation between childhood urbanicity and better cognitive performance. Multiset canonical correlation analysis (mCCA) identified an intercorrelated DNAm-FA-ReHo triplet, which showed significant pairwise correlations (DNAm-FA: Bonferroni-adjusted P, Pbon = 4.99E−03, rho = 0.216; DNAm-ReHo: Pbon = 4.08E−03, rho = 0.239; ReHo-FA: Pbon = 1.68E−06, rho = 0.328). Causal mediation analysis revealed that 1) ReHo mediated 10.86% childhood urbanicity effects on the speed of processing and 2) childhood urbanicity alters ReHo through DNA methylation in the cadherin and Wnt signaling pathways (mediated effect: 48.55%). The mediation effect of increased ReHo in the superior temporal gyrus underlying urbanicity impact on a better speed of processing was further validated in an independent cohort. Our work suggests a mediation role for ReHo, particularly increased brain activity in the superior temporal gyrus, in the urbanicity-associated speed of processing.

Disjoint subspaces for common and distinct component analysis: Application to the fusion of multi-task FMRI data

BackgroundData-driven methods such as independent component analysis (ICA) makes very few assumptions on the data and the relationships of multiple datasets, and hence, are attractive for the fusion of medical imaging data. Two important extensions of ICA for multiset fusion are the joint ICA (jICA) and the multiset canonical correlation analysis and joint ICA (MCCA-jICA) techniques. Both approaches assume identical mixing matrices, emphasizing components that are common across the multiple datasets. However, in general, one would expect to have components that are both common across the datasets and distinct to each dataset. New methodWe propose a general framework, disjoint subspace analysis using ICA (DS-ICA), which identifies and extracts not only the common but also the distinct components across multiple datasets. A key component of the method is the identification of these subspaces and their separation before subsequent analyses, which helps establish better model match and provides flexibility in algorithm and order choice. ComparisonWe compare DS-ICA with jICA and MCCA-jICA through both simulations and application to multiset functional magnetic resonance imaging (fMRI) task data collected from healthy controls as well as patients with schizophrenia. ResultsThe results show DS-ICA estimates more components discriminative between healthy controls and patients than jICA and MCCA-jICA, and with higher discriminatory power showing activation differences in meaningful regions. When applied to a classification framework, components estimated by DS-ICA results in higher classification performance for different dataset combinations than the other two methods. ConclusionThese results demonstrate that DS-ICA is an effective method for fusion of multiple datasets.

Remote Photoplethysmography With an EEMD-MCCA Method Robust Against Spatially Uneven Illuminations

Remote photoplethysmography (rPPG) is an attractive video-based technique to monitor heart rate (HR) information in telehealth screening. The subjects are required to stay stationary in the remote screening scenario and ambient lights become the main interference source of measuring HR with rPPG. In this paper, we introduce a novel approach robust against spatially uneven illumination interference in rPPG by combining ensemble empirical mode decomposition (EEMD) with multiset canonical correlation analysis (MCCA). We adopt the following procedures to ensure that the pulses dominate the correlations across multiple signal sets while the illumination noises are as diverse as possible. Specifically, a group of optimal regions of interest (ROIs) are selected according to the quality indicators defined from the green channel in each candidate ROI. A multi-channel signal set is then constructed by decomposing the green signal with EEMD. Only those intrinsic mode functions (IMFs) with the dominant frequencies falling into the interested HR range are utilized as the input of MCCA. The canonical variables (CVs) with the highest cross correlations are derived as the underlying candidate pulses. Finally, fast Fourier transform (FFT) is employed to calculate the dominant frequency and the target HR is determined based on both the quasi-periodic property and the continuity of HR. The proposed EEMD-MCCA method is validated on both the in-house BSIPL-RPPG and the public COHFACE databases, which achieves superior performance over several typical rPPG methods. This study will provide a promising tool for realistic rPPG applications in telehealth screening.

Multivariate statistical process control methods for batch production: a review focused on applications

ABSTRACT In this paper, we highlight the basic techniques of multivariate statistical process control (MSPC) under the dimensionality criteria, such as Multiway Principal Component Analysis, Multiway Partial Squares, Structuration à Trois Indices de la Statistique, Tucker3, Parallel Factors, Multiway Independent Component Analysis, Multiset Canonical Correlation Analysis, Slow Features Analysis, and Parallel Coordinates. Furthermore, we summarize the procedures of each statistical technique and the usage of multivariate control charts. In addition, we review the most significant proposals and applications in practical cases. Finally, we compare and discuss the benefits and limitations within methods.

Detection of Important Scenes in Baseball Videos via Bidirectional Time Lag Aware Deep Multiset Canonical Correlation Analysis

A novel method for detection of important scenes in baseball videos based on correlation maximization between heterogeneous modalities via bidirectional time lag aware deep multiset canonical correlation analysis (BiTl-dMCCA) is presented in this paper. The proposed method enables detection of important scenes by collaboratively using baseball videos and their corresponding tweets. The technical contributions of this paper are twofold. First, since there are time lags between not only “tweets and corresponding multiple previous events” but also “events and corresponding multiple following posted tweets”, the proposed method considers these bidirectional time lags. Specifically, the representation of such bidirectional time lags into the derivation of their covariance matrices is newly introduced. Second, the proposed method adopts textual, visual and audio features calculated from tweets and videos as multi-modal time series features. Important scenes are detected as abnormal scenes via anomaly detection based on a generative adversarial network using multi-modal features projected by BiTl-dMCCA. The proposed method does not need any training data with annotation. Experimental results obtained by applying the proposed method to actual baseball matches show the effectiveness of the proposed method.

Learning Trajectory Patterns via Canonical Correlation Analysis

A substantial body of research has been devoted to the analysis of motion trajectories. Usually, a motion trajectory consists of a set of coordinates, which is called a raw trajectory. In this paper, the authors first use vectors for some artificially constructed global features, such as the mean discrete curvature and standard deviation of acceleration, to represent the raw trajectory data, and then apply a multiset canonical correlation analysis method to extract latent features from the artificially constructed features. The performance of the latent features is then measured by evaluating the accuracy and F1 score of a gradient boosting decision tree model for different datasets, which include paired sample datasets and unpaired sample datasets. The experimental results show that the classifier performance for MCCA features is much better than that obtained for the artificially constructed features, such as that for the motion distance or mean velocity.

ANy-way Independent Component Analysis.

Multimodal data fusion is a topic of great interest. Several fusion methods have been proposed to investigate coherent patterns and corresponding linkages across modalities, such as joint independent component analysis (jICA), multiset canonical correlation analysis (mCCA), mCCA+jICA, disjoint subspace using ICA (DS-ICA) and parallel ICA. JICA exploits source independence but assumes shared loading parameters. MCCA maximizes correlation linkage across modalities directly but is limited to orthogonal features. While there is no theoretical limit to the number of modalities analyzed together by jICA, mCCA, or the two-step approach mCCA+jICA, these approaches can only extract common features and require the same number of sources/components for all modalities. On the other hand, DS-ICA and parallel ICA can identify both common and distinct features but are limited to two modalities. DS-ICA assumes shared loading parameters among common features, which works well when links are strong. Parallel ICA simultaneously maximizes correlation between modalities and independence of sources, while allowing different number of sources for each modality. However, only a very limited number of modalities and linkage pairs can be optimized. To overcome these limitations, we propose aNy-way ICA, a new model to simultaneously maximize the independence of sources and correlations across modalities. aNy-way ICA combines infomax ICA and Gaussian independent vector analysis (IVA-G) via a shared weight matrix model without orthogonality constraints. Simulation results demonstrate that aNy-way ICA not only accurately recovers sources and loadings, but also the true covariance/linkage patterns, whether different modalities have the same or different number of sources. Moreover, aNy-way ICA outperforms mCCA and mCCA+jICA in terms of source and loading recovery accuracy, especially under noisy conditions.Clinical Relevance-This establishes a model for N-way data fusion of any number of modalities and linkage pairs, allowing different number of non-orthogonal sources for different modalities.

Artifact Removal in tACS-EEG Recordings: A Combined Methodology Based on the Empirical Wavelet Transform.

Transcranial alternating current stimulation (tACS) is a non-invasive brain stimulation technique that modulates brain activity, which yields promise for achieving desired behavioral outcomes in different contexts. Combining tACS with electroencephalography (EEG) allows for the monitoring of the real-time effects of stimulation. However, the EEG signal recorded with simultaneous tACS is largely contaminated by stimulation-induced artifacts. In this work, we examine the combination of the empirical wavelet transform (EWT) with three blind source separation (BSS) methods: principal component analysis (PCA), multiset canonical correlation analysis (MCCA) and independent vector analysis (IVA), aiming to remove artifacts in tACS-contaminated EEG recordings. Using simulated data, we show that EWT followed by IVA achieves the best performance. Using experimental data, we show that BSS combined with EWT performs better compared to standard BSS methodology in terms of preserving useful information while eliminating artifacts.

Decision-Making Selector (DMS) for Integrating CCA-Based Methods to Improve Performance of SSVEP-Based BCIs.

Recent research has demonstrated improved performance of a brain-computer interface (BCI) using fusion based approaches. This paper proposes a novel decision-making selector (DMS) to integrate classification decisions of different frequency recognition methods based on canonical correlation analysis (CCA) which were used in decoding steady state visual evoked potentials (SSVEPs). The DMS method selects a decision more likely to be correct from two methods namely as M1 and M2 by separating the M1-false and M2-false trials. To measure the uncertainty of each decision, feature vectors were extracted using the largest and second largest correlation coefficients corresponding to all the stimulus frequencies. The proposed method was evaluated by integrating all pairs of 7 CCA-based algorithms, including CCA, individual template-based CCA (ITCCA), multi-set CCA (MsetCCA), L1-regularized multi-way CCA (L1-MCCA), filter bank CCA (FBCCA), extended CCA (ECCA), and task-related component analysis (TRCA). The experimental results obtained from a 40-target dataset of thirty-five subjects showed that the proposed DMS method was validated to obtain an enhanced performance by integrating the algorithms with close accuracies. The results suggest that the proposed DMS method is effective in integrating decisions of different methods to improve the performance of SSVEP-based BCIs.

Sensory Modality-Independent Activation of the Brain Network for Language.

The meaning of a sentence can be understood, whether presented in written or spoken form. Therefore, it is highly probable that brain processes supporting language comprehension are at least partly independent of sensory modality. To identify where and when in the brain language processing is independent of sensory modality, we directly compared neuromagnetic brain signals of 200 human subjects (102 males) either reading or listening to sentences. We used multiset canonical correlation analysis to align individual subject data in a way that boosts those aspects of the signal that are common to all, allowing us to capture word-by-word signal variations, consistent across subjects and at a fine temporal scale. Quantifying this consistency in activation across both reading and listening tasks revealed a mostly left-hemispheric cortical network. Areas showing consistent activity patterns included not only areas previously implicated in higher-level language processing, such as left prefrontal, superior and middle temporal areas, and anterior temporal lobe, but also parts of the control network as well as subcentral and more posterior temporal-parietal areas. Activity in this supramodal sentence-processing network starts in temporal areas and rapidly spreads to the other regions involved. The findings indicate not only the involvement of a large network of brain areas in supramodal language processing but also that the linguistic information contained in the unfolding sentences modulates brain activity in a word-specific manner across subjects.SIGNIFICANCE STATEMENT The brain can extract meaning from written and spoken messages alike. This requires activity of both brain circuits capable of processing sensory modality-specific aspects of the input signals as well as coordinated brain activity to extract modality-independent meaning from the input. Using traditional methods, it is difficult to disentangle modality-specific activation from modality-independent activation. In this work, we developed and applied a multivariate methodology that allows for a direct quantification of sensory modality-independent brain activity, revealing fast activation of a wide network of brain areas, both including and extending beyond the core network for language.

Spatial Filtering in SSVEP-Based BCIs: Unified Framework and New Improvements.

In the steady-state visual evoked potential (SSVEP)-based brain computer interfaces (BCIs), spatial filtering, which combines the multi-channel electroencephalography (EEG) signals in order to reduce the non-SSVEP-related component and thus enhance the signal-to-noise ratio (SNR), plays an important role in target recognition. Recently, various spatial filtering algorithms have been developed employing different prior knowledge and characteristics of SSVEPs, however how these algorithms interconnect and differ is not yet fully explored, leading to difficulties in further understanding, utilizing and improving them. We propose a unified framework under which the spatial filtering algorithms can be formulated as generalized eigenvalue problems (GEPs) with four different elements: data, temporal filter, orthogonal projection and spatial filter. Based on the framework, we design new spatial filtering algorithms for improvements through the choice of different elements. The similarities, differences and relationships among nineteen mainstream spatial filtering algorithms are revealed under the proposed framework. Particularly, it is found that they originate from the canonical correlation analysis (CCA), principal component analysis (PCA), and multi-set CCA, respectively. Furthermore, three new spatial filtering algorithms are developed with enhanced performance validated on two public SSVEP datasets with 45 subjects. The proposed framework provides insights into the underlying relationships among different spatial filtering algorithms and helps the design of new spatial filtering algorithms. This is a systematic study to explore, compare and improve the existing spatial filtering algorithms, which would be significant for further understanding and future development of high performance SSVEP-based BCIs.

Supervised Fractional-Order Embedding Geometrical Multi-View CCA (SFGMCCA) for Multiple Feature Integration

Techniques for integrating different types of multiple features effectively have been actively studied in recent years. Multiset canonical correlation analysis (MCCA), which maximizes the sum of pairwise correlations of inter-view (i.e., between different features), is one of the powerful methods for integrating different types of multiple features, and various MCCA-based methods have been proposed. This work focuses on a supervised MCCA variant in order to construct a novel effective feature integration framework. In this paper, we newly propose supervised fractional-order embedding geometrical multi-view CCA (SFGMCCA). This method constructs not only the correlation structure but also two types of geometrical structures of intra-view (i.e., within each feature) and inter-view simultaneously, thereby realizing more precise feature integration. This method also supports the integration of small sample and high-dimensional data by using the fractional-order technique. We conducted experiments using four types of image datasets, i.e., MNIST, COIL-20, ETH-80 and CIFAR-10. Furthermore, we also performed an fMRI dataset containing brain signals to verify the robustness. As a result, it was confirmed that accuracy improvements using SFGMCCA were statistically significant at the significance level of 0.05 compared to those using conventional representative MCCA-based methods.

Multi-Modal Subspace Fusion via Cauchy Multi-Set Canonical Correlations

Multi-set canonical correlation analysis (MCCA) is a famous multi-modal coherent subspace learning method. However, sample-based between-modal and within-modal covariance matrices of MCCA usually deviate from real covariance matrices due to noise information and limited sample size. The deviation will weaken the performance of MCCA, especially in image recognition. Aiming at this challenging issue, we correct singular values of sample covariance matrices with the employment of Cauchy estimate theory and further obtain Cauchy covariance matrices that are closer to real covariance matrices. On the basis of Cauchy covariance matrices, we develop a novel multi-modal subspace fusion method, i.e. Cauchy multi-set canonical correlations. By maximizing Cauchy correlations between different modalities and constraining Cauchy scatters of within-modal data, the method can learn a Cauchy coherent fusion subspace with well discriminative power from a few images. Experiment results have shown the effectiveness of the proposed method, promising to the aims of this research.

Multi-Set Canonical Correlation Analysis for 3D Abnormal Gait Behaviour Recognition Based on Virtual Sample Generation

Small sample dataset and two-dimensional (2D) approach are challenges to vision-based abnormal gait behaviour recognition (AGBR). The lack of three-dimensional (3D) structure of the human body causes 2D based methods to be limited in abnormal gait virtual sample generation (VSG). In this paper, 3D AGBR based on VSG and multi-set canonical correlation analysis (3D-AGRBMCCA) is proposed. First, the unstructured point cloud data of gait are obtained by using a structured light sensor. A 3D parametric body model is then deformed to fit the point cloud data, both in shape and posture. The features of point cloud data are then converted to a high-level structured representation of the body. The parametric body model is used for VSG based on the estimated body pose and shape data. Symmetry virtual samples, pose-perturbation virtual samples and various body-shape virtual samples with multi-views are generated to extend the training samples. The spatial-temporal features of the abnormal gait behaviour from different views, body pose and shape parameters are then extracted by convolutional neural network based Long Short-Term Memory model network. These are projected onto a uniform pattern space using deep learning based multi-set canonical correlation analysis. Experiments on four publicly available datasets show the proposed system performs well under various conditions.

Major depressive disorder identification by referenced multiset canonical correlation analysis with clinical scores.

A novel method based on multiset canonical correlation analysis (mCCA) and linear discriminant analysis (LDA) is presented to identify the major depressive disorder (MDD). The new method comprises two parts, namely, the mCCA-rreg and sparse LDA models. The mCCA-rreg model extends the classical canonical correlation model to calculate functional connections by restricting the references to a reference space and adding a spatial regularization term. The reference space is used to ensure that the model extracts important components first from several datasets simultaneously by decreasing the importance of the components in which we are uninterested. The spatial regularization term helps in avoiding the multicollinearity and overfitting problems under the low signal-to-noise ratio circumstance. The sparse LDA model extends the classical LDA model to extract a small subset of discriminative classification features by fusing clinical scores. In the real data experiment, we extract two functional connection modes from 45 subjects by the mCCA-rreg model. Then, we construct classifiers to identify the patients with MDD based on the connections selected by the sparse LDA model. The best accuracy is higher than 95%. The results show that the mCCA-rreg model can retrieve the important components characterized by a preassigned reference space and exclude the noise or components of no interest. The sparse LDA model can extract discriminative classification features related to clinical scores.

Efficient representations of EEG signals for SSVEP frequency recognition based on deep multiset CCA

Canonical correlation analysis (CCA) has been widely used for frequency recognition in steady-state visual evoked potential (SSVEP) based brain–computer interfaces (BCIs). However, linear CCA-based methods may be insufficient given the complexity of EEG signals. A nonlinear feature extraction method based on deep multiset CCA (DMCCA) is proposed for SSVEP recognition to fully utilize the real EEG and constructed sine–cosine signals. In DMCCA, neural networks are trained to learn the nonlinear representations of multiple sets of EEG signals at the same frequency by maximizing the overall correlation within the representations and reference signals. Therefore, reference signals are augmented with the extracted features for frequency recognition. Finally, the proposed method is evaluated using SSVEP signals collected from 10 subjects. DMCCA-based method outperforms others in terms of classification accuracy compared with CCA- and multiset CCA-based methods. The proposed DMCCA-based method has substantial potential for improving the recognition performance of SSVEP signals.

Composite nonlinear multiset canonical correlation analysis for multiview feature learning and recognition

SummaryIn this paper, we propose a composite nonlinear multiset canonical correlation projections (CNMCPs) framework where orthogonal constraints are imposed in each set. This makes CNMCP capable of learning uncorrelated low‐dimensional features with minimum redundancy in Hilbert space. With the CNMCP framework, we further present a particular algorithm called multikernel multiset canonical correlations or mKMCC, which introduces different weights into multiple nonlinear functions in all views. An alternating iterative optimization is designed for computational solution. Numerous experimental results on practical datasets have demonstrated the effectiveness and robustness of mKMCC, in contrast with existing kernel correlation learning approaches.

In search of multimodal brain alterations in Alzheimer's and Binswanger's disease.

Structural and functional brain abnormalities have been widely identified in dementia, but with variable replicability and significant overlap. Alzheimer's disease (AD) and Binswanger's disease (BD) share similar symptoms and common brain changes that can confound diagnosis. In this study, we aimed to investigate correlated structural and functional brain changes in AD and BD by combining resting-state functional magnetic resonance imaging (fMRI) and diffusion MRI. A group independent component analysis was first performed on the fMRI data to extract 49 intrinsic connectivity networks (ICNs). Then we conducted a multi-set canonical correlation analysis on three features, functional network connectivity (FNC) between ICNs, fractional anisotropy (FA) and mean diffusivity (MD). Two inter-correlated components show significant group differences. The first component demonstrates distinct brain changes between AD and BD. AD shows increased cerebellar FNC but decreased thalamic and hippocampal FNC. Such FNC alterations are linked to the decreased corpus callosum FA. AD also has increased MD in the frontal and temporal cortex, but BD shows opposite alterations. The second component demonstrates specific brain changes in BD. Increased FNC is mainly between default mode and sensory regions, while decreased FNC is mainly within the default mode domain and related to auditory regions. The FNC changes are associated with FA changes in posterior/middle cingulum cortex and visual cortex and increased MD in thalamus and hippocampus. Our findings provide evidence of linked functional and structural deficits in dementia and suggest that AD and BD have both common and distinct changes in white matter integrity and functional connectivity.

Latent common source extraction via a generalized canonical correlation framework for frequency recognition in SSVEP based brain–computer interfaces

Objective. This study introduces and evaluates a novel target identification method, latent common source extraction (LCSE), that uses subject-specific training data for the enhancement of detection of steady-state visual evoked potential (SSVEP). Approach. LCSE seeks to construct a common latent representation of the SSVEP signal subspace that is stable across multiple trials of electroencephalographic (EEG) data. The spatial filter thus obtained improves the signal-to-noise ratio (SNR) of the SSVEP components by removing nuisance signals that are irrelevant to the generalized signal representation learnt from the given data. In this study a comparison of SSVEP identification performance between the proposed method, extended canonical correlation analysis (ExtCCA) and multiset canonical correlation analysis (MsetCCA) was conducted using SSVEP benchmark data of 40 targets recorded from 35 subjects to validate the effectiveness of the LCSE framework. Main results. The results indicate that the LCSE framework significantly outperforms the other two methods in terms of both classification accuracy and information transfer rates (ITRs). Significance. The significant improvement in the target identification performance demonstrates that the proposed LCSE method can be seen as a promising potential candidate for efficient SSVEP detection in brain–computer interface (BCI) systems.

Transdiagnostic Multimodal Neuroimaging in Psychosis: Structural, Resting-State, and Task Magnetic Resonance Imaging Correlates of Cognitive Control

BackgroundDisorders with psychotic features, including schizophrenia and some bipolar disorders, are associated with impairments in regulation of goal-directed behavior, termed cognitive control. Cognitive control–related neural alterations have been studied in psychosis. However, studies are typically unimodal, and relationships across modalities of brain function and structure remain unclear. Thus, we performed transdiagnostic multimodal analyses to examine cognitive control–related neural variation in psychosis. MethodsStructural, resting, and working memory task imaging for 31 control participants, 27 participants with bipolar disorder, and 23 participants with schizophrenia were collected and processed identically to the Human Connectome Project, enabling identification of relationships with prior multimodal work. Two cognitive control–related independent components (ICs) derived from the Human Connectome Project using multiset canonical correlation analysis with joint IC analysis were used to predict performance in psychosis. De novo multiset canonical correlation analysis with joint IC analysis was performed, and the results were correlated with cognitive control. ResultsA priori working memory and cortical thickness maps significantly predicted cognitive control in psychosis. De novo multiset canonical correlation analysis with joint IC analysis identified an IC correlated with cognitive control that also discriminated groups. Structural contributions included insular and cingulate regions; task contributions included precentral, posterior parietal, cingulate, and visual regions; and resting-state contributions highlighted canonical network organization. Follow-up analyses suggested that correlations with cognitive control were primarily influenced by participants with schizophrenia. ConclusionsA priori and de novo imaging replicably identified a set of interrelated patterns across modalities and the healthy-to-psychosis spectrum, suggesting robustness of these features. Relationships between imaging and cognitive control performance suggest that shared symptomatology may be key to identifying transdiagnostic relationships in psychosis.