Multi-view Discriminant Analysis Research Articles

Multi-view robust regression for feature extraction

Recently, Multi-view Discriminant Analysis (MVDA) has been proposed and achieves good performance in multi-view recognition tasks. However, as an extension of LDA, this method still suffers from the small-class problem and has the sensitivity to outliers. In order to address these drawbacks and achieve better performance on multi-view recognition tasks, we proposed Multi-view Robust Regression (MVRR) for multi-view feature extraction. MVRR is a regression based method that imposes L2,1 norm as the metric of the loss function and the regularization term to improve robustness and obtain jointly sparse projection matrices for effective feature extraction. Moreover, we incorporate an orthogonal matrix to regress the extracted features to their scaled label to avoid the small-class problem. Therefore, MVRR guarantees the projection matrix to break through the restriction of the number of class for solving the small-class problem. We also propose an iterative algorithm to compute the optimal solution of MVRR and the convergence of MVRR is proved. Experiments are conducted on four databases to verify the performance of MVRR and the result illustrates that MVRR is robust on multi-view feature extraction.

Creating Universum for class imbalance via locality and its application in multiview subspace learning

Since the existing methods for generating Universum fail in class imbalance cases, we propose a new method for creating Universum via locality and betweenness, named L-CIBU. Compared to existing methods for generating Universum, L-CIBU not only generates appropriate Universum in the case of class imbalance but also has higher efficiency. Given the existence of class imbalance in multiview learning, we introduce Universum to multiview subspace learning (MvSL) to further enhance its performance and L-CIBU happens to be able to overcome this problem. As far as we know, this is the first time to introduce Universum into MvSL with more than two views. Specifically, we introduce Universum created by L-CIBU into multiview discriminant analysis (MvDA) and propose Universum MvDA (UMvDA). By analyzing several randomly generated imbalanced datasets, it is intuitively demonstrated that L-CIBU is more effective than FIBU and CIBU in the case of class imbalance. The experimental results on two real datasets show that the time required for creating the Universum using L-CIBU is 25%∼95% less than that of FIBU and CIBU. Moreover, UMvDA achieves an accuracy improvement of approximately 1%∼6% compared to MvDA, Universum canonical correlation analysis (UCCA), and Universum discriminant canonical correlation analysis (UDCCA).

Similarity-sequenced multi-view discriminant feature extraction for image recognition

Traditional multi-view feature extraction methods based on manifold learning frequently overlook the similarity sequence between samples, failing to capture the intrinsic manifold structure of raw nonlinear samples and restricting the recognition performance of multi-view learning. In this paper, we propose a novel similarity-sequenced multi-view discriminant feature extraction method, called Similarity -sequenced Multi-view Discriminant Correlation Analysis (SMDCA), which explicitly considers the sample sequences based on similarity. The method constructs similarity-sequenced discriminant scatters for preserving the sequence structure of within-class samples and develops between-class correlations with the similarity-sequence structure information for further constraining intrinsic manifold structure of cross-view samples. SMDCA can also simultaneously extract low-dimensional sequence features with well-discriminative power from multiple views. Extensive experiments exhibit that SMDCA can provide higher recognition accuracy and stronger robustness in image recognition tasks.

Multiview Jointly Sparse Discriminant Common Subspace Learning

Multiview data leads to the demand for classifying samples from various views, and the large gap between different views makes the classification task challenging. Recently, researchers have extended linear discriminant analysis (LDA) to multi-view scenarios. However, the extended methods are generally associated with the small-class problem, that is, the projection size is limited by the number of classes. In addition, they are sensitive to variations in images or outliers. To solve these problems, this study proposes a generalized robust multiview discriminant analysis (GRMDA) to obtain a linear transform for each view and for learning multiview jointly sparse discriminant common subspace. GRMDA aims to achieve both maximal between-class and minimal within-class variation for data from multiple views in a common space. Instead of formulating the ratio trace problem, we reformulate GRMDA inspired by maximum margin criterion (MMC) to address the small-class problem. Moreover, the proposed method achieves stronger robustness by reconstructing the within-class and between-class scatter terms from the definition of L2,1 norm. Furthermore, GRMDA ensures joint sparsity using the L2,1 norm-based regularization term. Additionally, we present an iterative algorithm, convergence proof, and complexity analysis. Experiments on six popular databases, that is, COIL100, USPS/MNIST, Extended Yale Face B, AR, BBCSport, and multiple feature datasets, were conducted to evaluate the performance of GRMDA against the state-of-the-art multiview methods. The experimental results demonstrate that the proposed method can achieve a significant performance with strong robustness and fast convergence.

Orthogonal multi-view analysis by successive approximations via eigenvectors

Orthogonality has been demonstrated to admit many desirable properties such as noise-tolerant, good for data visualization, and preserving distances. However, it is often incompatible with existing models and the resulting optimization problem is challenging even if compatible. To address these issues, we propose a trace ratio formulation for multi-view subspace learning to learn individual orthogonal projections for all views. The proposed formulation integrates the correlations within multiple views, supervised discriminant capacity, and distance preservation in a concise and compact way. It not only includes several existing models as special cases, but also inspires new models. Moreover, an efficient numerical method based on successive approximations via eigenvectors is presented to solve the associated optimization problem. The method is built upon an iterative Krylov subspace method which can easily scale up for high-dimensional datasets. Extensive experiments are conducted on various real-world datasets for multi-view discriminant analysis and multi-view multi-label classification. The experimental results demonstrate that the proposed models are consistently competitive to and often better than the compared methods.

Trace ratio criterion for multi-view discriminant analysis

Trace ratio criterion for multi-view discriminant analysis

Robust multi-view discriminant analysis with view-consistency

Multi-view discriminant analysis (MvDA) is a successful method to learn a single discriminant common space of multiple views. However, MvDA may encounter the robustness issue theoretically because of using F-norm as the metric. In this paper, a robust multi-view discriminant analysis with view-consistency is proposed by employing L1-norm as the metric, called as L1-MvDA-VC. where both inter-view and intra-view distances are characterized by L1-norm. The proposed L1-MvDA-VC not only can obtain discriminant common space, but also is robust to outliers. In addition, a simple and effective learning algorithm is designed for L1-MvDA-VC, and its convergence is proved theoretically. Experimental results on real data sets demonstrate that L1-MvDA-VC has better performance on contaminated data than that of MvDA, MvDA-VC, MvCCDA, and MULDA.

2D Multi-view Discriminant Analysis

With the advent of technology, we have seen an explosion in high-definition images. However, due to the large size of images, 1D methods are less suitable to handle such data. This issue becomes even more pronounced when working with multi-view datasets. We propose a generalized method, 2D Multi-view Discriminant Analysis (2DMvDA), to tackle this issue. 2DMvDA is a 2D multi-view classification method based on discriminant analysis. It uses the 2D image matrices directly instead of extracting 1D features from them, leading to a considerable reduction in the size of the data to be processed. We compare the proposed method with 1D Multi-view Discriminant Analysis (MvDA) and single-view 2D Linear Discriminant Analysis (2DLDA). We experimentally show that the proposed method requires less than 0.1% training time and memory than MvDA. Also, though it needs the same memory, it requires 30–70% less training time than 2DLDA. Using less time and memory, 2DMvDA achieves a classification accuracy that is at par or better than both of these methods.

Multi-view discriminant analysis with sample diversity for ECG biometric recognition

Currently, electrocardiogram (ECG) biometric recognition is a novel research trend, and many methods have been developed. Due to the influence of physical and psychological activities, there are heartbeats diversities of the same person. However, the existing ECG biometric recognition methods do not make use of sample diversity information. In this paper, we present a multi-view discriminant analysis approach in the consideration of sample diversity for ECG biometric recognition. Firstly, we propose a method of generating multiple views by using single lead ECG signal. Secondly, we present a multi-views learning framework, which takes sample diversity into account to generate a more discriminative subspace. Thirdly, to obtain a more robust solution, we introduce a denoising constraint to learn the relationships between different views, which can create a stable representation against ECG noise. At last, experimental results demonstrate that compared with the state-of-the-art methods on four databases, the proposed method can achieve competitive performance compared to state-of-the-art ECG biometric recognition methods.

Pairwise-Covariance Multi-view Discriminant Analysis for Robust Cross-View Human Action Recognition

Human action recognition (HAR) under different camera viewpoints is the most critical requirement for practical deployment. In this paper, we propose a novel method that leverages successful deep learning-based features for action representation and multi-view analysis to accomplish robust HAR under viewpoint changes. Specifically, we investigate various deep learning techniques, from 2D CNNs to 3D CNNs to capture spatial and temporal characteristics of actions at each separated camera view. A common feature space is then constructed to keep view-invariant features among extracted streams. This is carried out by learning a set of linear transformations that project private features into the common space in which the classes are well distinguished from each other. To this end, we first adopt Multi-view Discriminant Analysis (MvDA). The original MvDA suffers from odd situations in which the most class-discrepant common space could not be found because its objective is overly concentrated on pushing classes from the global mean but unaware of the distance between specific pairs of adjoining classes. We then introduce a pairwise-covariance maximizing extension that takes pairwise distances between classes into account, namely pc-MvDA. The novel method also differs in the way that could be more favorably applied for large high-dimensional multi-view datasets. Extensive experimental results on four datasets (IXMAS, MuHAVi, MICAGes, NTU RGB+D) show that pc-MvDA achieves consistent performance gain, especially for harder classes. The code is publicly available for research purpose at https://github.com/inspiros/pcmvda .

Multi-view Incremental Discriminant Analysis

In recent years, the use of multi-view data has attracted much attention resulting in many multi-view batch learning algorithms. However, these algorithms prove expensive in terms of training time and memory when used on the incremental data. In this paper, we propose Multi-view Incremental Discriminant Analysis (MvIDA), which updates the trained model to incorporate new data samples. MvIDA requires only the old model and newly added data to update the model. Depending on the nature of the increments, MvIDA is presented as two cases, sequential MvIDA and chunk MvIDA. We have compared the proposed method against the batch Multi-view Discriminant Analysis (MvDA) for its discriminability, order independence, the effect of the number of views, training time, and memory requirements. We have also compared our method with single-view Incremental Linear Discriminant Analysis (ILDA) for accuracy and training time. The experiments are conducted on four datasets with a wide range of dimensions per view. The results show that through order independence and faster construction of the optimal discriminant subspace, MvIDA addresses the issues faced by the batch multi-view algorithms in the incremental setting.

Uncorrelated Locality-Sensitive Multi-view Discriminant Analysis

Recently, multi-view feature learning technique has attracted lots of research interest. Discriminant analysis-based multi-view feature learning is an important research branch. Although some multi-view discriminant analysis methods have been presented, there still exists room for improvement. How to effectively explore the discriminant and local geometrical structure information simultaneously from multiple views is still an important research topic. In this paper, we propose a novel approach named uncorrelated locality-sensitive multi-view discriminant analysis, which jointly learns multiple view-specific transformations, such that in the projected subspace for each view, the within-class nearby samples are close to each other, while the between-class nearby samples are far apart. We provide a multi-view sample distance term to promote the one-to-one data consistency across views. Furthermore, we design uncorrelated constraints to reduce the redundancy among the transformations. Experiments on two widely used datasets demonstrate the effectiveness of the proposed approach.

Fractional generalized multiview discriminative projections and extension for multiview learning

In multi-view learning, there are a large number of small sample size problems in which the number of training samples is less than the dimension of feature vectors. For such problems, generalized multi-view linear discriminant analysis (GMLDA) usually fails to achieve good learning performance for many classification tasks. In this paper, we propose a new multi-view feature extraction method via fractional spectral modeling, namely, fractional generalized multi-view discriminant analysis (FGMDA), which is able to subsume GMLDA as a special case. In addition, an uncorrelated FGMDA (UFGMDA) is further proposed, where the extracted features are mutually uncorrelated in each view and thus have minimum redundancy. Extensive experimental results on visual recognition demonstrate the superiority of our proposed methods over the state-of-the-art methods in terms of classification accuracy.

Multi-view uncorrelated discriminant analysis via dependence maximization

This paper proposes a novel multi-view discriminant analysis based on Hilbert-Schmidt Independence Criterion (HSIC) and canonical correlation analysis (CCA). We use HSIC to identify a lower dimensional discriminant common subspace in which the dependence between multi-view features and the associated labels is maximized. CCA is utilized to achieve maximum correlation between different views in the common subspace. Motivated by the successful application of uncorrelated discriminant analysis, we further extend our approach to extract features with minimum redundancy. Experimental results validate the effectiveness of our proposed approaches.

Local feature based multi-view discriminant analysis

In many real-world applications, an object can be represented from multiple views or styles. Thus, it is important to design algorithms that are able to recognize objects from distinct views. To the end, a large number of approaches have been proposed to achieve the heterogeneous recognition tasks through the use of local features. However, most of them only focus on binary views and thus cannot be applied to multi-view analysis. In this paper, we propose a novel local feature based multi-view discriminant analysis approach (FMDA). The proposed approach consists of three steps: First, the input images are represented using representation matrices and local feature descriptor (LFD) matrices of their overlapping patches, where the representation matrices are the linear coefficients of the LFDs for different views. In this way, it brings two advantages, i.e., addressing the small sample size (SSS) problem and preserving the discriminative information while reducing the redundant information in the LFD matrices. Second, the multi-view discriminant representation and feature projections are learned by projecting the LFDs of different views into a common space using the Fisher criterion. Finally, a simple but effective view-similarity constraint is proposed to adaptively learn the relationships between different views. To verify the effectiveness of the proposed method, extensive experiments are carried out on the FERET, CAS-PEAL-R1, CUFSF and HFB databases comparing with some state-of-the-art methods.

Maximal granularity structure and generalized multi-view discriminant analysis for person re-identification

This paper proposes a novel descriptor called Maximal Granularity Structure Descriptor (MGSD) for feature representation and an effective metric learning method called Generalized Multi-view Discriminant Analysis based on representation consistency (GMDA-RC) for person re-identification (Re-ID). The proposed descriptor of MGSD captures rich local structural information from overlapping macro-pixels in an image, analyzes the horizontal occurrence of multi-granularity and maximizes the occurrence to extract a robust representation for viewpoint changes. As a result, the proposed descriptor of MGSD can obtain rich person appearance whilst being robust against different condition changes. Besides, considering multi-view information, we present a new GMDA-RC for different views, inspired by the observation that different views share similar data structures. The proposed metric learning method of GMDA-RC seeks multiple discriminant common spaces for multiple views by jointly learning multiple view-specific linear transforms. Finally, we evaluate the proposed method of (MGSD+GMDA-RC) on three publicly available person Re-ID datasets: VIPeR, CUHK-01 and Wide Area Re-ID dataset (WARD). For the VIPeR and CUHK-01, the experimental results show that our method significantly outperforms the state-of-the-art methods, achieving the rank-1 matching rates of 67.09%, 70.61%, and the improvements of 17.41%, 5.34%, respectively. For the WARD, we consider different pairwise camera views (camera 1–2, camera 1–3, camera 2–3) and our method can achieve the rank-1 matching rates of 64.33%, 59.42%, 70.32%, increasing of 5.68%, 11.04%, 9.06% compared with the state-of-the-art methods, respectively.

Multi-View Discriminant Analysis

In many computer vision systems, the same object can be observed at varying viewpoints or even by different sensors, which brings in the challenging demand for recognizing objects from distinct even heterogeneous views. In this work we propose a Multi-view Discriminant Analysis (MvDA) approach, which seeks for a single discriminant common space for multiple views in a non-pairwise manner by jointly learning multiple view-specific linear transforms. Specifically, our MvDA is formulated to jointly solve the multiple linear transforms by optimizing a generalized Rayleigh quotient, i.e., maximizing the between-class variations and minimizing the within-class variations from both intra-view and inter-view in the common space. By reformulating this problem as a ratio trace problem, the multiple linear transforms are achieved analytically and simultaneously through generalized eigenvalue decomposition. Furthermore, inspired by the observation that different views share similar data structures, a constraint is introduced to enforce the view-consistency of the multiple linear transforms. The proposed method is evaluated on three tasks: face recognition across pose, photo versus. sketch face recognition, and visual light image versus near infrared image face recognition on Multi-PIE, CUFSF and HFB databases respectively. Extensive experiments show that our MvDA achieves significant improvements compared with the best known results.

Multiview Uncorrelated Discriminant Analysis.

Multiview learning is more robust than single-view learning in many real applications. Canonical correlation analysis (CCA) is a popular technique to utilize information stemming from multiple feature sets. However, it does not exploit label information effectively. Later multiview linear discriminant analysis (MLDA) was proposed through combining CCA and linear discriminant analysis (LDA). Due to the successful application of uncorrelated LDA (ULDA), which seeks optimal discriminant features with minimum redundancy, we propose a new supervised learning method called multiview ULDA (MULDA) in this paper. This method combines the theory of ULDA with CCA. Then we adapt discriminant CCA (DCCA) instead of the CCA in MLDA and MULDA, and discuss about the effect of this modification. Furthermore, we generalize these methods to the nonlinear case by kernel-based learning techniques. The new method is called kernel multiview uncorrelated discriminant analysis (KMUDA). Then we modify kernel multiview discriminant analysis and KMUDA by replacing Kernel CCA with Kernel DCCA. Our methods are tested on different real datasets and compared with other state-of-the-art methods. Experimental results validate the effectiveness of our methods.

Intra-View and Inter-View Supervised Correlation Analysis for Multi-View Feature Learning

Multi-view feature learning is an attractive research topic with great practical success. Canonical correlation analysis (CCA) has become an important technique in multi-view learning, since it can fully utilize the inter-view correlation. In this paper, we mainly study the CCA based multi-view supervised feature learning technique where the labels of training samples are known. Several supervised CCA based multi-view methods have been presented, which focus on investigating the supervised correlation across different views. However, they take no account of the intra-view correlation between samples. Researchers have also introduced the discriminant analysis technique into multi-view feature learning, such as multi-view discriminant analysis (MvDA). But they ignore the canonical correlation within each view and between all views. In this paper, we propose a novel multi-view feature learning approach based on intra-view and inter-view supervised correlation analysis (I2SCA), which can explore the useful correlation information of samples within each view and between all views. The objective function of I2SCA is designed to simultaneously extract the discriminatingly correlated features from both inter-view and intra-view. It can obtain an analytical solution without iterative calculation. And we provide a kernelized extension of I2SCA to tackle the linearly inseparable problem in the original feature space. Four widely-used datasets are employed as test data. Experimental results demonstrate that our proposed approaches outperform several representative multi-view supervised feature learning methods.