Within-class Scatter Matrices Research Articles

Saliency-Based Multilabel Linear Discriminant Analysis.

Linear discriminant analysis (LDA) is a classical statistical machine-learning method, which aims to find a linear data transformation increasing class discrimination in an optimal discriminant subspace. Traditional LDA sets assumptions related to the Gaussian class distributions and single-label data annotations. In this article, we propose a new variant of LDA to be used in multilabel classification tasks for dimensionality reduction on original data to enhance the subsequent performance of any multilabel classifier. A probabilistic class saliency estimation approach is introduced for computing saliency-based weights for all instances. We use the weights to redefine the between-class and within-class scatter matrices needed for calculating the projection matrix. We formulate six different variants of the proposed saliency-based multilabel LDA (SMLDA) based on different prior information on the importance of each instance for their class(es) extracted from labels and features. Our experiments show that the proposed SMLDA leads to performance improvements in various multilabel classification problems compared to several competing dimensionality reduction methods.

Multiparty Data Publishing via Blockchain and Differential Privacy

Data are distributed between different parties. Collecting data from multiple parties for analysis and mining will serve people better. However, it also brings unprecedented privacy threats to the participants. Therefore, safe and reliable data publishing among multiple data owners is an urgent problem to be solved. We mainly study the problem of privacy protection in data publishing. For a centralized scenario, we propose the LDA-DP algorithm. First, the within-class mean vectors and the pooled within-class scatter matrix are perturbed by the Gaussian noise. Second, the optimal projection direction vector with differential privacy is obtained by the Fisher criterion. Finally, the low-dimensional projection data of the original data are obtained. For distributed scenarios, we propose the Mul-LDA-DP algorithm based on a blockchain and differential privacy technology. First, the within-class mean vectors and within-class scatter matrices of local data are perturbed by the Gaussian noise and uploaded to the blockchain network. Second, the projection direction vector is calculated in the blockchain network and returned to the data owner. Finally, the data owner uses the projection direction vector to generate low-dimensional projection data of the original data and upload it to the blockchain network for publishing. Furthermore, in a distributed scenario, we propose a correlated noise generation scheme that uses the additivity of the Gaussian distribution to mitigate the effects of noise and can achieve the same noise level as the centralized scenario. We measure the utility of the published data by the SVM misclassification rate. We conduct comparative experiments with similar algorithms on different real data sets. The experimental results show that the data released by the two algorithms can maintain good utility in SVM classification.

Tri-Direction 2D-Fisher Discriminant Analysis (T2D-FDA) for Feature Extraction

A new image feature extraction method for face recognition called Tri-direction 2D-Fisher Discriminant Analysis (T2D-FDA) is proposed to deal with the Small Sample Size (SSS) problem in conventional 1D-Fisher Discriminant Analysis (1D-FDA). Moreover, the essence of T2D-FDA is investigated, and the equivalence of the left-multiplying 2D-FDA of the original image matrices and the left-multiplying D2D-FDA of diagonal image matrices is verified if each column is viewed as a computational unit. Different from the 1D-LDA based approaches, the T2D-FDA is based on 2D image matrices rather than column vectors, so the image matrix does not need to be transformed into a long vector before feature extraction. The advantage arising in this way is that the SSS problem does not exist anymore because within-class scatter matrices constructed is full-rank in T2D-FDA. The proposed method is applied to face recognition where only few training images exist for each subject. Experiment results show T2D-FDA outperforms the current linear subspace methods in SSS problem on four public databases: ORL, Yale, AR and FERET face databases.

Optimized regularized linear discriminant analysis for feature extraction in face recognition

In a reduced dimensional space, linear discriminant analysis looks for a projective transformation that can maximizes separability among classes. Since linear discriminant analysis demands the within-class scatter matrix appear to non-singular, which cannot directly used in condition of small sample size (SSS) issues in which the dimension of image is much higher, while the number of samples isn’t unlimited. Both the between-class and within-class scatter matrices are always exceedingly ill-posed in SSS problems. And many algorithms are suffered from small sample size issues still. To solve SSS problems, many methods including regularized linear discriminant analysis were proposed. In this article, a way was presented by optimized regularized linear discriminant analysis for feature extraction in FR which can not only fix the singularity problem existing in scatter matrix but also the problem of parameter estimation. The experiment is conducted on several databases and promising results are obtained compared to some state-of-the-art methods to demonstrate the effectiveness of the proposed approach.

Semi-Supervised Tensorial Locally Linear Embedding for Feature Extraction Using PolSAR Data

Recently, the semi-supervised feature extraction, which uses both a few labeled samples and a large quantity of unlabeled samples to improve discrimination of extracted features, has attracted attention in PolSAR data processing. However, most of the feature extraction methods are based on matrix operations and need to re-arrange the multiple features into vectors. Such a vectorization way may break the spatial structure and correlation among the neighboring pixels, which leads to the degradation of classification performance. To address this issue and incorporate the superiority of semi-supervised learning, a novel semi-supervised tensorial locally linear embedding (STLLE) method is proposed for PolSAR data. Each pixel and its spatial neighbors in a local window are represented as a third-order tensor, whose first two orders represent the neighborhood pixels, and the third order denotes the multiple features. The feature extraction is converted to find an optimal projection that can transform the high-dimensional features to low-dimensional ones. This can be solved by designing an objective function, in which the tensorial discriminant item includes the between-class and within-class scatter matrices on each mode of the labeled tensors. A regularizer is based on tensorial locally linear embedding graph using labeled and unlabeled tensor samples. Moreover, the pairwise class constraints are also introduced. Finally, we can use all the extracted features with a classifier such as K-Nearest Neighbor. Experimental results on three real PolSAR datasets demonstrate that STLLE significantly improves classification accuracies compared with state-of-the-art methods, which clearly verifies that the extracted features preserve the data structures well and are more discriminative.

Feature Extraction Using Fractional-Order Embedding Direct Linear Discriminant Analysis

In this paper, a novel LDA-based dimensionality reduction method called fractional-order embedding direct LDA (FEDLDA) is proposed. More specifically, we redefine the fractional-order between-class and within-class scatter matrices which can significantly reduce the deviation of sample covariance matrices caused by the noise disturbance and limited number of training samples; then the novel feature extraction criterion based on the direct LDA (DLDA) and the idea of fractional-order embedding is applied. Experiments on AT&T, Yale and AR face image databases are performed to test and evaluate the effectiveness of the proposed algorithms. Extensive experimental results show that FEDLDA outperforms DLDA and other closely related methods in terms of classification accuracy and efficiency.

RECOGNITION OF LOCAL CAPITAL USING SYMMETRIC TWO-DIMENSIONAL LINEAR DISCRIMINANT ANALYSIS

Statistical pattern recognition is a system that aims to classify a number of objects to a number of categories or classes. Given a data matrix A, A = {Π1, Π2,…, Πk} where Πi consist of ni point data of ith class then patterns in each classes can classify and separate distance of within and between-class in datasets. In this paper, Symmetric Two-Dimensional Linear Discriminant Analysis proposed to maximize the between-class scatter matrices (Sb) and minimize the within-class scatter matrices (Sw), and can recognition the symmetric capital letter by hand writing such as A, B, C, D, E, H, I, K, M, O, S, U, V, W and Y by using ADL2-D algorithm.

Self-weighted discriminative feature selection via adaptive redundancy minimization

Abstract In this paper, a novel self-weighted orthogonal linear discriminant analysis (SOLDA) method is firstly proposed, such that optimal weight can be automatically achieved to balance both between-class and within-class scatter matrices. Since correlated features tend to have similar rankings, multiple adopted criteria might lead to the state that top ranked features are selected with large correlations, such that redundant information is brought about. To minimize associated redundancy, an original regularization term is introduced to the proposed SOLDA problem to penalize the high-correlated features. Different from other methods and techniques, we optimize redundancy matrix as a variable instead of setting it as a priori, such that correlations among all the features can be adaptively evaluated. Additionally, a brand new recursive method is derived to achieve the selection matrix heuristically, such that closed form solution can be obtained with holding the orthogonality. Consequently, self-weighted discriminative feature selection via adaptive redundancy minimization (SDFS-ARM) method can be summarized, such that non-redundant discriminative features could be selected correspondingly. Eventually, the effectiveness of the proposed SDFS-ARM method is further validated by the empirical results.

Block linear discriminant analysis for visual tensor objects with frequency or time information

Recently, due to the advancement of acquisition techniques, visual tensor data have been accumulated in a great variety of engineering fields, e.g., biometrics, neuroscience, surveillance and remote sensing. How to analyze and learn with such tensor objects thus becomes an important and growing interest in machine learning community. In this paper, we propose a block linear discriminant analysis (BLDA) algorithm to extract features for visual tensor objects such as multichannel/hyperspectral face images or human gait videos. Taking the inherent characteristic of such tensor data into account, we unfold tensor objects according to their spatial information and frequency/time information, and represent them in a block matrix form. As a result, the block form between-class and within-class scatter matrices are constructed, and a related block eigen-decomposition is solved to extract features for classification. Comprehensive experiments have been carried out to test the effectiveness of the proposed method, and the results show that BLDA outperforms existing algorithms like DATER, 2DLDA, GTDA, UMLDA, STDA and MPCA for visual tensor object analysis.

Nonnegative Discriminant Matrix Factorization

Nonnegative matrix factorization (NMF), which aims at obtaining the nonnegative low-dimensional representation of data, has received wide attention. To obtain more effective nonnegative discriminant bases from the original NMF, in this paper, a novel method called nonnegative discriminant matrix factorization (NDMF) is proposed for image classification. NDMF integrates the nonnegative constraint, orthogonality, and discriminant information in the objective function. NDMF considers the incoherent information of both factors in standard NMF and is proposed to enhance the discriminant ability of the learned base matrix. NDMF projects the low-dimensional representation of the subspace of the base matrix to regularize the NMF for discriminant subspace learning. Based on the Euclidean distance metric and the generalized Kullback-Leibler (KL) divergence, two kinds of iterative algorithms are presented to solve the optimization problem. The between- and within-class scatter matrices are divided into positive and negative parts for the update rules and the proofs of the convergence are also presented. Extensive experimental results demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art discriminant NMF algorithms.

Low-resolution face recognition with single sample per person

As a growing number of low-resolution (LR) face images are captured by surveillance cameras, LR face recognition has been a hot issue for recent years. Previous efforts on LR face recognition typically assume each subject has multiple high-resolution (HR) training samples. However, this assumption may not hold in some special cases such as law-enforcement where only a single HR sample per person exists in the training set. For LR face recognition in SSPP scenario, it often suffers from overfitting and singular matrix problems. In this paper, we are the first to investigate LR face recognition with single sample per person, and propose a cluster-based regularized simultaneous discriminant analysis (C-RSDA) method based on SDA. C-RSDA regularizes the between-class and within-class scatter matrices respectively with inter-cluster and intra-cluster scatter matrices, where the cluster-based scatter matrices are computed from unsupervised clustering. With the cluster-based scatter matrices, not only the singularity problem is resolved, but overfitting problem is overcomed as more variations are exploited from the limited training samples. Thus, the proposed C-RSDA enhances the discriminative power of the feature subspace. We extensively evaluate C-RSDA on recognizing LR face images captured in both controlled and uncontrolled environments. The encouraging experimental results demonstrate the effectiveness of the proposed approach.

Separability-Oriented Subclass Discriminant Analysis.

Linear discriminant analysis (LDA) is a classical method for discriminative dimensionality reduction. The original LDA may degrade in its performance for non-Gaussian data, and may be unable to extract sufficient features to satisfactorily explain the data when the number of classes is small. Two prominent extensions to address these problems are subclass discriminant analysis (SDA) and mixture subclass discriminant analysis (MSDA). They divide every class into subclasses and re-define the within-class and between-class scatter matrices on the basis of subclass. In this paper we study the issue of how to obtain subclasses more effectively in order to achieve higher class separation. We observe that there is significant overlap between models of the subclasses, which we hypothesise is undesirable. In order to reduce their overlap we propose an extension of LDA, separability oriented subclass discriminant analysis (SSDA), which employs hierarchical clustering to divide a class into subclasses using a separability oriented criterion, before applying LDA optimisation using re-defined scatter matrices. Extensive experiments have shown that SSDA has better performance than LDA, SDA and MSDA in most cases. Additional experiments have further shown that SSDA can project data into LDA space that has higher class separation than LDA, SDA and MSDA in most cases.

A Novel Generalized Fuzzy Canonical Correlation Analysis Framework for Feature Fusion and Recognition

In this paper, a novel CCA-based dimensionality reduction method called generalized fuzzy canonical correlation analysis (GFCCA) is proposed. GFCCA combines the generalized canonical correlation analysis and fuzzy set theory. GFCCA redefines the fuzzy between-class and within-class scatter matrices that relate directly to the samples distribution information. For nonlinear separated problems, we extend the kernel extension of GFCCA with positive definite kernels and indefinite kernels. Experiments on real-world data sets are performed to test and evaluate the effectiveness of the proposed algorithms.

Probability Maximum Margin Criterion for Crop Disease Recognition

<abstract> <sc><b>Abstract.</b> </sc>Crop disease seriously affects the yield and quality of crops and cause farmers to experience economic losses. Automatic detection of crop disease is an essential research topic to detect the disease symptoms as soon as they appear on the leaves. For crop disease management, it is important to automatically detect the crop diseases at a stage so as to treat them properly. In subspace dimension reduction-based data classification methods, the main topic is to design the between-class and within-class scatter matrices to obtain the mapping matrix from the training samples. But, there is much difference between the training samples. To eliminate the difference in designing the scatter matrices, in this article, based on the prior probability, a probability maximum margin criterion (MMC) is proposed for leaf spot image processing. Compared to the traditional MMC algorithm, in the proposed method, the two scatter matrices are expressed by the weighted mean vector of all training samples, which can overcome the influence of the outliers and the noise points, meanwhile improve the crop disease recognition rate. In this study, more than 100 classifying features are extracted from the disease leaf images of two kinds of cucumber diseases; probability MMC is performed for reduced dimensions in feature data processing, and then K-nearest-neighbor classifier is used to identify the cucumber diseases. The experimental results on a cucumber disease leaf image database show that the proposed method is effective for crop disease recognition.

Dual structural consistency based multi-modal correlation propagation projections for data representation

Canonical correlation analysis (CCA) is a powerful tool for analyzing multi-dimensional paired data. However, when facing semi-supervised multi-modal data (Also called multi-view Hou et al. (Pattern Recog 43(3):720–730, 2010) or multi-represented Kailing et al. (Clustering multi-represented objects with noise. In: Proceedings of the eighth Pacific-Asia conference on knowledge discovery and data mining (PAKDD). Sydney, Australia, pp 394–403) data. For convenience, we will uniformly call them multi-modal data hereafter.) which widely exist in real-world applications, CCA usually performs poorly due to ignoring useful supervised information. Meanwhile, due to the limited labeled training samples in the semi-supervised scenario, supervised extensions of CCA suffer from overfitting. Several semi-supervised extensions of CCA have been proposed recently. Nevertheless, they either just utilize the global structural information captured from the unlabeled data, or propagate label information by discovering the affinities just between the labeled and unlabeled data points in advance. In this paper, we propose a robust multi-modal semi-supervised feature extraction and fusion framework, termed as dual structural consistency based multi-modal correlation propagation projections (SCMCPP). SCMCPP guarantees the consistency between representation structure and hypotaxis structure in each modality and ensures the consistency of hypotaxis structure between two different modalities. By iteratively propagating labels and learning affinities, discriminative information of both given labels and estimated labels is utilized to improve the affinity construction and infer the remaining unknown labels. Moreover, probabilistic within-class scatter matrices in each modality and probabilistic correlation matrix between two modalities are constructed to enhance the discriminative power of features. Extensive experiments on several benchmark face databases demonstrate the effectiveness of our approach.

A new face recognition method via semi-discrete decomposition for one sample problem

Fisher linear discriminant analysis (FLDA) is a popular approach which has been widely used for feature extraction in face recognition tasks. FLDA is seeking optimal projection vectors by maximizing the ratio of between-class scatter matrices vs. within-class scatter matrices. However, the within-class scatter matrices cannot be calculated directly when one training sample each object is available. In this case, image decomposition plays an important role in the reconstructed within-class scatter matrices. So, the performance of image decomposition is crucial. In this paper, a based on semi-discrete decomposition (SDD) method is proposed to solve single training sample image per person problem. First, the original single image and its transpose are calculated via using SDD method in the training set. Then, the within-class scatter matrices are constituted by using the original image and its two approximation images of each person. And meanwhile, the optimal projection vectors are obtained via the FLDA algorithm in the new training set. Finally, we use the nearest neighbor classifier to complete the final classification. The performance of proposed method is evaluated on ORL, Yale and FERET databases. The experimental results show that the proposed method is outperforms singular value decomposition (SVD)-based and QR decomposition with column pivoting (QRCP)-based methods in terms of recognition rates.

A Class-Information-Based Sparse Component Analysis Method to Identify Differentially Expressed Genes on RNA-Seq Data.

With the development of deep sequencing technologies, many RNA-Seq data have been generated. Researchers have proposed many methods based on the sparse theory to identify the differentially expressed genes from these data. In order to improve the performance of sparse principal component analysis, in this paper, we propose a novel class-information-based sparse component analysis (CISCA) method which introduces the class information via a total scatter matrix. First, CISCA normalizes the RNA-Seq data by using a Poisson model to obtain their differential sections. Second, the total scatter matrix is gotten by combining the between-class and within-class scatter matrices. Third, we decompose the total scatter matrix by using singular value decomposition and construct a new data matrix by using singular values and left singular vectors. Then, aiming at obtaining sparse components, CISCA decomposes the constructed data matrix by solving an optimization problem with sparse constraints on loading vectors. Finally, the differentially expressed genes are identified by using the sparse loading vectors. The results on simulation and real RNA-Seq data demonstrate that our method is effective and suitable for analyzing these data.

A Hyper spectral Images Classification Method Based on Maximum Scatter Discriminant Analysis

To overcome “small sample size problem” problem faced by some hyper spectral classification methods, the Maximum Scatter Discriminant criterion is used to analyzed hyperspectral data. Maximum Scatter Discriminant analysis searches for the project axes by maximizing the difference of between-class scatter and within-class scatter matrices, which avoid to calculate the inverse of matrices. Experiment results on Indian Pines HSI data set show that the proposed method outperforms the other methods in terms of recognition accuracy. The proposed method is an effective and feasible method for hyper pectral data classification.

Unconstrained face verification with a dual-layer block-based metric learning

In this paper, a dual-layer block-based metric learning technique is proposed to better discriminate the face image pairs and accelerate the overall verification process under the unconstrained environment. The input images are processed as blocks to provide a richer base of face features. Our proposed method is formed by two layers, in which the first layer assists in extracting the compact block-based descriptors without the existence of full class label information and to refine the within-class and between-class scatter matrices while the second layer integrates the face descriptors of all blocks. The proposed scheme has computational advantage over the single metric learning method while it exploits the correlations among the multiple metrics from different descriptors. The performance of our proposed method is evaluated on the Labeled Faces in the Wild database and achieves an improved performance when compared with the state-of-the-art methods in terms of verification rate and computation time.

Phone classification via manifold learning based dimensionality reduction algorithms

Mechanical limitations imposed on the articulators during speech production lead to a limitation of the intrinsic dimensionality of speech signals. This limitation leads to a specific neighborhood structure of speech sounds when they are represented in a high-dimensional feature space. We investigate whether phone classification can be improved by exploiting this neighborhood structure, by means of extended variants of the conventional Linear Discriminant Analysis (LDA) based on manifold learning.In this extended LDA approach, the within-class and between-class scatter matrices are defined in terms of adjacency graphs. We compare extensions of LDA that use either a full adjacency graph or an adjacency graph defined in the neighborhood of the training observations. In addition, we apply different kernels for weighing the distances in the graphs via different kernels, of which the Adaptive Kernel is proposed in this paper.Experiments with TIMIT show that while LDA algorithms that use the full adjacency graph do not outperform traditional LDA, the algorithms that exploit only local information provide significantly better results than traditional LDA. These improvements are not uniform across different broad phonetic classes, which suggests that the added value of the neighborhood structure is phone class dependent. The structure is represented by locally different densities in the neighborhood of feature vectors that are representative of a specific phone in a specific context.