Scatter Matrices Research Articles

Scaling Up Class-Specific Kernel Discriminant Analysis for Large-Scale Face Verification

In this paper, a novel approximate solution of the criterion used in non-linear class-specific discriminant subspace learning is proposed. We build on the class-specific kernel spectral regression method, which is a two-step process formed by an eigenanalysis step and a kernel regression step. Based on the structure of the intra-class and out-of-class scatter matrices, we provide a fast solution for the first step. For the second step, we propose the use of approximate kernel space definitions. We analytically show that the adoption of randomized and class-specific kernels has the effect of regularization and Nystrom-based approximation, respectively. We evaluate the proposed approach in face verification problems and compare it with the existing approaches. Experimental results show the effectiveness and efficiency of the proposed approximate class-specific kernel spectral regression method, since it can provide satisfactory performance and scale well with the size of the data.

Design and Implementation of Resampling Techniques for Face Recognition using Classical LDA Algorithm in MATLAB

A number of applications require robust human face recognition under varying environmental lighting conditions and different facial expressions, which considerably vary the appearance of human face. However, in many face recognition applications, only a small number of training samples for each subject are available; these samples are not able to capture all the facial appearance variations, to utilize the resampling techniques to generate several subsets of samples from the original training dataset. A classic appearance-based recognizer, LDA-based classifier, is applied to each of the generated subsets to construct a LDA representation for face recognition. The discrimination power of various human facial features is studied and a new scheme for automatic face recognition (AFR) is proposed. This paper focuses on the linear discriminant analysis (LDA) of different aspects of human faces in the spatial as well as in the wavelet domain. The LDA of faces also provides us with a small set of features that carry the most relevant information for classification purposes. The features are obtained through eigenvector analysis of scatter matrices with the objective of maximizing between-class variations and minimizing within-class variations. The result is an efficient projection-based feature-extraction and classification scheme for AFR. Multisource data analysis are used to provide more reliable recognition results. For a medium-sized database of human faces, excellent classification accuracy is achieved with the use of very-lowdimensional feature vectors. Moreover, the method is applicable to many other image-recognition tasks.

Dual structural consistency based multi-modal correlation propagation projections for data representation

Canonical correlation analysis (CCA) is a powerful tool for analyzing multi-dimensional paired data. However, when facing semi-supervised multi-modal data (Also called multi-view Hou et al. (Pattern Recog 43(3):720–730, 2010) or multi-represented Kailing et al. (Clustering multi-represented objects with noise. In: Proceedings of the eighth Pacific-Asia conference on knowledge discovery and data mining (PAKDD). Sydney, Australia, pp 394–403) data. For convenience, we will uniformly call them multi-modal data hereafter.) which widely exist in real-world applications, CCA usually performs poorly due to ignoring useful supervised information. Meanwhile, due to the limited labeled training samples in the semi-supervised scenario, supervised extensions of CCA suffer from overfitting. Several semi-supervised extensions of CCA have been proposed recently. Nevertheless, they either just utilize the global structural information captured from the unlabeled data, or propagate label information by discovering the affinities just between the labeled and unlabeled data points in advance. In this paper, we propose a robust multi-modal semi-supervised feature extraction and fusion framework, termed as dual structural consistency based multi-modal correlation propagation projections (SCMCPP). SCMCPP guarantees the consistency between representation structure and hypotaxis structure in each modality and ensures the consistency of hypotaxis structure between two different modalities. By iteratively propagating labels and learning affinities, discriminative information of both given labels and estimated labels is utilized to improve the affinity construction and infer the remaining unknown labels. Moreover, probabilistic within-class scatter matrices in each modality and probabilistic correlation matrix between two modalities are constructed to enhance the discriminative power of features. Extensive experiments on several benchmark face databases demonstrate the effectiveness of our approach.

Fuzzy clustering-based feature extraction method for mental task classification

A brain computer interface (BCI) is a communication system by which a person can send messages or requests for basic necessities without using peripheral nerves and muscles. Response to mental task-based BCI is one of the privileged areas of investigation. Electroencephalography (EEG) signals are used to represent the brain activities in the BCI domain. For any mental task classification model, the performance of the learning model depends on the extraction of features from EEG signal. In literature, wavelet transform and empirical mode decomposition are two popular feature extraction methods used to analyze a signal having non-linear and non-stationary property. By adopting the virtue of both techniques, a theoretical adaptive filter-based method to decompose non-linear and non-stationary signal has been proposed known as empirical wavelet transform (EWT) in recent past. EWT does not work well for the signals having overlapped in frequency and time domain and failed to provide good features for further classification. In this work, Fuzzy c-means algorithm is utilized along with EWT to handle this problem. It has been observed from the experimental results that EWT along with fuzzy clustering outperforms in comparison to EWT for the EEG-based response to mental task problem. Further, in case of mental task classification, the ratio of samples to features is very small. To handle the problem of small ratio of samples to features, in this paper, we have also utilized three well-known multivariate feature selection methods viz. Bhattacharyya distance (BD), ratio of scatter matrices (SR), and linear regression (LR). The results of experiment demonstrate that the performance of mental task classification has improved considerably by aforesaid methods. Ranking method and Friedman’s statistical test are also performed to rank and compare different combinations of feature extraction methods and feature selection methods which endorse the efficacy of the proposed approach.

The use of a common location measure in the invariant coordinate selection and projection pursuit

Invariant coordinate selection (ICS) and projection pursuit (PP) are two methods that can be used to detect clustering directions in multivariate data by optimizing criteria sensitive to non-normality. In particular, ICS finds clustering directions using a relative eigen-decomposition of two scatter matrices with different levels of robustness; PP is a one-dimensional variant of ICS. Each of the two scatter matrices includes an implicit or explicit choice of location. However, when different measures of location are used, ICS and PP can behave counter-intuitively. In this paper we explore this behavior in a variety of examples and propose a simple and natural solution: use the same measure of location for both scatter matrices.

Early fault detection and diagnosis in bearings for more efficient operation of rotating machinery

Early fault detection and diagnosis plays an increasingly important role in various energy systems where it is critical to avoid deteriorating condition, degraded efficiency and unexpected failures. Rolling-element bearings are among the most common components of rotating machinery used for transformation of energy. Mechanical wear and defective bearings cause rotating machinery to decrease its efficiency, and thus increase energy consumption. A new technique for early fault detection and diagnosis in rolling-element bearings based on vibration signal analysis is presented. After normalization and the wavelet transform of vibration signals, the standard deviation as a measure of average energy and the logarithmic energy entropy as a measure of the degree of disorder are extracted in sub-bands of interest as representative features. Then the feature space dimension is optimally reduced to two using scatter matrices. In the reduced two-dimensional feature space the fault detection and diagnosis is performed by quadratic classifiers. Accuracy of the new technique was tested on four classes of the recorded vibrations signals, i.e. normal, with the fault of inner race, outer race and balls operation. An overall accuracy of 100% was achieved. The new technique will be further tested and implemented in a real production environment.

Local graph embedding based on maximum margin criterion via fuzzy set

Recently, many algorithms based on locally graph embedding are proposed for dimensional reduction in nonlinear data. However, these algorithms are not effective when dealing with face images affected by variations in illumination conditions, poses or perspectives and different facial expressions. So, distant data points are not deemphasized efficiently by locally graph embedding algorithms and it may degrade the performance of classification. In order to solve the aforementioned problem, this paper proposes a new efficient dimension reduction method—local graph embedding method based on maximum margin criterion via Fuzzy Set for face recognition. Firstly, the goal of this algorithm is preserved under nearest neighbor premise by constructing the fuzzy intrinsic graph and the fuzzy penalty graph. Secondly, two novel fuzzy Laplacian scatter matrices are calculated using Fuzzy K-Nearest Neighbor (FKNN) in the proposed method. Finally, Maximum Margin Criterion (MMC) is used to avoid the “small size sample” problem. The results of face recognition experiments on the ORL, YALE and AR face databases demonstrate the effectiveness of our proposed method.

A new face recognition method via semi-discrete decomposition for one sample problem

Fisher linear discriminant analysis (FLDA) is a popular approach which has been widely used for feature extraction in face recognition tasks. FLDA is seeking optimal projection vectors by maximizing the ratio of between-class scatter matrices vs. within-class scatter matrices. However, the within-class scatter matrices cannot be calculated directly when one training sample each object is available. In this case, image decomposition plays an important role in the reconstructed within-class scatter matrices. So, the performance of image decomposition is crucial. In this paper, a based on semi-discrete decomposition (SDD) method is proposed to solve single training sample image per person problem. First, the original single image and its transpose are calculated via using SDD method in the training set. Then, the within-class scatter matrices are constituted by using the original image and its two approximation images of each person. And meanwhile, the optimal projection vectors are obtained via the FLDA algorithm in the new training set. Finally, we use the nearest neighbor classifier to complete the final classification. The performance of proposed method is evaluated on ORL, Yale and FERET databases. The experimental results show that the proposed method is outperforms singular value decomposition (SVD)-based and QR decomposition with column pivoting (QRCP)-based methods in terms of recognition rates.

Fractional-Order Embedding Supervised Canonical Correlations Analysis with Applications to Feature Extraction and Recognition

Due to the noise disturbance and limited number of training samples, within-set and between-set sample covariance matrices in canonical correlations analysis (CCA) based methods usually deviate from the true ones. In this paper, we re-estimate the covariance matrices by embedding fractional order and incorporate the class label information. First, we illustrate the effectiveness of the fractional-order embedding model through theory analysis and experiments. Then, we quote fractional-order within-set and between-set scatter matrices, which can significantly reduce the deviation of sample covariance matrices. Finally, we incorporate the supervised information, novel generalized CCA and discriminative CCA are presented for multi-view dimensionality reduction and recognition, called fractional-order embedding generalized canonical correlations analysis and fractional-order embedding discriminative canonical correlations analysis. Extensive experimental results on various handwritten numeral, face and object recognition problems show that the proposed methods are very effective and obviously outperform the existing methods in terms of classification accuracy.

Maximum–minimum–median average MSD-based approach for face recognition

A new and efficient improved maximum scatter difference (MSD) model is introduced in this paper. The main weakness of the MSD model is that the class mean vector is constructed via class sample average when the within-class and between-class scatter matrices are formed. For a few of given samples with non-ideal conditions (e.g., variations of expression, pose and noisy environment), the assessment result is very weak by using the class sample average. That is because there will be some outliers in these samples. Therefore, the recognition performance of maximum scatter difference criterion will decline significantly. To solve the problem, in the traditional MSD model, we use within-class maximum–minimum–median average vector to construct within-class scatter matrix (Sw) and between-class scatter matrix (Sb) instead of within-class mean vector. The experimental results show that an improvement of the MSD model is possible with the proposed technique in ORL and Yale face database recognition problems.

Robust Independent Component Analysis Based on Two Scatter Matrices

Oja, Sirkiä, and Eriksson (2006) and Ollila, Oja, and Koivunen (2007) showed that, under general assumptions, any two scatter matrices with the so called independent components property can be used to estimate the unmixing matrix for the independent component analysis (ICA). The method is a generalization of Cardoso’s (Cardoso, 1989) FOBI estimate which uses the regular covariance matrix and a scatter matrix based on fourth moments. Different choices of the two scatter matrices are compared in a simulation study. Based on the study, we recommend always the use of two robust scatter matrices. For possible asymmetric independent components, symmetrized versions of the scatter matrix estimates should be used.

Online Condition Monitoring of Bearings to Support Total Productive Maintenance in the Packaging Materials Industry.

The packaging materials industry has already recognized the importance of Total Productive Maintenance as a system of proactive techniques for improving equipment reliability. Bearing faults, which often occur gradually, represent one of the foremost causes of failures in the industry. Therefore, detection of their faults in an early stage is quite important to assure reliable and efficient operation. We present a new automated technique for early fault detection and diagnosis in rolling-element bearings based on vibration signal analysis. Following the wavelet decomposition of vibration signals into a few sub-bands of interest, the standard deviation of obtained wavelet coefficients is extracted as a representative feature. Then, the feature space dimension is optimally reduced to two using scatter matrices. In the reduced two-dimensional feature space the fault detection and diagnosis is carried out by quadratic classifiers. Accuracy of the technique has been tested on four classes of the recorded vibrations signals, i.e., normal, with the fault of inner race, outer race, and ball operation. The overall accuracy of 98.9% has been achieved. The new technique can be used to support maintenance decision-making processes and, thus, to increase reliability and efficiency in the industry by preventing unexpected faulty operation of bearings.

A Class-Information-Based Sparse Component Analysis Method to Identify Differentially Expressed Genes on RNA-Seq Data.

With the development of deep sequencing technologies, many RNA-Seq data have been generated. Researchers have proposed many methods based on the sparse theory to identify the differentially expressed genes from these data. In order to improve the performance of sparse principal component analysis, in this paper, we propose a novel class-information-based sparse component analysis (CISCA) method which introduces the class information via a total scatter matrix. First, CISCA normalizes the RNA-Seq data by using a Poisson model to obtain their differential sections. Second, the total scatter matrix is gotten by combining the between-class and within-class scatter matrices. Third, we decompose the total scatter matrix by using singular value decomposition and construct a new data matrix by using singular values and left singular vectors. Then, aiming at obtaining sparse components, CISCA decomposes the constructed data matrix by solving an optimization problem with sparse constraints on loading vectors. Finally, the differentially expressed genes are identified by using the sparse loading vectors. The results on simulation and real RNA-Seq data demonstrate that our method is effective and suitable for analyzing these data.

Robust discriminative extreme learning machine for relevance feedback in image retrieval

Relevance feedback (RF) has long been an important approach for multi-media retrieval because of the semantic gap in image content, where SVM based methods are widely applied to RF of content-based image retrieval. However, RF based on SVM still has some limitations: (1) the high dimension of image features always make the RF time-consuming; (2) the model of SVM is not discriminative, because labels of image features are not sufficiently exploited. To solve above problems, we proposed robust discriminative extreme learning machine (RDELM) in this paper. RDELM involved both robust within-class and between-class scatter matrices to enhance the discrimination capacity of ELM for RF. Furthermore, an angle criterion dimensionality reduction method is utilized to extract the discriminative information for RDELM. Experimental results on four benchmark datasets (Corel-1K, Corel-5K, Corel-10K and MSRC) illustrate that our proposed RF method in this paper achieves better performance than several state-of-the-art methods.

Inference and mixture modeling with the Elliptical Gamma Distribution

The authors study modeling and inference with the Elliptical Gamma Distribution (EGD). In particular, Maximum likelihood (ML) estimation for EGD scatter matrices is considered, a task for which the authors present new fixed-point algorithms. The algorithms are shown to be efficient and convergent to global optima despite non-convexity. Moreover, they turn out to be much faster than both a well-known iterative algorithm of Kent & Tyler and sophisticated manifold optimization algorithms. Subsequently, the ML algorithms are invoked as subroutines for estimating parameters of a mixture of EGDs. The performance of the methods is illustrated on the task of modeling natural image statistics—the proposed EGD mixture model yields the most parsimonious model among several competing approaches.

Fault Diagnosis Method of Joint Fisher Discriminant Analysis Based on the Local and Global Manifold Learning and Its Kernel Version

Though Fisher discriminant analysis (FDA) is an outstanding method of fault diagnosis, it is usually difficult to extract the discriminant information in a complex industrial environment. One of the reasons is that, in such an environment, the discriminant information can not been extracted entirely due to the disturbances, non-Gaussianity and nonlinearity. In this paper, a method named Joint Fisher discriminant analysis (JFDA) is proposed to address the issues. First, JFDA removes outliers caused by disturbances according to the energy density of each datum. Then, for the non-Gaussianity and weakly nonlinearity, the novel scatter matrices are defined to extract both of the local and global discriminant information based on the manifold learning. Finally, the kernel JFDA (KJFDA) is investigated to hold the manifold assumption because the strongly nonlinearity may weaken the assumption and cause overlapping. The proposed method is applied to the Tennessee Eastman process (TEP). The results demonstrate that KJFDA shows a better performance of fault diagnosis than other improved versions of FDA.

A Hyper spectral Images Classification Method Based on Maximum Scatter Discriminant Analysis

To overcome “small sample size problem” problem faced by some hyper spectral classification methods, the Maximum Scatter Discriminant criterion is used to analyzed hyperspectral data. Maximum Scatter Discriminant analysis searches for the project axes by maximizing the difference of between-class scatter and within-class scatter matrices, which avoid to calculate the inverse of matrices. Experiment results on Indian Pines HSI data set show that the proposed method outperforms the other methods in terms of recognition accuracy. The proposed method is an effective and feasible method for hyper pectral data classification.

Unconstrained face verification with a dual-layer block-based metric learning

In this paper, a dual-layer block-based metric learning technique is proposed to better discriminate the face image pairs and accelerate the overall verification process under the unconstrained environment. The input images are processed as blocks to provide a richer base of face features. Our proposed method is formed by two layers, in which the first layer assists in extracting the compact block-based descriptors without the existence of full class label information and to refine the within-class and between-class scatter matrices while the second layer integrates the face descriptors of all blocks. The proposed scheme has computational advantage over the single metric learning method while it exploits the correlations among the multiple metrics from different descriptors. The performance of our proposed method is evaluated on the Labeled Faces in the Wild database and achieves an improved performance when compared with the state-of-the-art methods in terms of verification rate and computation time.

Two-dimensional discriminant locality preserving projections (2DDLPP) and its application to feature extraction via fuzzy set

This paper presents a new method for image feature extraction, namely, the fuzzy 2D discriminant locality preserving projections (F2DDLPP) based on the 2D discriminant locality preserving projections (2DDLPP) and fuzzy set theory. Firstly, we calculate the membership degree matrix by fuzzy k-nearest neighbor (FKNN), then we incorporate the membership degree matrix into the definition of the intra-class scatter matrix and inter-class scatter matrix, respectively. Secondly, we can get the fuzzy intra-class scatter matrix and fuzzy inter-class scatter matrix, respectively. The FKNN is implemented to achieve the distribution information of original samples, and this information is utilized to redefine corresponding scatter matrices. So, F2DDLPP can extract discriminative features from overlapping (outlier) samples which is different to the conventional 2DDLPP. Finally, Experiments on the Yale, ORL face databases, USPS database and PolyU palmprint database are demonstrated to verify the effectiveness of the proposed algorithm.

Phone classification via manifold learning based dimensionality reduction algorithms

Mechanical limitations imposed on the articulators during speech production lead to a limitation of the intrinsic dimensionality of speech signals. This limitation leads to a specific neighborhood structure of speech sounds when they are represented in a high-dimensional feature space. We investigate whether phone classification can be improved by exploiting this neighborhood structure, by means of extended variants of the conventional Linear Discriminant Analysis (LDA) based on manifold learning.In this extended LDA approach, the within-class and between-class scatter matrices are defined in terms of adjacency graphs. We compare extensions of LDA that use either a full adjacency graph or an adjacency graph defined in the neighborhood of the training observations. In addition, we apply different kernels for weighing the distances in the graphs via different kernels, of which the Adaptive Kernel is proposed in this paper.Experiments with TIMIT show that while LDA algorithms that use the full adjacency graph do not outperform traditional LDA, the algorithms that exploit only local information provide significantly better results than traditional LDA. These improvements are not uniform across different broad phonetic classes, which suggests that the added value of the neighborhood structure is phone class dependent. The structure is represented by locally different densities in the neighborhood of feature vectors that are representative of a specific phone in a specific context.