Discriminant Locality Preserving Projections Research Articles

Novel Discriminant Locality Preserving Projection Integrated With Monte Carlo Sampling for Fault Diagnosis

In complex industrial processes, the technique of fault diagnosis has been playing an increasingly considerable role in ensuring the safety of life and property. Unfortunately, the process data of complex industrial processes have the features of high dimension. Feature extraction from high-dimensional data is promising to coping with the fault data with high dimension. Recently, one of manifold learning methods named discriminant locality preserving projection achieves excellent performance in feature extraction. However, the performance of discriminant locality preserving projection (DLPP) is subject to the problem of matrix decomposition in the denominator of the objection function caused by the small sample size (SSS) issue. To overcome this limitation, novel DLPP integrated with Monte Carlo sampling is proposed to enhance the performance of feature extraction through dimensionality reduction. In the proposed MC-DLPP, Monte Carlo sampling is first utilized to generate fault samples for each fault type. With the aid of the virtually generated fault samples, the rank of the matrix in the denominator of the objection function of DLPP increases, thus well addressing the SSS problem. The Softmax classifier is used for fault diagnosis. To test the performance of the improved DLPP-based fault diagnosis, case studies using the Tennessee Eastman process are carried out. Simulation results confirm the presented MC-DLPP achieves superior accuracy in fault diagnosis.

An improved feature extraction method using low-rank representation for motor imagery classification

Motor imagery (MI) classification using electroencephalography (EEG) signal analysis is gaining significant interest for movement intent recognition, where feature extraction is critical for recognition accuracy. Traditional feature extraction methods ignore the spatial and neighborhood structure information of feature signals. In this paper, we assume that an EEG sample can be partitioned into a clean task-related EEG matrix and a noise matrix at first, because of the low-rank structure on underlying data representation disclosed by Low-Rank Representation (LRR). Then, Bilinear Two-Dimensional Discriminant Locality Preserving Projection (B2DDLPP) and LRR are combined to form a new feature extraction method known as Bilinear Low-Rank 2D Discriminant Locality Preserving Projection (BLRDLPP). It keeps the global structure information while preserving the local neighborhood relation. During experimentation, the proposed algorithm achieved classification accuracies of 73.26 % and 82.27 % on the four-class MI of the BCI Competition IV-2a dataset and III-3a dataset, respectively. The results demonstrate that the proposed method can effectively improve the acquisition of discriminant features.

A Novel Discriminant Locality Preserving Projections Method

Locality preserving projections (LPP) is a popular unsupervised dimensionality reduction method based on manifold learning. As a supervised version of the LPP method, discriminant locality preserving projections (DLPP) method has been recently proposed and paid much attention to by researchers. However, the DLPP method has the small-sample-size (SSS) problem. In this paper, in the view of the eigenvalues of scattering matrices of DLPP, they are first mapped to the new values by two polynomial functions, and with the properties of the matrix function of the two polynomial functions, the criterion of the DLPP method is reconstructed; thus, a novel dimensionality reduction method, named polynomial discriminant locality preserving projections (PDLPP) method, is proposed. The proposed PDLPP method has two advantages: one is that it addresses the SSS problem of DLPP, and the other is that, with the nonlinear mapping implied by PDLPP, the distance between inter-class samples is much enlarged and then the better performance of pattern classification is achieved. The experiments are conducted on the COIL-20 database, ORL, Georgia Tech, and AR face datasets, and the results show that the PDLPP is superior to state-of-the-art methods.

Fault diagnosis using novel AdaBoost based discriminant locality preserving projection with resamples

Fault diagnosis plays a pivotal role in ensuring the safety of process industries. However, due to the diversity of process faults and the high coupling of fault data, it becomes very difficult to achieve high accuracy in the fault diagnosis of complex industrial processes. To address this concern, in this article, a novel AdaBoost-based discriminant locality preserving projection (DLPP) with resamples (A-DLPPR) model is proposed. The proposed A-DLPPR model has two features: to address the problem of matrix decomposition in DLPP, the bootstrap method is utilized to generate groups of resample data, and to obtain high classification accuracy, the AdaBoost-based classification technique is adopted. Finally, an effective fault diagnosis model using the proposed A-DLPPR model can be established. To validate the effectiveness of the proposed A-DLPPR model, the Tennessee Eastman process (TEP) is selected, and case studies using different kinds of TEP faults are conducted. The simulation results indicate that the proposed A-DLPPR model can achieve higher fault diagnosis accuracy than some other models, which verifies that in the field of complex industrial processes, the proposed A-DLPPR method can be used as an effective model for fault diagnosis.

Two-Dimensional Discriminant Locality Preserving Projections for Crop Leaf Disease Detection

Crop diseases seriously affect the yield and quality of crops and decrease the economic income of the farmers...

Novel Pattern Recognition Using Bootstrap-Based Discriminant Locality-Preserving Projection and Its Application to Fault Diagnosis

For the sake of ensuring the safety of complex process industries, accurate fault diagnosis is very important and necessary. Pattern recognition-based techniques have been successfully and widely applied to fault diagnosis. In this article, a novel pattern recognition method using bootstrap-based discriminant locality-preserving projection is proposed. In the proposed bootstrap-based discriminant locality-preserving projection method, a bootstrap is used to resample and construct groups of within-class data. As a result, the matrix decomposition problem of the discriminant locality- preserving projection can be solved with the aid of within-class data. In addition, a novel adjacency graph consisting of a k-nearest-neighbor graph and a k-furthest-neighbor graph is adopted in the proposed model to seek the subspace that best discriminates the different classes, where the distance between class is maximized, while the distance within class is minimized. To verify the performance of the proposed bootstrap-based discriminant locality-preserving projection method, case studies using a two-dimensional synthetic dataset and the Tennessee Eastman process are carried out. Simulation results indicate that compared with some other methods, the proposed Bootstrap-DLPP method can achieve a better visual intuition and a higher accuracy in fault diagnosis.

Discriminant locality preserving projections based on L2,p-norm for image feature extraction and recognition

Conventional discriminant locality preserving projections (DLPP) is sensitive to outliers because the formulation of its objective function is based on L2-norm. Not only that, the learned features by DLPP are linear combinations of all the original features. So, it is hard to know which features play an important role in feature extraction, and the learned features often contain irrelevant information. In this paper, we propose a robust version of DLPP based on L2,p-norm with 0<p<1, termed DLPP-L2,p, for image feature extraction and recognition. DLPP-L2,p learns an optimal projection matrix by maximizing the L2,p-norm-based locality preserving between-class dispersion and minimizing the L2,p-norm-based locality preserving within-class dispersion simultaneously. Furthermore, by imposing an L2,p-norm penalty on the projection matrix to achieve row-sparsity, DLPP-L2,p can discard irrelevant features and transform relevant features simultaneously. Experimental results on several image datasets demonstrate the effectiveness and robustness of DLPP-L2,p.

Sparse two-dimensional discriminant locality-preserving projection (S2DDLPP) for feature extraction

Two-dimensional locality-preserving projection (2DLPP) is an unsupervised method, so it can’t use the discrimination information of the sample in the sparse data; elastic net regression can obtain a sparse results of the feature extraction. So, this paper presents a new method for image feature extraction, namely the sparse two-dimensional discriminant locality-preserving projection (S2DDLPP) based on the 2D discriminant locality-preserving projection (2DDLPP) and elastic net regression. By adding the between-class scatter and discrimination information into the objective function of 2DLPP, S2DDLPP uses elastic net regression to obtain an optimal sparse projection matrix with “minimizing the within-class scatter” and “maximizing the between-class scatter.” Compared with other methods (2DPCA, 2DPCA-L1, 2DLDA, 2DLPP, 2DDLPP, and 2DDLPP-L1), the experimental results on the ORL, Yale, AR and FERET face database show the effectiveness of the proposed algorithm.

Matrix exponential based discriminant locality preserving projections for feature extraction

Discriminant locality preserving projections (DLPP), which has shown good performances in pattern recognition, is a feature extraction algorithm based on manifold learning. However, DLPP suffers from the well-known small sample size (SSS) problem, where the number of samples is less than the dimension of samples. In this paper, we propose a novel matrix exponential based discriminant locality preserving projections (MEDLPP). The proposed MEDLPP method can address the SSS problem elegantly since the matrix exponential of a symmetric matrix is always positive definite. Nevertheless, the computational complexity of MEDLPP is high since it needs to solve a large matrix exponential eigenproblem. Then, in this paper, we also present an efficient algorithm for solving MEDLPP. Besides, the main idea for solving MEDLPP efficiently is also generalized to other matrix exponential based methods. The experimental results on some data sets demonstrate the proposed algorithm outperforms many state-of-the-art discriminant analysis methods.

Stationarity test and Bayesian monitoring strategy for fault detection in nonlinear multimode processes

Both of multimode and nonlinear characteristics have been common in modern industrial processes. In the present work, an improved monitoring strategy on the basis of a modified discriminant locality preserving projections (DLPP) algorithm and stationary test is proposed for nonlinear multimode process monitoring to analyze both within-mode and cross-mode information. First, the adjoined multiple local models are constructed in high-dimensional kernel space to extract within-mode information through kernel fuzzy c-means algorithm and kernel principal components. Afterwards, the weighted local representations are defined by integrating the local latent vectors within each mode and the corresponding posterior probabilities of data samples. Then, a modified DLPP algorithm is developed to preserve the local structure within each mode and separate different modes as far as possible by merging local models into a low-dimensional coordinated space. Augmented Dickey Fuller test (ADF test) strategy is utilized to decompose the final low-dimensional space into two subspaces including mode-specific subspace and mode-common subspace by checking the stability of the final global data. A Bayesian monitoring strategy is presented in mode-specific subspace and residual subspace to provide more reliable monitoring results. Finally, to illustrate the feasibility and effectiveness, the proposed algorithm is illustrated with multimode data generated from the Tennessee Eastman (TE) benchmark process and Three-phase flow process.

Two-Dimensional Discriminant Locality Preserving Projection Based on ℓ1-norm Maximization

In this paper, a new linear dimensionality reduction method named Two-Dimensional Discriminant Locality Preserving Projection Based on ℓ1-norm Maximization (2DDLPP-L1) is proposed for preprocessing of image data. 2DDLPP-L1 makes full use of the robustness of ℓ1-norm to noises and outliers. Furthermore, 2DDLPP-L1 is a 2D-based method which extracts image features directly from image matrices, avoiding instability and high complexity of matrix computation. Two graphs, separation graph and cohesiveness graph, are constructed with feature vectors as vertices to represent the inter-class separation and intra-class cohesiveness. An iterative algorithm with proof of convergence is proposed to solve the optimal projection matrix. Experiments on several face image databases demonstrate that the performance and robustness of 2DDLPP-L1 are better than its related methods.

Exponential Discriminant Locality Preserving Projection for face recognition

How to determine the low dimensional manifold is a challenging problem. Locality Preserving Projections (LPP) can gracefully deal with it. With the help of discriminant information provided by Discriminant Locality Preserving Projection (DLPP), the performance of face recognition can be significantly improved. In the real world, the DLPP has the Small Sample Size (SSS) problem. To deal with this issue, we utilize the matrix exponential to obtain more effective information, which can avoid the singular matrix׳s disadvantages. Thus, in this paper, we propose an effective and efficient algorithm called Exponential discriminant locality projections (EDLPP) for face recognition. The experimental results on three challenging benchmark datasets (ORL, YALE and LFW) demonstrate that the proposed EDLPP algorithm outperforms favorably against several state-of-the-art methods.

Discriminant sparse locality preserving projection for face recognition

In this paper, aiming at the drawback of the popular dimensionality reduction method Discriminant Sparse Neighborhood Preserving Embedding(DSNPE), i.e. the construction of its between-class scatter is too complexity, a novel algorithm Discriminant Locality Preserving Projection (DSLPP) is proposed. Our proposal retains the local sparse reconstructive relationships of DSNPE and constructs a novel between-class scatter by using all mean faces as sparse representation dictionary. In particular, DSLPP preserves the sparse reconstructive relationship of mean face, so then it can not only efficiently reduce the between-class scatter computation complexity of DSNPE, but also increase the discriminant power. In the experiment, we compare DSLPP with classic and state-of-the-art dimensionality reduction methods on the publicly available data sets such as ORL, Yale, UMIST, and AR, and further apply the Gabor feature into DSLPP on AR face database to further improve its performance. The experimental results show that DSLPP is able to obtain a better representation of the class information and achieve much higher recognition accuracy.

Discriminant Hyper-Laplacian Projections and its scalable extension for dimensionality reduction

Discriminant Locality Preserving Projections (DLPP) is one of the most influential supervised subspace learning algorithms that consider both discriminative and geometric (manifold) information. There is an obvious drawback of DLPP that it only considers the pairwise geometric relationship of samples. However, in many real-world issues, relationships among the samples are often more complex than pairwise. Naively squeezing the complex into pairwise ones will inevitably lead to loss of some information, which are crucial for classification and clustering. We address this issue via using the Hyper-Laplacian instead of the regular Laplacian in DLPP, which only can depict the pairwise relationship. This new DLPP algorithm is exactly a generalization of DLPP and we name it Discriminant Hyper-Laplacian Projection (DHLP). In order to make DHLP can be feasibly applied to big data dimensionality reduction, we adopt the spectral regression framework to reduce the computational complexity of DHLP from cubic-time to linear-time. We named this new DHLP algorithm Scalable Discriminant Hyper-Laplacian Projections (SDHLP). Six popular visual databases are adopted for validating our work. The results not only demonstrate the superiorities of DHLP and SDHLP in comparison with the state-of-the-art algorithms but also demonstrate the efficiency improvement of SDHLP over DHLP.

Discriminant Locality Preserving Projections Based on Neighborhood Maximum Margin

This paper took a research about the small size sample problem of the discriminant locality preserving projections method, and proposed the discriminant locality preserving projections method based on neighborhood maximum margin (NMMDLPP). Firstly, the training sample structured a weighted of K-nearest neighbor graph, and gave the weight parameter to each side of the nearest neighbor graph for obtaining the intraclass neighbors and interclass neighbors local geometry information of each point; then reduce the interval between the intraclass neighbors and increase the interval between the interclass neighbors with the result of transfer matrix, and applied the neighbor point optimal refactoring coefficient of the data to the objective function. This method chose the difference between the locality preserving between-class scatter and the locality preserving within-class scatter as the objective function to avoid of calculating the inversion of matrix. This method has conducted an experiment on the UMIST face database and Yale face database. Experimental results show that the NMMDLPP algorithm is superior to other algorithms in recognition rate. The recognition rate can reach more than 91.4%.

Discriminant locality preserving projections based on L1-norm maximization.

Conventional discriminant locality preserving projection (DLPP) is a dimensionality reduction technique based on manifold learning, which has demonstrated good performance in pattern recognition. However, because its objective function is based on the distance criterion using L2-norm, conventional DLPP is not robust to outliers which are present in many applications. This paper proposes an effective and robust DLPP version based on L1-norm maximization, which learns a set of local optimal projection vectors by maximizing the ratio of the L1-norm-based locality preserving between-class dispersion and the L1-norm-based locality preserving within-class dispersion. The proposed method is proven to be feasible and also robust to outliers while overcoming the small sample size problem. The experimental results on artificial datasets, Binary Alphadigits dataset, FERET face dataset and PolyU palmprint dataset have demonstrated the effectiveness of the proposed method.

ORTHOGONAL MAXIMUM MARGIN DISCRIMINANT PROJECTION WITH APPLICATION TO LEAF IMAGE CLASSIFICATION

A novel supervised dimensionality reduction method called orthogonal maximum margin discriminant projection (OMMDP) is proposed to cope with the high dimensionality, complex, various, irregular-shape plant leaf image data. OMMDP aims at learning a linear transformation. After projecting the original data into a low dimensional subspace by OMMDP, the data points of the same class get as near as possible while the data points of the different classes become as far as possible, thus the classification ability is enhanced. The main differences from linear discriminant analysis (LDA), discriminant locality preserving projections (DLPP) and other supervised manifold learning-based methods are as follows: (1) In OMMDP, Warshall algorithm is first applied to constructing both of the must-link and class-class scatter matrices, whose process is easily and quickly implemented without judging whether any pairwise points belong to the same class. (2) The neighborhood density is defined to construct the objective function of OMMDP, which makes OMMDP be robust to noise and outliers. Experimental results on two public plant leaf databases clearly demonstrate the effectiveness of the proposed method for classifying leaf images.

Direct Discriminant Locality Preserving Projection With Hammerstein Polynomial Expansion

Discriminant locality preserving projection (DLPP) is a linear approach that encodes discriminant information into the objective of locality preserving projection and improves its classification ability. To enhance the nonlinear description ability of DLPP, we can optimize the objective function of DLPP in reproducing kernel Hilbert space to form a kernel-based discriminant locality preserving projection (KDLPP). However, KDLPP suffers the following problems: 1) larger computational burden; 2) no explicit mapping functions in KDLPP, which results in more computational burden when projecting a new sample into the low-dimensional subspace; and 3) KDLPP cannot obtain optimal discriminant vectors, which exceedingly optimize the objective of DLPP. To overcome the weaknesses of KDLPP, in this paper, a direct discriminant locality preserving projection with Hammerstein polynomial expansion (HPDDLPP) is proposed. The proposed HPDDLPP directly implements the objective of DLPP in high-dimensional second-order Hammerstein polynomial space without matrix inverse, which extracts the optimal discriminant vectors for DLPP without larger computational burden. Compared with some other related classical methods, experimental results for face and palmprint recognition problems indicate the effectiveness of the proposed HPDDLPP.

Gabor filter based optical image recognition using Fractional Power Polynomial model based common discriminant locality preserving projection with kernels

This paper presents Gabor filter based optical image recognition using Fractional Power Polynomial model based Common Kernel Discriminant Locality Preserving Projection. This method tends to solve the nonlinear classification problem endured by optical image recognition owing to the complex illumination condition in practical applications, such as face recognition. The first step is to apply Gabor filter to extract desirable textural features characterized by spatial frequency, spatial locality and orientation selectivity to cope with the variations in illumination. In the second step we propose Class-wise Locality Preserving Projection through creating the nearest neighbor graph guided by the class labels for the textural features reduction. Finally we present Common Kernel Discriminant Vector with Fractional Power Polynomial model to reduce the dimensions of the textural features for recognition. For the performance evaluation on optical image recognition, we test the proposed method on a challenging optical image recognition problem, face recognition.

SAR Target Feature Extraction and Recognition Based on 2D-DLPP

In this paper, a new feature extraction algorithm named 2D-DLPP (Two-dimensional Discriminant Locality Preserving Projections) is used for SAR ATR (Synthetic Aperture Radar Automatic Target Recognition). First, SAR target images are preprocessed by log-transformation and 2D FFT, then 2D-DLPP is applied to extract target feature which can not only preserve local information by capturing the local geometry of manifold but also implement sample dimension reduction effectively. Finally, classification with SVM (Support Vector Machine) is performed to get the good recognition rate. Experimental results based on MSTAR (Moving and Stationary Target Acquisition and Recognition) SAR data demonstrate that 2D-DLPP can obtain more effective target feature and improve the recognition rate obviously compared with 2D-LDA (Two-dimensional Linear Discriminant Analysis).