Locality Preserving Research Articles

Semi-Supervised Subspace Learning for Pattern Classification via Robust Low Rank Constraint

Feature subspace learning is a crucial issue in pattern analysis. However, it remains challenging when partial samples are unlabeled, which will cause weak discrimination. In this paper, we present a novel semi-supervised learning model that is capable of utilizing labeled and unlabeled training data simultaneously to learn discriminative feature subspace while preserving their locality. To achieve this goal, we joint learning feature subspace and completed labels. In the framework, low rank representation model is firstly exploited to explore the similarity relationship among all training samples, including labeled and unlabeled data. Then, the learned representation coefficients are used to generate a dynamic neighbor graph for designing the locality preservation constraints on both of label propagation and feature subspace. Finally, the prediction and true label are used to enforce the discrimination of feature subspace in the semantic space. Extensive experiment indicates that our proposed approach is more competitive than other comparison methods, while the model shows more robustness when training datasets are contaminated with noise.

Feature Extraction of Double Pulse Metal Inert Gas Welding Based on Broadband Mode Decomposition and Locality Preserving Projection

A novel adaptive signal decomposition algorithm, broadband mode decomposition (BMD), is proposed for analyzing nonstationary broadband signals. Unavoidable error will occur when applying former time-frequency methods to broadband signals, which is caused by Gibbs phenomenon and the calculation of extrema. To overcome that problem, BMD is proposed by searching in the associative dictionary that contains both broadband and narrowband signals. The procedure of the proposed method is as follows: First, the collected datasets are analyzed by BMD and the composite multiscale fuzzy entropies (CMFEs) of the obtained effective components are calculated. Then, locality preserving projection (LPP) is applied for further feature extraction. Analysis results show BMD is more effective when drawing broadband feature from noise and BMD is adaptive for the quality monitoring of DPMIG welding.

Local class-specific discriminant analysis with variable weighting and its application in fault diagnosis

In this paper, local class-specific discriminant analysis with variable weighting (VW-LCSDA) model was developed for fault classification in industrial processes. Although FDA has been extensively studied in the field of fault diagnosis as a classic method of dimension reduction and classifying, the effectiveness of the FDA is degraded in many cases due to data distribution and number of classes. The proposed method weights the variable vectors by evaluating the impact of the fault on each feature. Besides, the in-class and out-of-class scatter matrixes of class-specific discriminant analysis (CSDA) are used for dimension reduction and classification. In addition, the local structure of the sample data is preserved with reference to the LPP(Locality preserving projection) method, which is used to optimize the scatter matrix. The proposed fault classification method is validated by the Tennessee-Eastman process. The simulation results show that the VW-LCSDA method is a fault diagnosis method superior to FDA, CSDA and variable-weighted CSDA (VW-CSDA).

Monitoring of industrial processes using robust global–local preserving projection

AbstractUsual multivariate statistical analysis (MSA) techniques are highly sensitive to outliers because they are based on the least squares fitting or the empirical mean and covariance of the data. Data‐driven process monitoring methods based on usual MSA techniques may be unreliable when outliers are present in the training data. To overcome this deficiency, a robust global–local preserving projection (RGLPP) method is proposed for dimension reduction of the high‐dimensional data contaminated by outliers, and then it is applied to the robust monitoring of industrial processes. Despite the presence of outliers, RGLPP yields robust projection directions that can preserve both global and local structures of the high‐dimensional data. Moreover, based on the robust projection directions of RGLPP, a method is developed for identifying outliers in a multivariate data set. An RGLPP‐based robust process monitoring method is also developed to achieve high‐performance monitoring when the training data of industrial processes contain outliers. The effectiveness and advantages of the proposed method are illustrated using an industrial case study.

Application of Locality Preserving Projection-Based Unsupervised Learning in Predicting the Oil Production for Low-Permeability Reservoirs

Summary Predicting well production in unconventional reservoirs has attracted much attention in recent years. However, it is still a challenge because of the heterogeneity of the unconventional formation and the uncertain physical properties of rock and the fluid-flow mechanism. Therefore, the objective of our study is to develop an efficient method for production forecast in low-permeability reservoirs. In this paper, we applied a locality preserving projection (LPP) -based machine-learning method to provide a novel way of predicting production in low-permeability reservoirs. Our model preserves the local geometric information inherent in the oil data and hence is capable of seizing its intrinsic nonlinear feature. Through LPP-based analysis, specific parameters (e.g., the total fluid pumped) are verified to have more important impact than others on estimating oil production. We conducted both parameter testing and comparison experiments, with results indicating that the LPP-based method has good applicability and efficiency in reasonably forecasting well production from low-permeability reservoirs. This work provides petroleum engineers an effective method for the prediction of oil production.

Stacked locality preserving autoencoder for feature extraction and its application for industrial process data modeling

Deep learning has recently caught much attention in the industrial processes, particularly for soft sensor applications. However, most traditional deep learning networks cannot extract local features for data modeling. To overcome this problem, a novel stacked locality preserving autoencoder (S-LPAE) is proposed in this paper. First, the neighborhood topological structure is built for the historical samples and the weights between the neighbor samples are calculated. Then, locality preserving autoencoder (LPAE) is designed to minimize both the reconstruction error and the additional local preserving constraint of the training dataset, with which the potential features can better preserve the local data structure. After that, multiple LPAE modules are sequentially stacked to construct the S-LPAE network to obtain deep locality-preserving features. Finally, the extracted features are directly used for the output prediction of soft sensor. To validate the performance of the proposed algorithm, it is applied to an industrial hydrocracking process to predict the 90% boiling point of aviation kerosene and the 50% boiling point of diesel.

Using Locality Preserving Projections to Improve the Performance of Kernel Clustering

Many clustering methods may have poor performance when the data structure is complex (i.e., the data has an aspheric shape or non-linear relationship). Inspired by this view, we proposed a clustering model which combines kernel function and Locality Preserving Projections (LPP) together. Specifically, we map original data into the high-dimensional feature space according to the idea of kernel function. Secondly, it is feasible to explore the local structure of data in clustering tasks. LPP is used to preserve the original local structure information of data to improve the validity of the clustering model. Finally, some outliers are often included in real data, so we embedded sparse regularization items in the model to adjust feature weights and remove outliers. In addition, we design a simple iterative optimization method to solve the final objective function and show the convergence of the optimization method in the experimental part. The experimental analysis of ten public data sets showed that our proposed method has better efficiency and performance in clustering tasks than existing clustering methods.

A deep learning model for plant lncRNA-protein interaction prediction with graph attention.

Long non-coding RNAs (lncRNAs) play a broad spectrum of distinctive regulatory roles through interactions with proteins. However, only a few plant lncRNAs have been experimentally characterized. We propose GPLPI, a graph representation learning method, to predict plant lncRNA-protein interaction (LPI) from sequence and structural information. GPLPI employs a generative model using long short-term memory (LSTM) with graph attention. Evolutionary features are extracted using frequency chaos game representation (FCGR). Manifold regularization and l2-norm are adopted to obtain discriminant feature representations and mitigate overfitting. The model captures locality preserving and reconstruction constraints that lead to better generalization ability. Finally, potential interactions between lncRNAs and proteins are predicted by integrating catboost and regularized Logistic regression based on L-BFGS optimization algorithm. The method is trained and tested on Arabidopsis thaliana and Zea mays datasets. GPLPI achieves accuracies of 85.76% and 91.97% respectively. The results show that our method consistently outperforms other state-of-the-art methods.

Local preserving logistic I-Relief for semi-supervised feature selection

Logistic I-Relief (LIR), a supervised feature selection method, has been successfully extended for semi-supervised feature selection. However, exiting semi-supervised methods based on LIR are incapable of maintaining the local structure of data. In this paper, we propose a local preserving logistic I-Relief (LPLIR) algorithm for multi-class semi-supervised feature selection. The goal of LPLIR is to maximize the expected margin of the given labeled data and remain the local structural information of all given data, where the local structural information is represented by a local preserving regularization. Additionally, extensive experiments conducted on real-world datasets demonstrate that LPLIR is superior to the existing state-of-the-art semi-supervised feature selection methods.

Multi-layer manifold learning with feature selection

Many fundamental problems in machine learning require some form of dimensionality reduction. To this end, two different strategies were used: Manifold Learning and Feature Selection. Manifold learning (or data embedding) attempts to compute a subspace from original data by feature recombination/transformation. Feature selection aims to select the most relevant features in the original space. In this paper, we propose a novel cooperative Manifold learning-Feature selection that goes beyond the simple concatenation of these two modules. Our basic idea is to transform a given shallow embedding to a deep variant by computing a cascade of embeddings in which each embedding undergoes feature selection and elimination. We use filter approaches in order to efficiently select irrelevant features at any stage of the process. For a case study, our proposed framework was used with two typical linear embedding algorithms: Local Discriminant Embedding (LDE) (a supervised technique) and Locality Preserving Projections (LPP) (unsupervised technique) on four challenging face databases and it has been conveniently compared with other cooperative schemes. Moreover, a comparison with several state-of-the-art manifold learning methods is provided. As it is exhibited by our experimental study, the proposed framework can achieve superior learning performance with respect to classic cooperative schemes and to many competing manifold learning methods.

Spatial-Spectral Joint Classification of Hyperspectral Image With Locality and Edge Preserving

Hyperspectral image (HSI) classification is an important part of its processing and application. Aiming at the problems of high data dimensionality and high spatial neighborhood correlation in HSI classification, we propose a spatial-spectral joint classification method of HSI with locality and edge preserving in this article. First, the input HSI is normalized, and the feature is extracted by principal component analysis. The first principal component image is taken as the guidance image. Second, guided filtering is used to extract the spatial features of each band separately. Then, the extracted spatial features are superimposed, and low-dimensional embedding is completed through local Fisher discriminant analysis. Finally, the obtained low-dimensional embedded features are input into a random forest classifier to get classification results. The experimental results of two HSI show that the proposed method achieves higher classification accuracy than other related methods. In the case of randomly selecting 10% and 1% samples from each class of ground object as training samples, the overall classification accuracy is improved to 99.57% and 97.79%, respectively. This method effectively uses the spatial and local information of the image in low dimensional embedding, and preserves the boundaries of the ground objects, thus improving the classification effect.

Locality Preserving Discriminant Projection for Speaker Verification

In this paper, a manifold subspace learning algorithm based on locality preserving discriminant projection (LPDP) is used for speaker verification. LPDP can overcome the deficiency of the total variability factor analysis and locality preserving projection (LPP). LPDP can effectively use the speaker label information of speech data. Through optimization, LPDP can maintain the inherent manifold local structure of the speech data samples of the same speaker by reducing the distance between them. At the same time, LPDP can enhance the discriminability of the embedding space by expanding the distance between the speech data samples of different speakers. The proposed method is compared with LPP and total variability factor analysis on the NIST SRE 2010 telephone-telephone core condition. The experimental results indicate that the proposed LPDP can overcome the deficiency of LPP and total variability factor analysis and can further improve the system performance.

Local Sensitive Dual Concept Factorization for Unsupervised Feature Selection

In this paper, we present a novel Local Sensitive Dual Concept Learning (LSDCL) method for the task of unsupervised feature selection. We first reconstruct the original data matrix by the proposed dual concept learning model, which inherits the merit of co-clustering based dual learning mechanism for more interpretable and compact data reconstruction. We then adopt the local sensitive loss function, which emphasizes more on most similar pairs with small errors to better characterize the local structure of data. In this way, our method can select features with better clustering results by more compact data reconstruction and more faithful local structure preserving. An iterative algorithm with convergence guarantee is also developed to find the optimal solution. We fully investigate the performance improvement by the newly developed terms, individually and simultaneously. Extensive experiments on benchmark datasets further show that LSDCL outperforms many state-of-the-art unsupervised feature selection algorithms.

Discriminative Multiple Kernel Concept Factorization for Data Representation

Concept Factorization (CF) improves Nonnegative matrix factorization (NMF), which can be only performed in the original data space, by conducting factorization within proper kernel space where the structure of data become much clear than the original data space. CF-based methods have been widely applied and yielded impressive results in optimal data representation and clustering tasks. However, CF methods still face with the problem of proper kernel function design or selection in practice. Most existing Multiple Kernel Clustering (MKC) algorithms do not sufficiently consider the intrinsic neighborhood structure of base kernels. In this paper, we propose a novel Discriminative Multiple Kernel Concept Factorization method for data representation and clustering. We first extend the original kernel concept factorization with the integration of multiple kernel clustering framework to alleviate the problem of kernel selection. For each base kernel, we extract the local discriminant structure of data via the local discriminant models with global integration. Moreover, we further linearly combine all these kernel-level local discriminant models to obtain an integrated consensus characterization of the intrinsic structure across base kernels. In this way, it is expected that our method can achieve better results by more compact data reconstruction and more faithful local structure preserving. An iterative algorithm with convergence guarantee is also developed to find the optimal solution. Extensive experiments on benchmark datasets further show that the proposed method outperforms many state-of-the-art algorithms.

Fault diagnosis for industrial robots based on a combined approach of manifold learning, treelet transform and Naive Bayes

This research introduces a novel fault diagnosis method for an industrial robot based on manifold learning algorithms, Treelet Transform (TT) and Naive Bayes. The vibration signals of an industrial robot working under three working conditions are acquired as the raw data. Three typical manifold learning algorithms, Principal Component Analysis (PCA), Locality Preserving Projections (LPPs), and Isometric Feature Mapping (ISOMAP), are utilized to extract three-dimensional features from the vibration signals. Then, these features were combined into nine-dimensional features and, these nine-dimensional features were reduced to three-dimensional feature vectors by TT. Finally, a Naive Bayes model is trained with these three-dimensional feature vectors. Experimental results show that compared with the three methods, PCA, LPP, and ISOMAP, the accuracy of the proposed combined method is higher than the single method. The fault diagnosis method presented in this paper is easy to implement and can effectively identify the fault types.

Narrow Operating Space Based on the Inversion of Latent Structures Model for Glycosylation Process

N-linked glycan distribution plays a significant role in the generation of therapeutic proteins. It is challenging to determine the operating conditions when developing a new biopharmaceutical product with the desired glycan distributions. The glycosylation is a high complex nonlinear system, and it is difficult to develop a reliable first-principle model that heavily relies on experimentation. Our goal is to develop a nonlinear data-driven model and find an appropriate operating space included kinds of input combination from process variables based on this model to ensure the desired product quality. A methodology is proposed based on the inversion of a nonlinear latent-variable model (locality preserving projection to latent structures, LPPLS) to identify a subspace of the knowledge space. The normal operating points of the input variables are designed based on the LPPLS inversion, and the range of operating conditions are expanded around the normal operation points through the prediction uncertainty analysis of forward and inversion model simultaneously. Finally, the designated operation space from LPPLS inversion is applied in an benchmark glycosylation model.

Unsupervised Large Graph Embedding Based on Balanced and Hierarchical K-means

There are many successful spectral based unsupervised dimensionality reduction methods, including Laplacian Eigenmap (LE), Locality Preserving Projection (LPP), Spectral Regression (SR), etc. We discover that LPP and SR are equivalent if the symmetric similarity matrix is {doubly stochastic}, Positive Semi-Definite (PSD) and with rank p, where p is the reduced dimension. Since solving SR is believed faster than solving LPP based on some related literature, the discovery promotes us to seek to construct such specific similarity matrix to speed up LPP solving procedures. We then propose an unsupervised linear method called Unsupervised Large Graph Embedding (ULGE). ULGE starts with a similar idea as LPP but adopts an efficient approach to construct anchor-based similarity matrix and then performs spectral analysis on it. Moreover, since conventional anchor generation strategies suffer kinds of problems, we propose an efficient and effective anchor generation strategy, called Balanced K-means based Hierarchical K-means (BHKH). The computational complexity of ULGE can reduce to O(ndm), which is a significant improvement compared to conventional methods need $O(n^2d)$ at least, where n, d and m are the number of samples, dimensions, and anchors, respectively. Extensive experiments on several publicly available datasets demonstrate the efficiency and effectiveness of the proposed method.

Subspace Learning and Feature Selection via Orthogonal Mapping

A plethora of dimensionality reduction (DR) techniques, stemming from statistics, machine learning, or graph theory, has been developed. Their ultimate objective is to eliminate data redundancy without a significant information loss. A general formulation, known as graph embedding, offers a unified framework for describing several well-known DR algorithms. In this paper, the inclusion of several DR algorithms within this unified framework is demonstrated. The enforcement of orthogonality to the projection matrix within this framework is proven to be of vital importance, since Orthogonal Neighborhood Preserving Projections, Orthogonal Locality Preserving Projections, Orthogonal Isometric Projection, Orthogonal Linear Discriminant Analysis, and Orthogonal Local Tangent Space Alignment algorithms outperform their non-orthogonal counterparts, e.g., Neighborhood Preserving Embedding, Locality Preserving Projections, Isometric Mapping, Linear Discriminant Analysis, and Local Tangent Space Alignment. One may simultaneously also impose the $\ell _{2,1}$ norm regularization on the projection matrix, seeking for row-sparsity. This leads to the known Joint Feature Selection and Subspace Learning (JFSSL) framework. All DR algorithms are employed within JFSSL. It is proven that the use of orthogonal mapping algorithms within JFSSL against their non-orthogonal counterparts does not improve the recognition rate of NN classifier.

Deep Domain Adaptation Based on Adversarial Network With Graph Regularization

Although most transfer learning methods can reduce the difference of the feature distributions between the source and target domains effectively, some classes in the two domains may still be misaligned after domain adaptation, especially for the classes with similar features such as “bicycle” and “motorcycle”. Therefore, a graph regularization based adversarial network model is proposed, whose innovations mainly include the following two aspects: First, a constraint function which is used to measure the difference between the features belonged to different classes is proposed, whose purpose is that not only the training accuracy is taken into account during supervised training, but also the difference between classes should be enlarged as much as possible; Then, a graph regularization constraint function is proposed, which makes all the classes have good local preserving properties after domain adaptation, and further reduces the possibility of all classes being misaligned. Experimental results on several cross-domain benchmark datasets show that our newly proposed approach outperforms state of the art methods.

Local Preserving Class Separation Framework to Identify Gestational Diabetes Mellitus Mother Using Ultrasound Fetal Cardiac Image

In the presence of gestational diabetes mellitus (GDM), the fetus is exposed to a hyperinsulinemia environment. This environment can cause a wide range of metabolic and fetal cardiac structural alterations. Fetal myocardial hypertrophy predominantly affecting the interventricular septum possesses a morphology of disarray similar to hypertrophic cardiomyopathy, and may be present in some GDM neonates after birth. Myocardial thickness may increase in GDM fetuses independent of glycemic control status and fetal weight. Fetal echocardiography performed on fetuses with GDM helps in assessing cardiac structure and function, and to diagnose myocardial hypertrophy. There are few studies in the literature which have established evidence for morphologic variation associated with cardiac hypertrophy among fetuses of GDM mothers. In this study, fetal ultrasound images of normal, pregestational diabetes mellitus (preGDM) and GDM mothers were used to develop a computer aided diagnostic (CAD) tool. We proposed a new method called local preserving class separation (LPCS) framework to preserve the geometrical configuration of normal and preGDM/GDM subjects. The generated shearlet based texture features under LPCS framework showed promising results compared with deep learning algorithms. The proposed method achieved a maximum accuracy of 98.15% using a support vector machine (SVM) classifier. Hence, this paradigm can be helpful to physicians in detecting fetal myocardial hypertrophy in preGDM/GDM mothers.