Local Manifold Research Articles

Self-paced regularized adaptive multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0<p≤1) is employed to measure the learning error and as the sparse regularization to accommodate various sparsity requirements across different datasets. Moreover, hypergraph Laplacian matrices are constructed for each view to better preserve the local manifold structure and encode high-order relationships within the data space. They are adaptively assigned weights to learn the underlying correlated and diverse information among different views. An iterative optimization algorithm is proposed to solve SPAMUFS and the convergence and computational complexity are also analyzed. The effectiveness of SPAMUFS is substantiated by comparing with eight state-of-the-art algorithms on nine public multi-view datasets.

Bearing Fault Diagnosis Based on Local Manifold Discriminant Domain Adaptation

Bearing Fault Diagnosis Based on Local Manifold Discriminant Domain Adaptation

Nonlinear Quality‐Related Fault Detection Using Neighborhood Embedding Neural Orthogonal Mapping Algorithm for Batch Process

AbstractQuality‐related fault detection has become a hot research topic in recent years. It is not reliable to measure quality‐related relationships only by mutual information among process variables and single‐quality variables. Frequent alarms for quality‐unrelated faults seriously affect the normal operation of industrial production. At the same time, the strong nonlinearity of the process data leads to the difficulty of feature extraction. In this paper, we propose a fault detection algorithm based on nonlinear quality‐related neighborhood embedding neural orthogonal mapping (QR‐NENOM). First, quality‐related and quality‐unrelated variables are selected by Bayesian fusion mutual information, and the weighted method of mutual information is used to enhance the quality‐related information and suppress the quality‐unrelated information. Second, local manifold information is obtained by reconstructing nearest neighbors of process data, and key features are extracted by the nonlinear method composed of neural network and orthogonal mapping. Then, statistical indicators are established to complete fault detection. Finally, the nonlinear feature extraction ability of NENOM is verified by numerical examples, and the QR‐NENOM algorithm proposed in this paper is applied to the penicillin fermentation process. Comparative experiments show that QR‐NENOM has better detection performance for quality‐related faults and fewer alarms for quality‐unrelated faults.

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

Swarming in stellar streams: Unveiling the structure of the Jhelum stream with ant colony-inspired computation

The halo of the Milky Way galaxy hosts multiple dynamically coherent substructures known as stellar streams that are remnants of tidally disrupted orbiting systems such as globular clusters (GCs) and dwarf galaxies (DGs). A particular case is that of the Jhelum stream, which is known for its unusual and complex morphology. Using the available data from the Gaia DR3 catalog, we extracted a region on the sky that contains Jhelum, and fine-tuned this selection by enforcing limits on the magnitude and proper motion of the selected stars. We then applied the novel Locally Aligned Ant Technique (LAAT) on the position and proper motion space of stars belonging to the selected region to highlight the stars that are closely aligned with a local manifold in the data and the stars belonging to regions of high local density. We find that the overdensity representing the stream in proper motion space is composed of two components, and show the correspondence of these two signals to the previously reported narrow and broad spatial components of Jhelum. We then made use of the radial velocity measurements provided by the S5 survey and confirm, for the first time, a separation between the stars belonging to the two components in radial velocity. We show that the narrow and broad components have velocity dispersions of 4.84+1.23-0.79 km s-1 and 19.49+2.19-1.84 km s−1, and metallicity dispersions of 0.15+0.18-0.10 and 0.34+0.13-0.09, respectively. These measurements, as well as the given difference in component widths, could be explained with a probable scenario where Jhelum is the remnant of a GC embedded within a DG and where both were accreted onto the Milky Way during their infall. Although the properties of Jhelum could be explained with this merger scenario, other progenitors of the narrow component remain possible such as a nuclear star cluster or a DG. To rule these possibilities out, we would need more observational data of member stars of the stream. Our analysis shows that the internal structure of streams holds great information on their past formation history, and therefore provides further insight into the merger history of the Milky Way.

MLFLHMDA: predicting human microbe-disease association based on multi-view latent feature learning.

A growing body of research indicates that microorganisms play a crucial role in human health. Imbalances in microbial communities are closely linked to human diseases, and identifying potential relationships between microbes and diseases can help elucidate the pathogenesis of diseases. However, traditional methods based on biological or clinical experiments are costly, so the use of computational models to predict potential microbe-disease associations is of great importance. In this paper, we present a novel computational model called MLFLHMDA, which is based on a Multi-View Latent Feature Learning approach to predict Human potential Microbe-Disease Associations. Specifically, we compute Gaussian interaction profile kernel similarity between diseases and microbes based on the known microbe-disease associations from the Human Microbe-Disease Association Database and perform a preprocessing step on the resulting microbe-disease association matrix, namely, weighting K nearest known neighbors (WKNKN) to reduce the sparsity of the microbe-disease association matrix. To obtain unobserved associations in the microbe and disease views, we extract different latent features based on the geometrical structure of microbes and diseases, and project multi-modal latent features into a common subspace. Next, we introduce graph regularization to preserve the local manifold structure of Gaussian interaction profile kernel similarity and add -norms to the projection matrix to ensure the interpretability and sparsity of the model. The AUC values for global leave-one-out cross-validation and 5-fold cross validation implemented by MLFLHMDA are 0.9165 and 0.8942+/-0.0041, respectively, which perform better than other existing methods. In addition, case studies of different diseases have demonstrated the superiority of the predictive power of MLFLHMDA. The source code of our model and the data are available on https://github.com/LiangzheZhang/MLFLHMDA_master.

Local Sample-Weighted Multiple Kernel Clustering With Consensus Discriminative Graph.

Multiple kernel clustering (MKC) is committed to achieving optimal information fusion from a set of base kernels. Constructing precise and local kernel matrices is proven to be of vital significance in applications since the unreliable distant-distance similarity estimation would degrade clustering performance. Although existing localized MKC algorithms exhibit improved performance compared with globally designed competitors, most of them widely adopt the KNN mechanism to localize kernel matrix by accounting for τ -nearest neighbors. However, such a coarse manner follows an unreasonable strategy that the ranking importance of different neighbors is equal, which is impractical in applications. To alleviate such problems, this article proposes a novel local sample-weighted MKC (LSWMKC) model. We first construct a consensus discriminative affinity graph in kernel space, revealing the latent local structures. Furthermore, an optimal neighborhood kernel for the learned affinity graph is output with naturally sparse property and clear block diagonal structure. Moreover, LSWMKC implicitly optimizes adaptive weights on different neighbors with corresponding samples. Experimental results demonstrate that our LSWMKC possesses better local manifold representation and outperforms existing kernel or graph-based clustering algorithms. The source code of LSWMKC can be publicly accessed from https://github.com/liliangnudt/LSWMKC.

Multigraph approximate-representation learning for hyperspectral band selection

ABSTRACT Unsupervised hyperspectral image (HSI) band selection methods have been attracting ever-increasing attention. However, the local structural features captured by most of the existing methods suffer from a certain degree of risk of feature ambiguity. Moreover, these methods typically have difficulty preserving the latent manifold structure of the HSI in selected subsets of the bands. To address these problems, firstly, the graph approximate representation learning (GARL) model is proposed in this article by incorporating graph-regularized sparse coding into self-representation learning to preserve the local manifold structure information of HSI in the selected subset of bands. Meanwhile, to alleviate the ambiguity of the local graph structures, we design a local-global feature relationship expression model to construct the latent long-distance contextual connectivity (LDCC) graphs between the obtained local graph structures. Then, to preserve HSI’s local and non-local manifold structure information as much as possible in the selected subset of bands, a multigraph approximate representation learning (MGARL) model is proposed in this article by incorporating the obtained LDCC graph into the proposed GARL model. Next, we design a solution method to solve the proposed formulation. Finally, extensive experimental results on four datasets reveal the promising performance of MGARL over some state-of-the-art methods.

Unsupervised feature selection via dual space-based low redundancy scores and extended OLSDA

Spectral clustering is a widely used method for unsupervised feature selection (UFS) to generate pseudo labels. Nonetheless, it is acknowledged that graph algorithms suffer from issues such as redundancy and the dissatisfaction of connectivity, which greatly affect the quality of the learning local manifold. Moreover, existing UFS methods usually ignore the linear dependency among features and the role of non-negative semantics of the predicted labels. Hence, a novel algorithm using dual space-based low redundancy scores and extended orthogonal least square discriminant analysis (OLSDA), abbreviated as DLSEO, is proposed in this paper. Specifically, extended OLSDA is employed to derive non-negative clustering labels and a manifold structure, which avoids explicitly constructing a Laplacian graph. The dual space-based low redundancy scores related to mutual information eliminate redundant data and features, preventing their interference in the feature selection process. Moreover, ℓ2,1−2-norm is introduced to simultaneously guarantee the feature weight matrix's sparsity and ensure the selection of salient features. The results of experiments conducted on twelve benchmark datasets compared with several relevant methods show the superiority of DLSEO.

Discriminative and robust least squares regression for semi-supervised image classification

Due to the ability to leverage information from both unlabeled and labeled data, semi-supervised classification has found extensive applications in various practical scenarios. However, there are two major drawbacks: (1) Traditional graph construction leads to high algorithmic complexity, which limits efficiency. (2) The decision boundary might be blurred by boundary points in common cases. To cope with these issues, inspired by classical Least Squares Regression (LSR), we present a novel semi-supervised classification algorithm termed as Discriminative and Robust LSR (DRLSR) for semi-supervised image classification. First of all, a manifold regularization term is designed and introduced to an LSR-based semi-supervised method to preserve local manifold structures and smooth structures of the subspace, which strengthens the discrimination ability. Meanwhile, preservation of local manifold structures also contributes to restrain decision boundaries from being blurred by boundary points, which strengthens robustness of the algorithm. After that, an efficient alternative optimization method is applied to our algorithm. Evidence of the effectiveness of DRLSR are compelled by extensive experimental results.

An industrial process monitoring method and its application with fractal-based structure preserving embedding

Maintaining stable industrial process operation is a crucial research topic in modern process monitoring. However, industrial process data often contains many redundant correlated variables, which can disrupt the process of data dimensionality reduction. To address this, a novel dimensionality reduction method termed Fractal-based Structural Preservation Embedding (FSPE) is proposed. This introduces a new feature selection approach via fractal analysis, which effectively eliminates redundant variables without altering data orientation, enhancing interpretability. The data structures are then preserved at spatial and temporal scales by solving a dual-objective optimization function. Spatially, it preserves the local data manifold structure while considering global information. Temporally, it maintains time series by searching for nearest neighboring points in time. The efficacy of the FSPE monitoring method is verified through simulations involving the Tennessee Eastman Process (TEP) and the Electric Furnace Magnesia Furnace (EFMF).

Center stable manifold for ground states of nonlinear Schrödinger equations with internal modes

We study the dynamics of solutions of nonlinear Schrödinger equation (NLS) near unstable ground states. The existence of the local center stable manifold around ground states and the asymptotic stability for the solutions on the manifold is proved. The novelty of our result is that we allow the existence of internal modes.

Multi-label Feature selection with adaptive graph learning and label information enhancement

The high dimensionality and complexity of multi-label data make obtaining accurate label sets in practical applications difficult. Noisy data in the labels will affect the model’s classification performance. Existing methods only reconstruct logical labels into semantic labels to mitigate the effect of inherent noise and do not specifically address the noise. In order to solve this problem and eliminate the effect of noise, this paper proposes a multi-label feature selection method with adaptive graph learning and label information enhancement (AGLE). First, each label is represented as a combination of other labels using global label correlation, which can reconstruct the original label matrix to obtain the self-expression label matrix. Based on this, similar labels in two matrices can be considered reliable labels, and not similar may be noise. Therefore, constructing a predictive label matrix to weigh the learning of original and self-expression labels can enhance reliable information. Second, an adaptive graph is utilized to learn global label correlation to ensure that the self-expression label matrix can be continuously optimized. Then, the local label manifold structure is utilized to constrain the adaptive graph for more stable performance. Finally, the adaptive graph and ℓ2,1-norm are combined to guide learning the feature weight matrix. For the proposed method, an iterative algorithm based on alternating optimization is used to compute the weight matrix. Comparing experimental results with several state-of-the-art algorithms demonstrates the effectiveness of the AGLE method for the noise problem.

Locality Cross-domain Discriminant Analysis for Membranous Nephropathy Recognition Using Microscopic Hyperspectral Imaging.

Cross-domain methods have been proposed to learn the domain invariant knowledge that can be transferred from the source domain to the target domain. Existing cross-domain methods attempt to minimize the distribution discrepancy of the domains. However, these methods fail to explore the domain invariant subspace due to the samples of different classes between two domains may overlap in the new subspace. They consider the features in the original space data that may be unnecessary or irrelevant to the final classification, and neglect to preserve the local manifold structure between two domains. To solve these problems, a novel feature extraction method called Locality Cross-domain Discriminant Analysis (LCDA) is proposed. LCDA first aligns the distributions and avoids overlap between two domains. Then, LCDA exploits the local manifold structure to maintain the discriminative capability of the low-dimensional projection matrices. Finally, a robust constraint is utilized to preserve the robustness of the projection matrices. The proposed LCDA not only avoids overlap between different classes but also explores the local manifold information. Experiment results on the medical membranous nephropathy hyperspectral dataset demonstrate that the proposed LCDA has better performance than other relevant feature extraction methods.

Improved Multi-Distance ARMF Integrated With LTSA Based Pattern Matching Method and Its Application in Fault Diagnosis

Effective dimensionality reduction (DR) and classification in fault diagnosis remain a significant challenge, primarily due to the increasing scale of industrial processes and the non-linear and high-dimensional features of process data. To address this challenge, we present local tangent space alignment (LTSA) integrated with a multi-distance adaptive order morphological filter (MARMF) fault diagnosis method (LTSA-MARMF). In LTSA-MARMF, LTSA that preserves the local manifold structure using tangent space is first utilized for DR to provide the required feature space data for ARMF. Next, the cosine distance and dynamic time warping distance are introduced into the distance error of ARMF, considering the spatial similarity and dynamic features to improve classification accuracy. Finally, the distance-matching result of the pattern is applied to determine the type of fault. Through simulations, it is evident that LTSA-MARMF can achieve more satisfactory fault diagnosis accuracy than other related methods on the Tennessee-Eastman process (TEP) and the actual Grid-connected PV System (GPVS). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is inspired by the difficult-to-handle high-dimensional and non-linear features of process data but is also applicable to high-dimensional and non-linear data from other industrial processes. The DR and classification are important aspects of fault diagnosis. In this paper, a novel pattern-matching method is utilized for fault diagnosis, which uses LTSA and the modified multi-distance ARMF for DR and classification, respectively. In terms of mathematics, the distance error of ARMF is analyzed. The combination of multi-distance is used to enhance the accuracy of fault diagnosis. The preliminary experiments show that LTSA-MARMF is feasible, but has not been tested in the plant. We will consider testing LTSA-MARMF in an actual plant in future research.

EEG-Based Driver Mental Fatigue Recognition in COVID-19 Scenario Using a Semi-Supervised Multi-View Embedding Learning Model

With the spread of COVID-19 in recent years, wearing masks has increased the difficulty of driver mental fatigue recognition. Electroencephalogram (EEG) signal has become an important physiological signal index to reflect the driver’s mental state. However, the drivers’ EEG data is plagued by inadequate labels and multi-view data, which makes classification difficult. To solve this problem, this study proposes a semi-supervised multi-view sparse regularization and graph embedding learning (SMSG) model. To obtain discriminative feature representations of semi-supervised EEG data, SMSG fully mines diverse information from multiple views based on sparse regularization embedding and graph embedding technology. SMSG employs the graph embedding to capture the discriminative structure and local manifold structure on multi-view data. Furthermore, SMSG learns the common shared regularization embedding and private regularization embedding factors to preserve the consistency and diversity of the multi-view data. Through self-adaptive learning, the weights of each view can be directly solved adaptively. This works also introduces kernel trick to project the SMSG model into the nonlinear reproducing kernel Hilbert space (RKHS), which can obtain more approximate EEG feature representation. Experiments on the real dataset verify the effectiveness of the SMSG model for EEG-based driver mental fatigue recognition.

Tensor Coupled Learning of Incomplete Longitudinal Features and Labels for Clinical Score Regression.

Longitudinal data with incomplete entries pose a significant challenge for clinical score regression over multiple time points. Although many methods primarily estimate longitudinal scores with complete baseline features (i.e., features collected at the initial time point), such snapshot features may overlook beneficial latent longitudinal traits for generalization. Alternatively, certain completion approaches (e.g., tensor decomposition technology) have been proposed to impute incomplete longitudinal data before score estimation, most of which, however, are transductive and cannot utilize label semantics. This work presents a tensor coupled learning (TCL) paradigm of incomplete longitudinal features and labels for clinical score regression. The TCL enjoys three advantages: 1) It drives semantic-aware factor matrices and collaboratively deals with incomplete longitudinal entries (of features and labels), during which a dynamic regularizer is designed for adaptive attribute selection. 2) It establishes a closed loop connecting baseline features and the coupled factor matrices, which enables inductive inference of longitudinal scores relying on only baseline features. 3) It reinforces the information encoding of baseline data by preserving the local manifold of longitudinal feature space and detecting the temporal alteration across multiple time points. Extensive experiments demonstrate the remarkable performance improvement of our method on clinical score regression with incomplete longitudinal data.

LMdist: Local Manifold distance accurately measures beta diversity in ecological gradients.

Differentiating ecosystems poses a complex, high-dimensional problem constrained by capturing relevant variation across species profiles. Researchers use pairwise distances and subsequent dimensionality reduction to highlight variation in a few dimensions. Despite popularity in analysis of ecological data, these low-dimensional visualizations can contain geometric abnormalities such as "arch" and "horseshoe" effects, potentially obscuring the impact of environmental gradients. These abnormalities appear in ordination but are in fact a product of oversaturated large pairwise distances. We present Local Manifold distance (LMdist), an unsupervised algorithm which adjusts pairwise beta diversity measures to better represent true ecological distances between samples. Beta diversity measures can have a bounded dynamic range in depicting long environmental gradients with high species turnover. Using a graph structure, LMdist projects pairwise distances onto a manifold and traverses the manifold surface to adjust pairwise distances at the upper end of the beta diversity measure's dynamic range. This allows for values beyond the range of the original measure. Not all datasets will have oversaturated pairwise distances, nor will capture variation that resembles a manifold, so LMdist adjusts only those pairwise values which may be undervalued in the presence of a sampled gradient. The adjusted distances serve as input for ordination and statistical testing. We demonstrate on real and simulated data that LMdist effectively recovers distances along known gradients and along complex manifolds such as the Swiss roll dataset. LMdist enables more powerful statistical tests for gradient effects and reveals variation orthogonal to the gradient. Available on GitHub at https://github.com/knights-lab/LMdist.

Multiview Data Clustering with Similarity Graph Learning Guided Unsupervised Feature Selection.

In multiview data clustering, consistent or complementary information in the multiview data can achieve better clustering results. However, the high dimensions, lack of labeling, and redundancy of multiview data certainly affect the clustering effect, posing a challenge to multiview clustering. A clustering algorithm based on multiview feature selection clustering (MFSC), which combines similarity graph learning and unsupervised feature selection, is designed in this study. During the MFSC implementation, local manifold regularization is integrated into similarity graph learning, with the clustering label of similarity graph learning as the standard for unsupervised feature selection. MFSC can retain the characteristics of the clustering label on the premise of maintaining the manifold structure of multiview data. The algorithm is systematically evaluated using benchmark multiview and simulated data. The clustering experiment results prove that the MFSC algorithm is more effective than the traditional algorithm.

Fault Detection-Based Multiple Local Manifold Learning and Its Application to Blast Furnace Ironmaking Process

Process safety plays a vital role in the modern process industry. To prevent undesired accidents caused by malfunctions or other disturbances in complex industrial processes, considerable attention has been paid to data-driven fault detection techniques. To explore the underlying manifold structure, manifold learning methods including Laplacian eigenmaps, locally linear embedding, and Hessian eigenmaps have been utilized in data-driven fault detection. However, only the partial local structure information is extracted from the aforementioned methods. This paper proposes fused local manifold learning (FLML), which synthesizes the typical manifold learning methods to find the underlying manifold structure from different angles. A more comprehensive local structure is discovered under a unified framework by constructing an objection optimization function for process data dimension reduction. The proposed method takes advantage of different manifold learning methods. Based on the proposed dimension reduction method, a new data-driven fault detection method is developed. Hotelling’s T2 and Q statistics are established for the purpose of fault detection. Experiments on an industrial benchmark Tennessee Eastman process whose average MDR and average FAR of FLML T2 are 7.58% and 0.21% and a real blast furnace ironmaking process whose MDR and FAR of FLML T2 are 2.80% and 0.00% are carried out to demonstrate the superiority and effectiveness of the proposed method.