Local Manifold Structure Research Articles

Linear projection-based noise filtering framework for image denoising

Image denoising remains a fundamental challenge in digital image processing due to the inevitable presence of noise during image acquisition and transmission. While existing noise filtering methods predominantly focus on local spatial information, they often overlook crucial structural information from other perspectives, such as local manifold and global structures. To address this limitation, we propose a novel linear projection-based noise filtering (LPNF) framework grounded in linear projection learning theory. This framework innovatively learns a linear projection for noise filtering by incorporating multiple structural information sources - local spatial, local manifold, and global structures - through well-defined criteria. We present two specialized implementations of the LPNF framework: PCA-based LPNF (LPNF-PCA) and LPP-based LPNF (LPNF-LPP). The LPNF-PCA simultaneously leverages local spatial and global information, while LPNF-LPP integrates both local manifold and spatial information for enhanced denoising performance. Comprehensive experiments conducted on four standard test images with various noise types demonstrate that both LPNF-PCA and LPNF-LPP consistently outperform state-of-the-art denoising methods in terms of both quantitative.

Semi-supervised convolutional generative adversarial networks for dynamic fault classification with manifold regularization

Data-driven fault classification identifies and categorizes faults or anomalies in industrial systems, to ensure efficient and safe operations. This paper addresses the challenges of semi-supervised learning for dynamic fault classification in industrial processes. The key problems involve utilizing three unique characteristics of typical industrial process dataset: 1) the presence of unlabeled samples due to high labeling costs and required expert knowledge, 2) the local manifold structure arising from strong variable coupling, and 3) time correlation inherent in time-series sensing data of dynamic processes. To tackle these challenges, a semi-supervised fault classification model, SSDCGAN, is proposed, based on deep convolutional Generative Adversarial Networks. The model captures dynamic temporal information through a moving window technique and leverages manifold regularization to maintain classifier consistency along the manifold’s tangent direction. Evaluations on the Tennessee Eastman (TE) benchmark demonstrate that SSDCGAN enhances fault classification accuracy, outperforming current methods.

Locality Cross-domain Discriminant Analysis for Membranous Nephropathy Recognition Using Microscopic Hyperspectral Imaging.

Cross-domain methods have been proposed to learn the domain invariant knowledge that can be transferred from the source domain to the target domain. Existing cross-domain methods attempt to minimize the distribution discrepancy of the domains. However, these methods fail to explore the domain invariant subspace due to the samples of different classes between two domains may overlap in the new subspace. They consider the features in the original space data that may be unnecessary or irrelevant to the final classification, and neglect to preserve the local manifold structure between two domains. To solve these problems, a novel feature extraction method called Locality Cross-domain Discriminant Analysis (LCDA) is proposed. LCDA first aligns the distributions and avoids overlap between two domains. Then, LCDA exploits the local manifold structure to maintain the discriminative capability of the low-dimensional projection matrices. Finally, a robust constraint is utilized to preserve the robustness of the projection matrices. The proposed LCDA not only avoids overlap between different classes but also explores the local manifold information. Experiment results on the medical membranous nephropathy hyperspectral dataset demonstrate that the proposed LCDA has better performance than other relevant feature extraction methods.

ANS-SCMC: A matrix completion method based on adaptive neighbourhood similarity and sparse constraints for predicting microbe-disease associations.

The use of matrix completion methods to predict the association between microbes and diseases can effectively improve treatment efficiency. However, the similarity measures used in the existing methods are often influenced by various factors such as neighbourhood size, choice of similarity metric, or multiple parameters for similarity fusion, making it challenging. Additionally, matrix completion is currently limited by the sparsity of the initial association matrix, which restricts its predictive performance. To address these problems, we propose a matrix completion method based on adaptive neighbourhood similarity and sparse constraints (ANS-SCMC) for predict microbe-disease potential associations. Adaptive neighbourhood similarity learning dynamically uses the decomposition results as effective information for the next learning iteration by simultaneously performing local manifold structure learning and decomposition. This approach effectively preserves fine local structure information and avoids the influence of weight parameters directly involved in similarity measurement. Additionally, the sparse constraint-based matrix completion approach can better handle the sparsity challenge in the association matrix. Finally, the algorithm we proposed has achieved significantly higher predictive performance in the validation compared to several commonly used prediction methods proposed to date. Furthermore, in the case study, the prediction algorithm achieved an accuracy of up to 80% for the top 10 microbes associated with type 1 diabetes and 100% for Crohn's disease respectively.

Fast Discriminant Analysis With Adaptive Reconstruction Structure Preserving.

Neighborhood reconstruction methods have been widely applied to feature engineering. Existing reconstruction-based discriminant analysis methods normally project high-dimensional data into a low-dimensional space while preserving the reconstruction relationships among samples. However, there are three limitations: 1) the reconstruction coefficients are learned based on the collaborative representation of all sample pairs, which requires the training time to be the cube of the number of samples; 2) these coefficients are learned in the original space, ignoring the interference of the noise and redundant features; and 3) there is a reconstruction relationship between heterogeneous samples; this will enlarge the similarity of heterogeneous samples in the subspace. In this article, we propose a fast and adaptive discriminant neighborhood projection model to tackle the above drawbacks. First, the local manifold structure is captured by bipartite graphs in which each sample is reconstructed by anchor points derived from the same class as that sample; this can avoid the reconstruction between heterogeneous samples. Second, the number of anchor points is far less than the number of samples; this strategy can reduce the time complexity substantially. Third, anchor points and reconstruction coefficients of bipartite graphs are updated adaptively in the process of dimensionality reduction, which can enhance the quality of bipartite graphs and extract discriminative features simultaneously. An iterative algorithm is designed to solve this model. Extensive results on toy data and benchmark datasets show the effectiveness and superiority of our model.

Integrability of nearly trans-Sasakian manifolds

The geometry of integrable nearly trans-Sasakian manifolds (NST-manifolds) is studied in this paper. In particular, we consider as NST-manifolds with an integrable structure, normal NST-manifolds, and NST-manifolds satisfying the condition N(2)=0. Local structure of such manifolds is also described. We give a classification of NST-manifolds of constant Φ-holomorphic sectional curvature, as well as satisfying the axiom of Φ-holomorphic planes. NST-manifolds with a completely integrable first fundamental distribution are discussed.

Improving deep metric learning via self-distillation and online batch diffusion process

Currently, deep metric learning (DML) methods typically rely on class labels to keep positive samples as closely clustered as possible, while distancing them from negative samples. However, this approach tends to disregard some vital information inherent in the data, such as intra-class information variation, which can hinder the generalization of the trained model. To address this issue, we propose an online batch diffusion-based self-distillation method (OBD-SD), which consists of a progressive self-distillation (PSD) technique and an online batch diffusion process (OBDP). Specifically, PSD is a simple yet effective self-distillation technique that is beneficial for the diversity of embedding representations. OBDP uses the diffusion process to reveal the intrinsic relationships among samples in the mini-batch and produce better soft distance targets. Our approach combines PSD with OBDP, which can extract richer relational information among samples by capturing the local geometric structure of manifolds in each mini-batch. OBD-SD is a highly flexible framework that can be integrated into state-of-the-art DML methods. Our experimental results on CUB200, CARS196, and Stanford Online Products datasets demonstrate that our OBD-SD consistently enhances the performance of existing DML methods, without requiring additional training time, resulting in competitive results.

Joint Projected Fuzzy Neighborhood Preserving C-means Clustering with Local Adaptive Learning

Graph-based dimensionality reduction approaches are extensively applied for preserving the intrinsic structures of high-dimensional data. To capture discriminative information from data clusters that lack a clear manifold structure, previous studies have combined clustering algorithms with local manifold structure preservation. These methods assume that two neighbor data points share the same cluster label. However, the assumption may not hold for Fuzzy C-Means which may assign two nearby data points to different clusters depending on the distance between data points and cluster centroids. This inconsistency degrades the performance in clustering and dimensionality reduction. To tackle this issue, we propose a novel approach for dimensionality reduction, termed joint Projected Fuzzy Neighborhood Preserving C-means Clustering with Local Adaptive Learning (PFNPCM). PFNPCM learns cluster centroids, sparse membership grades, embedded subspaces, and structured graphs simultaneously and extracts both global and local structures of data. Through clustering data that preserves the local neighborhood structure in a low-dimensional space instead of the original one, PFNPCM can effectively capture both global and local information from the inherent manifold structure and cluster structure. The employment of ℓ2,1-norm as the distance metric in the clustering and the preservation of neighborhood structure further ensure the robustness of clustering results against noise or outliers. To achieve the consistency of the captured global and local discriminative information, PFNPCM adopts a novel fuzzy local similarity measure based on the manifold structure, and the manifold assumption thus holds when partitioning unlabeled samples into subgroups. Massive experiments on diverse well-known datasets demonstrate that PFNPCM is superior to some state-of-the-art methods.

Sparse semi-supervised multi-label feature selection based on latent representation

With the rapid development of the Internet, there are a large number of high-dimensional multi-label data to be processed in real life. To save resources and time, semi-supervised multi-label feature selection, as a dimension reduction method, has been widely used in many machine learning and data mining. In this paper, we design a new semi-supervised multi-label feature selection algorithm. First, we construct an initial similarity matrix with supervised information by considering the similarity between labels, so as to learn a more ideal similarity matrix, which can better guide feature selection. By combining latent representation with semi-supervised information, a more ideal pseudo-label matrix is learned. Second, the local manifold structure of the original data space is preserved by the manifold regularization term based on the graph. Finally, an effective alternating iterative updating algorithm is applied to optimize the proposed model, and the experimental results on several datasets prove the effectiveness of the approach.

Self-paced regularized adaptive multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0<p≤1) is employed to measure the learning error and as the sparse regularization to accommodate various sparsity requirements across different datasets. Moreover, hypergraph Laplacian matrices are constructed for each view to better preserve the local manifold structure and encode high-order relationships within the data space. They are adaptively assigned weights to learn the underlying correlated and diverse information among different views. An iterative optimization algorithm is proposed to solve SPAMUFS and the convergence and computational complexity are also analyzed. The effectiveness of SPAMUFS is substantiated by comparing with eight state-of-the-art algorithms on nine public multi-view datasets.

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

MLFLHMDA: predicting human microbe-disease association based on multi-view latent feature learning.

A growing body of research indicates that microorganisms play a crucial role in human health. Imbalances in microbial communities are closely linked to human diseases, and identifying potential relationships between microbes and diseases can help elucidate the pathogenesis of diseases. However, traditional methods based on biological or clinical experiments are costly, so the use of computational models to predict potential microbe-disease associations is of great importance. In this paper, we present a novel computational model called MLFLHMDA, which is based on a Multi-View Latent Feature Learning approach to predict Human potential Microbe-Disease Associations. Specifically, we compute Gaussian interaction profile kernel similarity between diseases and microbes based on the known microbe-disease associations from the Human Microbe-Disease Association Database and perform a preprocessing step on the resulting microbe-disease association matrix, namely, weighting K nearest known neighbors (WKNKN) to reduce the sparsity of the microbe-disease association matrix. To obtain unobserved associations in the microbe and disease views, we extract different latent features based on the geometrical structure of microbes and diseases, and project multi-modal latent features into a common subspace. Next, we introduce graph regularization to preserve the local manifold structure of Gaussian interaction profile kernel similarity and add -norms to the projection matrix to ensure the interpretability and sparsity of the model. The AUC values for global leave-one-out cross-validation and 5-fold cross validation implemented by MLFLHMDA are 0.9165 and 0.8942+/-0.0041, respectively, which perform better than other existing methods. In addition, case studies of different diseases have demonstrated the superiority of the predictive power of MLFLHMDA. The source code of our model and the data are available on https://github.com/LiangzheZhang/MLFLHMDA_master.

Multigraph approximate-representation learning for hyperspectral band selection

ABSTRACT Unsupervised hyperspectral image (HSI) band selection methods have been attracting ever-increasing attention. However, the local structural features captured by most of the existing methods suffer from a certain degree of risk of feature ambiguity. Moreover, these methods typically have difficulty preserving the latent manifold structure of the HSI in selected subsets of the bands. To address these problems, firstly, the graph approximate representation learning (GARL) model is proposed in this article by incorporating graph-regularized sparse coding into self-representation learning to preserve the local manifold structure information of HSI in the selected subset of bands. Meanwhile, to alleviate the ambiguity of the local graph structures, we design a local-global feature relationship expression model to construct the latent long-distance contextual connectivity (LDCC) graphs between the obtained local graph structures. Then, to preserve HSI’s local and non-local manifold structure information as much as possible in the selected subset of bands, a multigraph approximate representation learning (MGARL) model is proposed in this article by incorporating the obtained LDCC graph into the proposed GARL model. Next, we design a solution method to solve the proposed formulation. Finally, extensive experimental results on four datasets reveal the promising performance of MGARL over some state-of-the-art methods.

Discriminative and robust least squares regression for semi-supervised image classification

Due to the ability to leverage information from both unlabeled and labeled data, semi-supervised classification has found extensive applications in various practical scenarios. However, there are two major drawbacks: (1) Traditional graph construction leads to high algorithmic complexity, which limits efficiency. (2) The decision boundary might be blurred by boundary points in common cases. To cope with these issues, inspired by classical Least Squares Regression (LSR), we present a novel semi-supervised classification algorithm termed as Discriminative and Robust LSR (DRLSR) for semi-supervised image classification. First of all, a manifold regularization term is designed and introduced to an LSR-based semi-supervised method to preserve local manifold structures and smooth structures of the subspace, which strengthens the discrimination ability. Meanwhile, preservation of local manifold structures also contributes to restrain decision boundaries from being blurred by boundary points, which strengthens robustness of the algorithm. After that, an efficient alternative optimization method is applied to our algorithm. Evidence of the effectiveness of DRLSR are compelled by extensive experimental results.

An industrial process monitoring method and its application with fractal-based structure preserving embedding

Maintaining stable industrial process operation is a crucial research topic in modern process monitoring. However, industrial process data often contains many redundant correlated variables, which can disrupt the process of data dimensionality reduction. To address this, a novel dimensionality reduction method termed Fractal-based Structural Preservation Embedding (FSPE) is proposed. This introduces a new feature selection approach via fractal analysis, which effectively eliminates redundant variables without altering data orientation, enhancing interpretability. The data structures are then preserved at spatial and temporal scales by solving a dual-objective optimization function. Spatially, it preserves the local data manifold structure while considering global information. Temporally, it maintains time series by searching for nearest neighboring points in time. The efficacy of the FSPE monitoring method is verified through simulations involving the Tennessee Eastman Process (TEP) and the Electric Furnace Magnesia Furnace (EFMF).

Multi-label Feature selection with adaptive graph learning and label information enhancement

The high dimensionality and complexity of multi-label data make obtaining accurate label sets in practical applications difficult. Noisy data in the labels will affect the model’s classification performance. Existing methods only reconstruct logical labels into semantic labels to mitigate the effect of inherent noise and do not specifically address the noise. In order to solve this problem and eliminate the effect of noise, this paper proposes a multi-label feature selection method with adaptive graph learning and label information enhancement (AGLE). First, each label is represented as a combination of other labels using global label correlation, which can reconstruct the original label matrix to obtain the self-expression label matrix. Based on this, similar labels in two matrices can be considered reliable labels, and not similar may be noise. Therefore, constructing a predictive label matrix to weigh the learning of original and self-expression labels can enhance reliable information. Second, an adaptive graph is utilized to learn global label correlation to ensure that the self-expression label matrix can be continuously optimized. Then, the local label manifold structure is utilized to constrain the adaptive graph for more stable performance. Finally, the adaptive graph and ℓ2,1-norm are combined to guide learning the feature weight matrix. For the proposed method, an iterative algorithm based on alternating optimization is used to compute the weight matrix. Comparing experimental results with several state-of-the-art algorithms demonstrates the effectiveness of the AGLE method for the noise problem.

EEG-Based Driver Mental Fatigue Recognition in COVID-19 Scenario Using a Semi-Supervised Multi-View Embedding Learning Model

With the spread of COVID-19 in recent years, wearing masks has increased the difficulty of driver mental fatigue recognition. Electroencephalogram (EEG) signal has become an important physiological signal index to reflect the driver’s mental state. However, the drivers’ EEG data is plagued by inadequate labels and multi-view data, which makes classification difficult. To solve this problem, this study proposes a semi-supervised multi-view sparse regularization and graph embedding learning (SMSG) model. To obtain discriminative feature representations of semi-supervised EEG data, SMSG fully mines diverse information from multiple views based on sparse regularization embedding and graph embedding technology. SMSG employs the graph embedding to capture the discriminative structure and local manifold structure on multi-view data. Furthermore, SMSG learns the common shared regularization embedding and private regularization embedding factors to preserve the consistency and diversity of the multi-view data. Through self-adaptive learning, the weights of each view can be directly solved adaptively. This works also introduces kernel trick to project the SMSG model into the nonlinear reproducing kernel Hilbert space (RKHS), which can obtain more approximate EEG feature representation. Experiments on the real dataset verify the effectiveness of the SMSG model for EEG-based driver mental fatigue recognition.

Improved Multi-Distance ARMF Integrated With LTSA Based Pattern Matching Method and Its Application in Fault Diagnosis

Effective dimensionality reduction (DR) and classification in fault diagnosis remain a significant challenge, primarily due to the increasing scale of industrial processes and the non-linear and high-dimensional features of process data. To address this challenge, we present local tangent space alignment (LTSA) integrated with a multi-distance adaptive order morphological filter (MARMF) fault diagnosis method (LTSA-MARMF). In LTSA-MARMF, LTSA that preserves the local manifold structure using tangent space is first utilized for DR to provide the required feature space data for ARMF. Next, the cosine distance and dynamic time warping distance are introduced into the distance error of ARMF, considering the spatial similarity and dynamic features to improve classification accuracy. Finally, the distance-matching result of the pattern is applied to determine the type of fault. Through simulations, it is evident that LTSA-MARMF can achieve more satisfactory fault diagnosis accuracy than other related methods on the Tennessee-Eastman process (TEP) and the actual Grid-connected PV System (GPVS). <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is inspired by the difficult-to-handle high-dimensional and non-linear features of process data but is also applicable to high-dimensional and non-linear data from other industrial processes. The DR and classification are important aspects of fault diagnosis. In this paper, a novel pattern-matching method is utilized for fault diagnosis, which uses LTSA and the modified multi-distance ARMF for DR and classification, respectively. In terms of mathematics, the distance error of ARMF is analyzed. The combination of multi-distance is used to enhance the accuracy of fault diagnosis. The preliminary experiments show that LTSA-MARMF is feasible, but has not been tested in the plant. We will consider testing LTSA-MARMF in an actual plant in future research.

Locality Robust Domain Adaptation for cross-scene hyperspectral image classification

Domain adaptation (DA) has become a widely used technique for cross-scene hyperspectral image (HSI) classification. Most DA methods aim to learn a domain invariant subspace that reduce the domain discrepancy between source and target domains. However, some of them fail to explore the local manifold structure between different domains, while also neglecting the negative influences of abnormal features. To solve these problems, we propose a Locality Robust Domain Adaptation (LRDA) method for cross-domain data recognition. In LRDA, the statistical alignment is applied to reduce the domain-shift between the source and target domains. Then, LRDA combines the row-sparsity constraint and the discriminant regularization term to learn a robust projection matrix, while maintaining the discriminative capability of the matrix. Furthermore, a manifold regularization term is proposed to automatically learn the nearest neighbors and weights between two domains. The proposed LRDA not only reduces the discrepancy between two domains but also explores the local neighbor information between them. Experiment results on three HSI datasets illustrate that the proposed LRDA has better performance than other related methods.

Binary multi-view clustering with spectral embedding

Techniques for large-scale multi-view clustering have attracted increasing attention in recent years. To explore complementary and consensus information across multiple views, mapping multi-view data into a hamming space for binary multi-view learning is highly effective. However, for binary multi-view clustering, it is difficult to adopt the local manifold structure of original data into the consensus representation. Therefore, a novel algorithm named Binary Multi-View Clustering With Spectral Embedding (BMVC_SE) is proposed in this study. This algorithm unifies binary consensus representation learning and multi-view spectral learning into a joint framework. Specifically, this algorithm exploits the shareable and individual information across multi-view data to learn a binary consensus representation. Furthermore, spectral rotation technique is employed to embed manifold information from original multi-view data into binary consensus representation. Note that a multi-view GPI algorithm is proposed in this study to obtain the better local manifold information. We also provide an effective optimization method to solve the existing method. Extensive experimental results show that our method outperforms some state-of-the-art methods.