Manifold Regularization Research Articles (Page 1)

Incorporating manifold regularization into adaptive label thresholding algorithm for online semi-supervised multi-label classification

Adaptive spatiotemporal neighborhood feature learning based on manifold regularization autoencoder for industrial process monitoring

Similarity-guided multi-view functional brain network fusion.

Single-cell RNA sequencing (scRNA-seq) offers significant opportunities to reveal cellular heterogeneity and diversity. Accurate cell type identification is critical for downstream analyses and understanding the mechanisms of heterogeneity. However, challenges arise from the high dimensionality, sparsity, and noise of scRNA-seq data. While various low-rank representation (LRR)-based clustering methods have been developed, many existing approaches may inaccurately capture relationships or conflate true patterns with noise. To address these limitations, we introduce a novel clustering algorithm that integrates low-rank matrix decomposition with local graph regularization (LRMGC). This approach applies a tri-decomposition strategy to the representation matrix to derive an aligned core matrix, and characterizes the "distance" between cells in a lower-dimensional space through a local manifold regularization term. Rather than relying on the kernel norm of the representation matrix, the Schatten p-norm is applied to the core matrix to robustly learn the similarity matrix against noise and outliers, while maintaining the high-dimensional noisy data's underlying subspace structure for accurate and robust clustering. Additionally, the final similarity matrix is obtained by applying the angular alignment strategy on the similarity matrix. Comprehensive experiments and comparisons with advanced methods on scRNA-seq datasets demonstrate LRMGC's superior performance and reliability in uncovering cell type composition. Furthermore, a variety of downstream analyses, such as marker gene identification, functional enrichment analysis, rare cell recognition, and cell-cell communication, also demonstrate the effectiveness of LRMGC.

A graph learning approach for kernel-based system identification with manifold regularization

In the real world, most data are unlabeled, which drives the development of semi-supervised learning (SSL). Among SSL methods, least squares regression (LSR) has attracted attention for its simplicity and efficiency. However, existing semi-supervised LSR approaches suffer from challenges such as the insufficient use of unlabeled data, low pseudo-label accuracy, and inefficient label propagation. To address these issues, this paper proposes dual label propagation-driven least squares regression with feature selection, named DLPLSR, which is a pseudo-label-free SSL framework. DLPLSR employs a fuzzy-graph-based clustering strategy to capture global relationships among all samples, and manifold regularization preserves local geometric consistency, so that it implements the dual label propagation mechanism for comprehensive utilization of unlabeled data. Meanwhile, a dual-feature selection mechanism is established by integrating orthogonal projection for maximizing feature information with an ℓ2,1-norm regularization for eliminating redundancy, thereby jointly enhancing the discriminative power. Benefiting from these two designs, DLPLSR boosts learning performance without pseudo-labeling. Finally, the objective function admits an efficient closed-form solution solvable via an alternating optimization strategy. Extensive experiments on multiple benchmark datasets show the superiority of DLPLSR compared to state-of-the-art LSR-based SSL methods.

In recent years, the surge in data-driven applications across various domains has spurred heightened interest in semi-supervised learning applied to graphs. This surge is attributed to the ubiquitous presence of graph data structures in real-world contexts, such as social networks’ interpersonal relationships, recommender systems’ user behavior graphs, and bioinformatics’ molecular interaction networks. However, for certain data types like images, not only is there a dearth of explicit graph structure, but also the existence of multiple view description methods complicates matters further. The intricacies of multi-view data pose challenges in directly applying traditional semi-supervised learning techniques to graphs. Consequently, researchers have begun exploring the fusion of semi-supervised learning with deep learning to leverage its wealth of information and enhance model efficacy. Effectively amalgamating graph structures with multi-view data remains a challenging problem necessitating further research. This paper introduces the Linear projection Fused Graph-based Semi-supervised Classification (LFGSC) method tailored for multi-view data, building upon the Graph Convolutional Network (GCN) architecture. Firstly, for each view, we leverage a semi-supervised approach that provides the concurrent estimation of the corresponding graph and the flexible linear data representations in a low-dimensional feature space. Subsequently, an adaptive and unified graph is generated, followed by the utilization of a fully connected network to fuse the projected features further and reduce dimensionality. Finally, the fused features and graph are inputted into a GCN to conduct semi-supervised classification. During training, the model incorporates cross-entropy loss, manifold regularization loss, graph auto-encoder loss, and supervised contrastive loss. Leveraging linear transformation significantly diminishes the input feature dimensions for GCN, thereby achieving high accuracy while substantially reducing computational overhead. Furthermore, experimental results conducted on various bench-marked multi-view image datasets demonstrate the superiority of LFGSC over existing semi-supervised learning methods for multi-view scenarios. (Source code: https://github.com/BiJingjun/LFGSC.)

Selecting input data points in the context of high-dimensional, nonlinear, and complex data in Riemannian space is challenging. While optimal experimental design theory is well-established in Euclidean space, its extension to Riemannian manifolds remains underexplored. Li and Del Castillo recently obtained new theoretical results on D-optimal and G-optimal designs on Riemannian manifolds. This paper follows their framework to investigate A-optimal designs on such manifolds. We prove an equivalence theorem for A-optimality under the manifold regularization model. Based on this result, a sequential algorithm for identifying A-optimal designs on manifold data is developed. Numerical studies using both synthetic and real datasets show the validity of the proposed method.

Multi-label feature selection via nonlinear mapping and manifold regularization

Since the rapid progress in multimedia and sensor technologies, multiview clustering (MVC) has become a prominent research area within machine learning and data mining, experiencing significant advancements over recent decades. MVC is distinguished from single-view clustering by its ability to integrate complementary information from multiple distinct data perspectives and enhance clustering performance. However, the efficacy of MVC methods is predicated on the availability of complete views for all samples-an assumption that frequently fails in practical scenarios where data views are often incomplete. To surmount this challenge, various approaches to incomplete MVC (IMVC) have been proposed, with deep neural networks emerging as a favored technique for their representation learning ability. Despite their promise, previous methods commonly adopt sample-level (e.g., features) or affinity-level (e.g., graphs) guidance, neglecting the discriminative label-level guidance (i.e., pseudo-labels). In this work, we propose a novel deep IMVC method termed pseudo-label propagation for deep IMVC (PLP-IMVC), which integrates high-quality pseudo-labels from the complete subset of incomplete data with deep label propagation networks to obtain improved clustering results. In particular, we first design a local model (PLP-L) that leverages pseudo-labels to their fullest extent. Then, we propose a global model (PLP-G) that exploits manifold regularization to mitigate the label noises, promote view-level information fusion, and learn discriminative unified representations. Experimental results across eight public benchmark datasets and three evaluation metrics prove our method's efficacy, demonstrating superior performance compared to 18 advanced baseline methods.

With the rapid advancement of 3D scanning technology, point clouds have become a crucial data type in computer vision and machine learning. However, learning robust representations for point clouds remains a significant challenge due to their irregularity and sparsity. In this paper, we propose a novel Dual Manifold Regularization (DMR) framework that makes full use of the properties of positive and negative curvature in manifolds to improve the representation of point clouds. Specifically, we leverage DMR based on hyperbolic and hyperspherical manifolds to address the limitations of traditional single-manifold regularization techniques, including inadequate generalization ability and adaptability to data diversity, as well as the difficulty of capturing complex relationships between data. To begin, we utilize the tree-like structure of the hyperbolic manifold to model the part-whole hierarchical relationships within point clouds. This allows for a more comprehensive representation of the data, improving the model's capability to understand complex shapes. Additionally, we construct positive samples through topological consistency augmentation and employ contrastive learning techniques in the hyperspherical manifold to capture more discriminative features within the data. Our experimental results show that our method outperforms traditional supervised learning and single-manifold regularization techniques in point cloud analysis. Specifically, for shape classification, DMR achieves a new State-Of-The-Art (SOTA) performance with 94.8% Overall Accuracy (OA) on ModelNet40 and 90.7% OA on ScanObjectNN, surpassing the recent SOTA model without increasing the baseline parameters.

Accurate localization of the instrument tip within the hepatic vein is crucial for the success of transjugular intrahepatic portosystemic shunt (TIPS) procedures. Real-time tracking of the instrument tip in X-ray images is greatly influenced by vessel deformation due to patient's pose variation, respiratory motion, and puncture manipulation, frequently resulting in failed punctures. We propose a novel framework called deformable instrument tip tracking (DITT) to obtain the real-time tip positioning within the 3D deformable vasculature. First, we introduce a pose alignment module to improve the rigid matching between the preoperative vessel centerline and the intraoperative instrument centerline, in which the accurate matching of 3D/2D centerline features is implemented with an adaptive point sampling strategy. Second, a respiration compensation module using monoplane X-ray image sequences is constructed and provides the motion prior to predict intraoperative liver movement. Third, a deformation correction module is proposed to rectify the vessel deformation during procedures, in which a manifold regularization and the maximum likelihood-based acceleration are introduced to obtain the accurate and fast deformation learning. Experimental results on simulated and clinical datasets show an average tracking error of 1.59 0.57 mm and 1.67 0.54 mm, respectively. Our framework can track the tip in 3D vessel and dynamically overlap the branch roadmapping onto X-ray images to provide real-time guidance. Accurate and fast (43ms per frame) tip tracking with the proposed framework possesses a good potential for improving the outcomes of TIPS treatment and minimizes the usage of contrast agent.

Non-sparse multiple kernel learning is efficient but not directly able to be applied in a semi-supervised scenario; therefore, we extend it to semi-supervised learning by using a manifold regularization. The manifold regularization is based on a graph constructed on all the data samples including the labeled and the unlabeled, and forces the regularized classifier smooth along the graph. In this study, we propose the manifold regularized p-norm multiple kernels model and provide the solutions with proofs. The risk bound is briefly introduced based on the local Rademacher complexity. Experiments on several datasets and comparisons with several methods show that the efficiency of the proposed model to be used in semi-supervised scenario.

In the field of video image processing, high definition is one of the main directions for future development. Faced with the curse of dimensionality caused by the increasingly large amount of ultra-high-definition video data, effective dimensionality reduction techniques have become increasingly important. Linear discriminant analysis (LDA) is a supervised learning dimensionality reduction technique that has been widely used in data preprocessing for dimensionality reduction and video image processing tasks. However, traditional LDA methods are not suitable for the dimensionality reduction and processing of small high-dimensional samples. In order to improve the accuracy and robustness of linear discriminant analysis, this paper proposes a new distributed sparse manifold constraint (DSC) optimization LDA method, called DSCLDA, which introduces L2,0-norm regularization for local sparse feature representation and manifold regularization for global feature constraints. By iterating the hard threshold operator and transforming the original problem into an approximate non-convex sparse optimization problem, the manifold proximal gradient (ManPG) method is used as a distributed iterative solution. Each step of the algorithm has an explicit solution. Simulation experiments have verified the correctness and effectiveness of this method. Compared with several advanced sparse linear discriminant analysis methods, this method effectively improves the average classification accuracy by at least 0.90%.

Enhancing Stability of the Weakly Supervised Regression Algorithm Using Manifold Regularization and Fuzzy Clustering

Machine learning models have been widely used in the field of cutting tool wear identification, achieving favorable results. However, in actual industrial scenarios, obtaining sufficient labeled samples is time consuming and costly, while unlabeled samples are abundant and easy to collect. This situation significantly affects the model’s performance. To address this challenge, a novel semi-supervised method, based on long short-term memory (LSTM) networks, is provided. The proposed method leverages both small labeled and abundant unlabeled data to improve tool wear identification performance. The proposed method trains an initial tool wear regression model using LSTM, using a small amount of labeled samples. It then uses manifold regularization to generate pseudo-labels for the unlabeled samples. These pseudo-labeled samples are combined with the original labeled samples to retrain the MR–LSTM model iteratively to improve its performance. This process continues until a termination condition is met. The method considers the correlation between sample labels and feature structures, as well as the correlation between global and local sample labels. Experiments involving milling tool wear identification demonstrate that the proposed method significantly outperforms support vector regression (SVR) and recurrent neural network (RNN)-based methods, when a small amount of labeled samples and abundant unlabeled samples are available. The average R2 values in terms of the proposed method’s predicted results can reach above 0.95. The proposed method is a potential technique for low-cost tool wear identification, without the need to collect a large number of labeled samples.

A robust semi-supervised regressor with correntropy-induced manifold regularization and adaptive graph.

Diabetic retinopathy (DR) has long been recognized as a common complication of diabetes, making accurate automated grading of its severity essential. Color fundus photographs play a crucial role in the grading of DR. With the advancement of artificial intelligence technologies, numerous researchers have conducted studies on DR grading based on deep features and radiomic features extracted from color fundus photographs. We combine deep features and radiomic features to design a feature fusion algorithm. First, we utilize convolutional neural networks to extract deep features from color fundus photographs and employ radiomic methodologies to extract radiomic features. Subsequently, we design a label relaxation-based collaborative learning algorithm for feature fusion. We validate the effectiveness of the proposed method on two fundus image datasets: the DR1 Dataset and the MESSIDOR Dataset. The proposed method achieved 96.86 of AUC on DR1 and 96.34 of AUC on MESSIDOR, which are better than state-of-the-art methods. Also, the divergence between the training AUC and testing AUC increases substantially after the removal of manifold regularization. Label relaxation can enhance the distinguishability of training samples in the label space, thereby improving the model's classification accuracy. Additionally, graph constraints based on manifold learning methods can mitigate overfitting caused by label relaxation.

Semi-Supervised Stochastic Configuration Networks Based on Manifold Regularization Framework

Dynamic Brain Network Modeling Based on Nonlinear Low-Rank Manifold Regularization