Local Manifold Regularization Research Articles

Class-driven nonnegative matrix factorization with manifold regularization for data clustering

Nonnegative matrix factorization (NMF) is an effective technique to extract the underlying low-dimensional structure of data by utilizing its parts-based representation, which has been widely used in feature extraction and machine learning. However, NMF is an unsupervised learning algorithm without utilizing the discriminative prior information. In this paper, we put forward a new class-driven NMF with manifold regularization (MCDNMF) algorithm, which incorporates both the local manifold regularization and the label information of data into the NMF model. Specifically, MCDNMF not only encodes the local geometrical structure of data space by using the manifold regularization, but also takes the available label information by introducing the class-driven constraint. This class-driven constraint forces the new representations of data points to be more similar within the same class while different between other classes. Therefore, the discriminative abilities of clustering are greatly boosted. Experimental results on several datasets validate the effectiveness of proposed MCDNMF in comparison with the other state-of-the-art methods.

Manifold regularized deep canonical variate analysis with interpretable attribute guidance for three-phase flow process monitoring

Oil–gas–water three-phase flow has multiple flow states, exhibiting dynamic, nonlinear, and instantaneous behaviors. Monitoring and analysis of flow state are crucial for ensuring safe operation of industrial processes. However, the absence of a universal definition for three-phase flow states, owing to the complex flow characteristics and structures, leads to that labeling three-phase flow states by categories is impractical. For comprehensive monitoring and analysis, the flow states should be described from both global (e.g., the dominant phase) and local (e.g., the interphase structure) aspects. Given the aforementioned challenges, the work aims to address the accurate identification of typical three-phase flow states and to meticulously monitor and analyze the continuous evolutionary process of three-phase flow. To achieve it, the manifold regularized deep canonical variate analysis with attribute guidance (Ag-MRDCVA) is proposed, which not only introduces an innovative monitoring paradigm that designs state attributes for process description but also facilitates knowledge transfer from typical flow states to transition states. For enhancing the model’s feature embedding capabilities, an attribute guidance module is introduced to increase supervision information at both class and attribute levels. Then, convolutional neural network (CNN) backbones are developed with a dual objective of global serial correlation and local manifold regularization, which capture spatiotemporal information at both global and local scales, facilitating effective attribute encoding and characterization of flow states. Finally, the attribute evolution heat map and a monitoring metric (MM) collectively offer a compelling and comprehensive analysis of the three-phase flow process. Extensive experiments demonstrate that Ag-MRDCVA surpasses existing methods, showcasing its ability to minutely monitor the process while providing reasonable explanations.

Multiview Data Clustering with Similarity Graph Learning Guided Unsupervised Feature Selection.

In multiview data clustering, consistent or complementary information in the multiview data can achieve better clustering results. However, the high dimensions, lack of labeling, and redundancy of multiview data certainly affect the clustering effect, posing a challenge to multiview clustering. A clustering algorithm based on multiview feature selection clustering (MFSC), which combines similarity graph learning and unsupervised feature selection, is designed in this study. During the MFSC implementation, local manifold regularization is integrated into similarity graph learning, with the clustering label of similarity graph learning as the standard for unsupervised feature selection. MFSC can retain the characteristics of the clustering label on the premise of maintaining the manifold structure of multiview data. The algorithm is systematically evaluated using benchmark multiview and simulated data. The clustering experiment results prove that the MFSC algorithm is more effective than the traditional algorithm.

Semi-supervised local manifold regularization model based on dual representation for industrial soft sensor development

Data-driven soft sensors are crucial for predicting key quality variables in the process industry. In most processes, the dynamic characteristics are obvious, and the relationship between the primary and secondary variables is strongly nonlinear. Moreover, the worst thing is that labeled samples are usually scarce due to certain technical difficulties or measurement costs. To deal with these issues, this paper first proposes a semi-supervised manifold regularization model based on dual representation (SsMRM-DR). Then, a semi-supervised local manifold regularization model based on dual representation (SsLMRM-DR) is further proposed from the viewpoint of local manifold regularization. In the SsLMRM-DR, a space–time weighted similarity calculation method is designed to reduce the influence of measurement noise on similarity calculation. At the same time, a local manifold regularization method is developed to use unlabeled samples more efficiently and to improve the prediction accuracy and computational efficiency of the SsLMRM-DR. A numerical example and a real industrial process are used to evaluate the performance of the proposed schemes. The results show that SsLMRM-DR is effective and it has a good application prospect of soft sensors in the industry.

Unsupervised domain adaptation based on adaptive local manifold learning

Transfer learning is known to an effective method dealing with domain shift. When the same task is shared in different domains, it is usually called domain adaptation. The problem of distribution difference is inevitable in domain adaption works. The subspace method can transform the data into a new feature representation, which is helpful to reduce the distribution differences. At present, many researchers have made extensive exploration on subspace learning in domain adaptation works. The weakness of many existed domain adaptation methods based on subspace learning either ignores the local manifold information or has the problem of parameter selection in local manifold regularization term which may limit the effectiveness of cross - domain image classification. Therefore, a novel transfer learning method termed unsupervised domain adaptation based on adaptive local manifold learning (UDA-ALML) is proposed in this paper, which is mainly utilized to cross-domain image classification. For the sake of preserving the structure information of original data, the proposed method combines sparse representation, manifold learning and low rank representation to learn the transformation matrix. To be specific, the weight matrix in traditional local manifold regularization term is replaced by the reconstruction coefficient matrix. Large quantities of experiments show that it has a remarkable performance in cross-domain image recognition.

Depth Map Reconstruction and Enhancement With Local and Patch Manifold Regularized Deep Depth Priors

The depth map captured by depth sensors (e.g., the time of flight (ToF) and Kinect) is often prone to low resolution, degradation, noise, and poor quality. This paper proposes a novel model for the robust depth estimation of RGB-D images through local and nonlocal manifold regularizations. The first stage called deep depth prior manifold (DDPM), is inspired partly by the deep depth prior (DDP) model, that is a deep convolutional neural network (CNN) integrated with a local manifold regularization term. The local neighboring relationships between depth pixels and color images are employed to promote smoothing in the results. The Laplacian Eigenmap technique used for local manifold modeling produces over-smooth depth map. To improve the quality of the reconstructed image, a nonlocal manifold modeling stage was suggested, where the similarity between the depth and the corresponding color image is determined by characterizing their matching aspects. These objectives are aggregated within an optimization problem. Moreover, to extract edges better considering visual nonlocal characteristics, the structured low-rank Hankel approximation was adopted to better eliminate depth degradations, and to extract highly promoted edges and sharp points. Three types of the degradations were handling in this work, containing undersampling, ToF-like, and Kinect-like degradations. Experimental results indicate that the proposed method outperformed the state-of-the-art restoration techniques on standard benchmark images, in terms of well-known criteria like PSNR.

Free-Breathing & Ungated Cardiac MRI Using Iterative SToRM (i-SToRM).

We introduce a local manifold regularization approach to recover dynamic MRI data from highly undersampled measurements. The proposed scheme relies on the manifold structure of local image patches at the same spatial location in a free-breathing cardiac MRI dataset; this approach is a generalization of the SmooThness Regularization on Manifolds (SToRM) scheme that exploits the global manifold structure of images in the dataset. Since the manifold structure of the patches varies depending on the spatial location and is often considerably simpler than the global one, this approach significantly reduces the data demand, facilitating the recovery from shorter scans. Since the navigator-based estimation of manifold structure pursued in SToRM is not feasible in this setting, a reformulation of SToRM is introduced. Specifically, the regularization term of the cost function involves the sum of robust distances between images sub-patches in the dataset. The optimization algorithm alternates between updating the images and estimating the manifold structure of the image patches. The utility of the proposed scheme is demonstrated in the context of in-vivo prospective free-breathing cardiac CINE MRI imaging with multichannel acquisitions and simulated phantoms. The new framework facilitates a reduction in scan time, as compared to the SToRM strategy.

Salient object detection by local and global manifold regularized SVM model

In this paper, a bottom-up saliency detection algorithm is proposed by introducing the manifold regularized Support Vector Machine (SVM) model. The local manifold regularization can well capture local saliency cues. However, its local-to-global mechanism often ignores the completeness of salient objects especially when the contrast between the foreground and background is low, while the global regularization can discover the long-range semantic structure but easily neglect the details. To comprehensively utilize their advantages, we respectively construct a local regularizer (LR) and a global regularizer (GR), and incorporate them into the SVM models, which are then self-trained across superpixels by pseudo labeled ones. The LR-SVM and GR-SVM models are further refined by updating training sample with higher-level object proposals. Extensive experiments on five benchmark datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art saliency detection methods. In addition, we show that the manifold regularized SVM model can be easily applied to some existing saliency models and achieve significant performance improvement.

Semi-supervised least squares nonnegative matrix factorization and graph-based extension

Abstract Nonnegative matrix factorization (NMF) has been widely used in information retrieval, computer vision, and face recognition because it captures the underlying intrinsic structure of data by using its parts-based representations in the low dimensional space. In this paper, we first propose a novel semi-supervised least squares NMF (SLSNMF) method. This SLSNMF uses hyperplanes to separate the labelled points from different classes and minimizes the least squares loss. By considering the maximum margin principle, the discriminative abilities of clustering representations are greatly enhanced. We further present a graph-based least squares NMF (GLSNMF) method by incorporating the local manifold regularization into SLSNMF. Clustering experiments on five popular databases verify the effectiveness of our proposed SLSNMF and GLSNMF compared to the other state-of-the-art methods.

Graph-based discriminative nonnegative matrix factorization with label information

Nonnegative matrix factorization (NMF) is a very effective technique for image representation, which has been widely applied in computer vision and pattern recognition. This is because it can capture the underlying intrinsic structure of data by using its parts-based representations in the low dimensional space. However, NMF is an unsupervised learning method without utilizing both the available label information and the manifold geometrical structure of data space. In this paper, we propose a new semi-supervised graph-based discriminative nonnegative matrix factorization (GDNMF) method, which incorporates the local manifold regularization and the label information of the data into the NMF. GDNMF not only encodes the local geometrical structure of the data space by constructing k-nearest graph, but also takes into account the available label information. Thus, the discriminative abilities of clustering representations are greatly enhanced. Clustering experiments on five popular databases verify the effectiveness of our proposed GDNMF compared to the other state-of-the-art methods.

Graph-based discriminative concept factorization for data representation

Nonnegative Matrix Factorization (NMF) and Concept Factorization (CF) have been widely used for different purposes such as feature learning, dimensionality reduction and image clustering in data representation. However, CF is a variant of NMF, which is an unsupervised learning method without making use of the available label information to guide the clustering process. In this paper, we put forward a semi-supervised discriminative concept factorization (SDCF) method, which utilizes the limited label information of the data as a discriminative constraint. This constraint forces the representation of data points within the same class should be very close together or aligned on the same axis in the new representation. Furthermore, in order to utilize the local manifold regularization, we propose a novel semi-supervised graph-based discriminative concept factorization (GDCF) method, which incorporates the local manifold regularization and the label information of the data into the CF to improve the performance of CF. GDCF not only encodes the local geometrical structure of the data space by constructing K-nearest graph, but also takes into account the available label information. Thus, the discriminative abilities of data representations are enhanced in the clustering tasks. Experimental results on several databases expose the strength of our proposed SDCF and GDCF methods compared to the state-of-the-art methods.

Manifold regularized non-negative matrix factorization with label information

Non-negative matrix factorization (NMF) as a popular technique for finding parts-based, linear representations of non-negative data has been successfully applied in a wide range of applications, such as feature learning, dictionary learning, and dimensionality reduction. However, both the local manifold regularization of data and the discriminative information of the available label have not been taken into account together in NMF. We propose a new semisupervised matrix decomposition method, called manifold regularized non-negative matrix factorization (MRNMF) with label information, which incorporates the manifold regularization and the label information into the NMF to improve the performance of NMF in clustering tasks. We encode the local geometrical structure of the data space by constructing a nearest neighbor graph and enhance the discriminative ability of different classes by effectively using the label information. Experimental comparisons with the state-of-the-art methods on theCOIL20, PIE, Extended Yale B, and MNIST databases demonstrate the effectiveness of MRNMF.