Graph Regularized Non-negative Matrix Factorization Research Articles

Manifold ranking graph regularization non-negative matrix factorization with global and local structures

Non-negative matrix factorization (NMF) is a recently popularized technique for learning parts-based, linear representations of non-negative data. Although the decomposition rate of NMF is very fast, it still suffers from the following deficiency: It only revealed the local geometry structure; global geometric information of data set is ignored. This paper proposes a manifold ranking graph regularization non-negative matrix factorization with local and global geometric structure (MRLGNMF) to overcome the above deficiency. In particular, MRLGNMF induces manifold ranking to the non-negative matrix factorization with Sinkhorn distance. Numerical results show that the new algorithm is superior to the existing algorithm.

A novel framework for the NMF methods with experiments to unmixing signals and feature representation

Non-negative matrix factorization (NMF) can be used in clustering, feature representation or blind source separation. Many NMF methods have been developed including least squares (LS) error, Kullback–Leibler (KL) divergence, Itakura–Saito (IS) divergence, Bregman-divergence, α -divergence, β -divergence, γ -divergence, convex, constrained, graph-regularized NMFs. The main contribution of this paper is to develop a framework to generalize the existing NMF methods and also provide new NMF methods. This paper constructs a general optimization model and develops a generic updating rule with a simple structure using a surrogate function, which possesses similar properties as the standard NMF methods. The experimental results, obtained using several standard databases, demonstrate the power of the work in which some new methods provide performance superior to that of the other existing methods.

Prediction of Microbe-Disease Associations by Graph Regularized Non-Negative Matrix Factorization.

More and more evidence shows that microbes play crucial roles in human health and disease. The exploration of the relationship between microbes and diseases will help people to better understand the underlying pathogenesis and have important implications for disease diagnosis and prevention. However, the known associations between microbes and diseases are very less. We proposed a new method called non-negative matrix factorization microbe-disease associations (NMFMDA), which used Gaussian interaction profile kernel similarity measure, to calculate microbial similarity and disease similarity, and applied a logistic function to regulate disease similarity. And, based on the known microbe-disease associations, a graph-regularized non-negative matrix factorization model was utilized to simultaneously identify potential microbe-disease associations. Moreover, fivefold cross-validation was utilized to evaluate the performance of our method. It reached the reliable area under receiver operating characteristic curve (AUC) of 0.8891, higher than other state-of-the-art methods. Finally, the case studies on three complex human diseases (i.e., asthma, inflammatory bowel disease, and colon cancer) demonstrated the good performance of our method. In summary, our method can be considered as an effective computational model for predicting potential disease-microbe associations.

Sparse dual graph-regularized NMF for image co-clustering

Abstract Nonnegative matrix factorization (NMF) as fundamental technique for clustering has been receiving more and more attention. This is because it can effectively reduce high dimensional data and produce parts-based, linear image representations of nonnegative data. For practical clustering tasks, NMF ignores the geometric structures of both data manifold and feature manifold. In addition, recent research results showed that leveraging sparseness can greatly improve the ability of learning parts. Motivated by the two aspects above mentioned, we propose a novel co-clustering algorithm to enhance the clustering performance, called sparse dual graph-regularized nonnegative matrix factorization (SDGNMF). It aims for finding a parts-based, linear representation of the non-negative data and facilitating the learning tasks. SDGNMF jointly incorporates the dual graph-regularized and sparseness constraints as additional conditions to uncover the intrinsic geometrical, discriminative structures of the data space and feature space. The iterative updating scheme for the optimization problem of SDGNMF and its convergence proofs are also given in detail. Experimental results of clustering on three benchmark datasets demonstrated that SDGNMF algorithm outperforms the compared state-of-the-art methods in image co-clustering.

A Robust Manifold Graph Regularized Nonnegative Matrix Factorization Algorithm for Cancer Gene Clustering.

Detecting genomes with similar expression patterns using clustering techniques plays an important role in gene expression data analysis. Non-negative matrix factorization (NMF) is an effective method for clustering the analysis of gene expression data. However, the NMF-based method is performed within the Euclidean space, and it is usually inappropriate for revealing the intrinsic geometric structure of data space. In order to overcome this shortcoming, Cai et al. proposed a novel algorithm, called graph regularized non-negative matrices factorization (GNMF). Motivated by the topological structure of the GNMF-based method, we propose improved graph regularized non-negative matrix factorization (GNMF) to facilitate the display of geometric structure of data space. Robust manifold non-negative matrix factorization (RM-GNMF) is designed for cancer gene clustering, leading to an enhancement of the GNMF-based algorithm in terms of robustness. We combine the -norm NMF with spectral clustering to conduct the wide-ranging experiments on the three known datasets. Clustering results indicate that the proposed method outperforms the previous methods, which displays the latest application of the RM-GNMF-based method in cancer gene clustering.

Improved hypergraph regularized Nonnegative Matrix Factorization with sparse representation

As a commonly used data representation technique, Nonnegative Matrix Factorization (NMF) has received extensive attentions in the pattern recognition and machine learning communities over decades, since its working mechanism is in accordance with the way how the human brain recognizes objects. Inspired by the remarkable successes of manifold learning, more and more researchers attempt to incorporate the manifold learning into NMF for finding a compact representation,which uncovers the hidden semantics and respects the intrinsic geometric structure simultaneously. Graph regularized Nonnegative Matrix Factorization (GNMF) is one of the representative approaches in this category. The core of such approach is the graph, since a good graph can accurately reveal the relations of samples which benefits the data geometric structure depiction. In this paper, we leverage the sparse representation to construct a sparse hypergraph for better capturing the manifold structure of data, and then impose the sparse hypergraph as a regularization to the NMF framework to present a novel GNMF algorithm called Sparse Hypergraph regularized Nonnegative Matrix Factorization (SHNMF). Since the sparse hypergraph inherits the merits of both the sparse representation and the hypergraph model, SHNMF enjoys more robustness and can better exploit the high-order discriminant manifold information for data representation. We apply our work to address the image clustering issue for evaluation. The experimental results on five popular image databases show the promising performances of the proposed approach in comparison with the state-of-the-art NMF algorithms.

Multi-view clustering based on graph-regularized nonnegative matrix factorization for object recognition

Various datasets from sensors are used for object recognition, and different features may be extracted from the same dataset in processing. Different datasets thus describe representations or views of the same object. Fusing the information from this multi-view dataset can improve recognition performance. However, such different views have varying quality levels. In this paper, we discuss multi-view clustering based on graph-regularized nonnegative matrix factorization with fusing useful information effectively to improve recognition accuracy. Useful information is enhanced via graph embedding, and redundant information is removed using the orthogonal constraint in each view for clustering. Experimental results on several real datasets demonstrate the effectiveness of our approach in improving the clustering performance of datasets.

Molecular classification of prostate adenocarcinoma by the integrated somatic mutation profiles and molecular network

Prostate cancer is one of the most common cancers in men and a leading cause of cancer death worldwide, displaying a broad range of heterogeneity in terms of clinical and molecular behavior. Increasing evidence suggests that classifying prostate cancers into distinct molecular subtypes is critical to exploring the potential molecular variation underlying this heterogeneity and to better treat this cancer. In this study, the somatic mutation profiles of prostate cancer were downloaded from the TCGA database and used as the source nodes of the random walk with restart algorithm (RWRA) for generating smoothed mutation profiles in the STRING network. The smoothed mutation profiles were selected as the input matrix of the Graph-regularized Nonnegative Matrix Factorization (GNMF) for classifying patients into distinct molecular subtypes. The results were associated with most of the clinical and pathological outcomes. In addition, some bioinformatics analyses were performed for the robust subtyping, and good results were obtained. These results indicated that prostate cancers can be usefully classified according to their mutation profiles, and we hope that these subtypes will help improve the treatment stratification of this cancer in the future.

Image annotation using multi-view non-negative matrix factorization with different number of basis vectors

Automatic Image Annotation (AIA) helps image retrieval systems by predicting tags for images. In this paper, we propose an AIA system using Non-negative Matrix Factorization (NMF) framework. The NMF framework discovers a latent space, by factorizing data into a set of non-negative basis and coefficients. To model the images, multiple features are extracted, each one represents images from a specific view. We use multi-view graph regularization NMF and allow NMF to choose a different number of basis vectors for each view. For tag prediction, each test image is mapped onto the multiple latent spaces. The distances of images in these spaces are used to form a unified distance matrix. The weights of distances are learned automatically. Then a search-based method is used to predict tags based on tags of nearest neighbors’. We evaluate our method on three datasets and show that it is competitive with the current state-of-the-art methods.

Reducing Dimensionality in Text Mining using Conjugate Gradients and Hybrid Cholesky Decomposition

Generally, data mining in larger datasets consists of certain limitations in identifying the relevant datasets for the given queries. The limitations include: lack of interaction in the required objective space, inability to handle the data sets or discrete variables in datasets, especially in the presence of missing variables and inability to classify the records as per the given query, and finally poor generation of explicit knowledge for a query increases the dimensionality of the data. Hence, this paper aims at resolving the problems with increasing data dimensionality in datasets using modified non-integer matrix factorization (NMF). Further, the increased dimensionality arising due to non-orthogonally of NMF is resolved with Cholesky decomposition (cdNMF). Initially, the structuring of datasets is carried out to form a well-defined geometric structure. Further, the complex conjugate values are extracted and conjugate gradient algorithm is applied to reduce the sparse matrix from the data vector. The cdNMF is used to extract the feature vector from the dataset and the data vector is linearly mapped from upper triangular matrix obtained from the Cholesky decomposition. The experiment is validated against accuracy and normalized mutual information (NMI) metrics over three text databases of varied patterns. Further, the results prove that the proposed technique fits well with larger instances in finding the documents as per the query, than NMF, neighborhood preserving: nonnegative matrix factorization (NPNMF), multiple manifolds non-negative matrix factorization (MMNMF), robust non-negative matrix factorization (RNMF), graph regularized non-negative matrix factorization (GNMF), hierarchical non-negative matrix factorization (HNMF) and cdNMF.

ℓ1/2-norm regularized nonnegative low-rank and sparse affinity graph for remote sensing image segmentation

Segmentation of real-world remote sensing images is a challenge due to the complex texture information with high heterogeneity. Thus, graph-based image segmentation methods have been attracting great attention in the field of remote sensing. However, most of the traditional graph-based approaches fail to capture the intrinsic structure of the feature space and are sensitive to noises. A l1/2-norm regularization-based graph segmentation method is proposed to segment remote sensing images. First, we use the occlusion of the random texture model (ORTM) to extract the local histogram features. Then, a l1/2-norm regularized low-rank and sparse representation (L1/2NNLRS) is implemented to construct a l1/2-regularized nonnegative low-rank and sparse graph (L1/2NNLRS-graph), by the union of feature subspaces. Moreover, the L1/2NNLRS-graph has a high ability to discriminate the manifold intrinsic structure of highly homogeneous texture information. Meanwhile, the L1/2NNLRS representation takes advantage of the low-rank and sparse characteristics to remove the noises and corrupted data. Last, we introduce the L1/2NNLRS-graph into the graph regularization nonnegative matrix factorization to enhance the segmentation accuracy. The experimental results using remote sensing images show that when compared to five state-of-the-art image segmentation methods, the proposed method achieves more accurate segmentation results.

Online graph regularized non-negative matrix factorization for large-scale datasets

Non-negative matrix factorization (NMF) has been widely used to reduce dimensionality of data in image processing and various applications. Incorporating the geometric structure into NMF, graph regularized non-negative matrix factorization (GNMF) has shown significant performance improvement in comparison to conventional NMF. However, both NMF and GNMF require the data matrix to reside in the memory, which gives rise to tremendous pressure for computation and storage. Moreover, this problem becomes more serious if the scale of datasets increases dramatically. In this paper, we propose an online GNMF (OGNMF) method to process the incoming data in an incremental manner, i.e., OGNMF processes one data point or one chunk of data points one by one. By utilizing buffering and random projection tree strategy, OGNMF scales gracefully to large-scale datasets. Experimental results on popular text corpora and image databases demonstrate that OGNMF achieves better performance than the existing online NMF algorithms in terms of both accuracy and normalized mutual information, and outperforms the existing batch GNMF algorithms in terms of scalability.

Manifold-Preserving Common Subspace Factorization for Feature Matching

A method, called Manifold-preserving Common Subspace Factorization, is presented which can be used for feature matching. Motivated by the Graph Regularized Non-negative Matrix Factorization (GNMF) algorithm (Deng et al. 2011), we developed GNMF algorithm by considering a joint factorization of the two feature matrices, which share a common basis matrix. An iterative multiplicative updating algorithm is proposed to optimize the objective, and its convergence is guaranteed theoretically. Our feature matching algorithm operates on the new representations in the common subspace generated by basis vectors. Experiments are conducted on the synthetic and real-world data. The results show that the Manifold-preserving common subspace factorization algorithm provides better matching rates than other matrix-factorization techniques.

Local coordinate based graph-regularized NMF for image representation

Non-negative matrix factorization (NMF) has been a powerful data representation tool which has been widely applied in pattern recognition and computer vision due to its simplicity and effectiveness. However, existing NMF methods suffer from one or both of the following deficiencies: (1) they cannot theoretically guarantee the decomposition results to be sparse, and (2) they completely neglect geometric structure of data, especially when some examples are heavily corrupted. In this paper, we propose a local coordinate based graph regularized NMF method (LCGNMF) to simultaneously overcome both deficiencies. In particular, LCGNMF enforces the learned coefficients to be sparse by incorporating the local coordinate constraint over both factors meanwhile preserving the geometric structure of the data by incorporating graph regularization. To enhance the robustness of NMF, LCGNMF removes the effect of the outliers via the maximum correntropy criterion (MCC). LCGNMF is difficult because the MCC induced objective function is neither quadratic nor convex. We therefore developed a multiplicative update rule to solve LCGNMF and theoretically proved its convergence. Experiments of image clustering on several popular image datasets verify the effectiveness of LCGNMF compared to the representative methods in quantities.

Semi-supervised concept factorization for document clustering

Nonnegative Matrix Factorization (NMF) and Concept Factorization (CF) are two popular methods for finding the low-rank approximation of nonnegative matrix. Different from NMF, CF can be applied not only to the matrix containing negative values but also to the kernel space. Based on NMF and CF, many methods, such as Graph regularized Nonnegative Matrix Factorization (GNMF) and Locally Consistent Clustering Factorization (LCCF) can significantly improve the performance of clustering. Unfortunately, these are unsupervised learning methods. In order to enhance the clustering performance with the supervisory information, a Semi-Supervised Concept Factorization (SSCF) is proposed in this paper by incorporating the pairwise constraints into CF as the reward and penalty terms, which can guarantee that the data points belonging to a cluster in the original space are still in the same cluster in the transformed space. By comparing with the state-of-the-arts algorithms (KM, NMF, CF, LCCF, GNMF, PCCF), experimental results on document clustering show that the proposed algorithm has better performance in terms of accuracy and mutual information.

Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization

Nonnegative matrix factorization (NMF), a popular part-based representation technique, does not capture the intrinsic local geometric structure of the data space. Graph regularized NMF (GNMF) was recently proposed to avoid this limitation by regularizing NMF with a nearest neighbor graph constructed from the input data set. However, GNMF has two main bottlenecks. First, using the original feature space directly to construct the graph is not necessarily optimal because of the noisy and irrelevant features and nonlinear distributions of data samples. Second, one possible way to handle the nonlinear distribution of data samples is by kernel embedding. However, it is often difficult to choose the most suitable kernel. To solve these bottlenecks, we propose two novel graph-regularized NMF methods, AGNMFFS and AGNMFMK, by introducing feature selection and multiple-kernel learning to the graph regularized NMF, respectively. Instead of using a fixed graph as in GNMF, the two proposed methods learn the nearest neighbor graph that is adaptive to the selected features and learned multiple kernels, respectively. For each method, we propose a unified objective function to conduct feature selection/multi-kernel learning, NMF and adaptive graph regularization simultaneously. We further develop two iterative algorithms to solve the two optimization problems. Experimental results on two challenging pattern classification tasks demonstrate that the proposed methods significantly outperform state-of-the-art data representation methods.

Co-clustering over multiple dynamic data streams based on non-negative matrix factorization

Clustering multiple data streams has become an active area of research with many practical applications. Most of the early work in this area focused on one-sided clustering, i.e., clustering data streams based on feature correlation. However, recent research has shown that data streams can be grouped based on the distribution of their features, while features can be grouped based on their distribution across data streams. In this paper, an evolutionary clustering algorithm is proposed for multiple data streams using graph regularization non-negative matrix factorization (EC-NMF) in which the geometric structure of both the data and feature manifold is considered. Instead of directly clustering multiple data streams periodically, EC-NMF works in the low-rank approximation subspace and incorporates prior knowledge from historic results with temporal smoothness. Furthermore, we develop an iterative algorithm and provide convergence and correctness proofs from a theoretical standpoint. The effectiveness and efficiency of the algorithm are both demonstrated in experiments on real and synthetic data sets. The results show that the proposed EC-NMF algorithm outperforms existing methods for clustering multiple data streams evolving over time.

Limited-Memory Fast Gradient Descent Method for Graph Regularized Nonnegative Matrix Factorization

Graph regularized nonnegative matrix factorization (GNMF) decomposes a nonnegative data matrix to the product of two lower-rank nonnegative factor matrices, i.e., and () and aims to preserve the local geometric structure of the dataset by minimizing squared Euclidean distance or Kullback-Leibler (KL) divergence between X and WH. The multiplicative update rule (MUR) is usually applied to optimize GNMF, but it suffers from the drawback of slow-convergence because it intrinsically advances one step along the rescaled negative gradient direction with a non-optimal step size. Recently, a multiple step-sizes fast gradient descent (MFGD) method has been proposed for optimizing NMF which accelerates MUR by searching the optimal step-size along the rescaled negative gradient direction with Newton's method. However, the computational cost of MFGD is high because 1) the high-dimensional Hessian matrix is dense and costs too much memory; and 2) the Hessian inverse operator and its multiplication with gradient cost too much time. To overcome these deficiencies of MFGD, we propose an efficient limited-memory FGD (L-FGD) method for optimizing GNMF. In particular, we apply the limited-memory BFGS (L-BFGS) method to directly approximate the multiplication of the inverse Hessian and the gradient for searching the optimal step size in MFGD. The preliminary results on real-world datasets show that L-FGD is more efficient than both MFGD and MUR. To evaluate the effectiveness of L-FGD, we validate its clustering performance for optimizing KL-divergence based GNMF on two popular face image datasets including ORL and PIE and two text corpora including Reuters and TDT2. The experimental results confirm the effectiveness of L-FGD by comparing it with the representative GNMF solvers.

A New Method to Construct Neighbor Graph for Non-negative Matrix Factorization on Manifold

Recently manifold method has achieved a lot of attentions in face recognition, by constructing a neighbor graph, the algorithms can preserve the geometric structure in the data. So the Graph regularized Non-negative Matrix Factorization (GNMF) which imposed manifold into NMF was achieved grate successful. Experiments show the construction of the neighbor graph is critical for GNMF especially when discriminant information is unknown. In this paper, we propose a new method to construct the neighbor graph for GNMF, by considering the 2D information of pictures, our method would better preserve the geometrical information, experiments show that GNMF with achieves better performance than traditional neighbor graph construct method in GNMF.

Low-Rank Matrix Approximation with Manifold Regularization

This paper proposes a new model of low-rank matrix factorization that incorporates manifold regularization to the matrix factorization. Superior to the graph-regularized nonnegative matrix factorization, this new regularization model has globally optimal and closed-form solutions. A direct algorithm (for data with small number of points) and an alternate iterative algorithm with inexact inner iteration (for large scale data) are proposed to solve the new model. A convergence analysis establishes the global convergence of the iterative algorithm. The efficiency and precision of the algorithm are demonstrated numerically through applications to six real-world datasets on clustering and classification. Performance comparison with existing algorithms shows the effectiveness of the proposed method for low-rank factorization in general.