Semi-supervised Non-negative Matrix Factorization Research Articles

Semi-Supervised Nonnegative Matrix Factorization via Constraint Propagation.

As is well known, nonnegative matrix factorization (NMF) is a popular nonnegative dimensionality reduction method which has been widely used in computer vision, document clustering, and image analysis. However, traditional NMF is an unsupervised learning mode which cannot fully utilize the priori or supervised information. To this end, semi-supervised NMF methods have been proposed by incorporating the given supervised information. Nevertheless, when little supervised information is available, the improved performance will be limited. To effectively utilize the limited supervised information, this paper proposed a novel semi-supervised NMF method (CPSNMF) with pairwise constraints. The method propagates both the must-link and cannot-link constraints from the constrained samples to unconstrained samples, so that we can get the constraint information of the entire data set. Then, this information is reflected to the adjustment of data weight matrix. Finally, the weight matrix is incorporated as a regularization term to the NMF objective function. Therefore, the proposed method can fully utilize the constraint information to keep the geometry of the data distribution. Furthermore, the proposed CPSNMF is explored with two formulations and corresponding update rules are provided to solve the optimization problems. Thorough experiments on standard databases show the superior performance of the proposed method.

Document Classification using Matrix Decomposition with Varied Viewpoints

Classification results are not unique and can vary according to the user’s viewpoint. If a document classification system ignores the user’s viewpoints, classification will be different from the result desired by the user, and the difference between the user’s desired result and the system’s produced result can cause some inhibitions and oversights in information retrieval. Extracting the user’s viewpoints from the classification examples performed preliminarily by the user allows us to configure classifications that reflect the user’s desire. In this study, we propose four methods to extract viewpoints and three methods to classify documents using Nonnegative Matrix Factorization (NMF) matrix decomposition. We exhibit the results of comparative experiments with the original NMF, Semi-Supervised NMF (SSNMF) and our proposed methods.

Label propagation based semi-supervised non-negative matrix factorization for feature extraction

As a feature extraction method, Non-negative Matrix Factorization (NMF) has attracted much attention due to its effective application to data classification and clustering tasks. In this paper, a novel algorithm named Label propagation based Semi-supervised Non-negative Matrix Factorization (LpSNMF) is proposed. For the sake of making full use of label information, our LpSNMF algorithm takes the distribution relationships between the labeled and unlabeled data samples into consideration and integrates the procedures of class label propagation and matrix factorization into a joint framework. Moreover, an iterative updating optimization scheme is developed to solve the objective function of the proposed LpSNMF and the convergence of our scheme is also proven. Extensive experimental results on several UCI benchmark data sets and four image data sets (such as Yale, CMU PIE, UMIST, and COIL20) demonstrate that by propagating the label information and factorizing the matrix alternately, our algorithm can obtain better performance than some other algorithms.

An Improved Semi Supervised Nonnegative Matrix Factorization Based Tumor Clustering with Efficient Infomax ICA Based Gene Selection

In cancer class discovery, tumor clustering becomes an active area of research. An exact identification of the type of tumors is essential for efficient treatment of cancer at the prior stage. Numerous techniques have been proposed and used to examine gene expression data, which clustering algorithms are inappropriate to many real-world problems whereas less amount of domain information only available. Adding the domain knowledge can help a clustering algorithm, to improve the quality of clustering result. In this paper, a semi-supervised non-negative matrix factorization (SS-NMF) structure is proposed for selected gene clustering and the genes are selected using proposed infomax ICA approach. Proposed system gives the tumor clustering result in terms of representing pairwise constraints on a small number of data objects that are to be clustered together while specifying whether “cannot” or “must”. Use of iterative procedure perform symmetric trifactorization data similarity matrix to conclude the cluster result. Finally, the experiments on the gene expression dataset verified that the proposed scheme can attain improved clustering results

Class-Driven Non-Negative Matrix Factorization for Image Representation

Non-negative matrix factorization (NMF) is a useful technique to learn a parts-based representation by decomposing the original data matrix into a basis set and coefficients with non-negative constraints. However, as an unsupervised method, the original NMF cannot utilize the discriminative class information. In this paper, we propose a semi-supervised class-driven NMF method to associate a class label with each basis vector by introducing an inhomogeneous representation cost constraint. This constraint forces the learned basis vectors to represent better for their own classes but worse for the others. Therefore, data samples in the same class will have similar representations, and consequently the discriminability in new representations could be boosted. Some experiments carried out on several standard databases validate the effectiveness of our method in comparison with the state-of-the-art approaches.

Semi-supervised non-negative matrix factorization for image clustering with graph Laplacian

Non-negative matrix factorization (NMF) plays an important role in multivariate data analysis, and has been widely applied in information retrieval, computer vision, and pattern recognition. NMF is an effective method to capture the underlying structure of the data in the parts-based low dimensional representation space. However, NMF is actually an unsupervised method without making use of supervisory information of data. In recent years, semi-supervised learning has received a lot of attentions, because partial label information can significantly improve learning quality of the algorithms. In this paper, we propose a novel semi-supervised non-negative matrix factorization (SEMINMF) algorithm, which not only utilizes the local structure of the data characterized by the graph Laplacian, but also incorporates the label information as the fitting constraints to learn. Hence, it can learn from labeled and unlabeled data. By this means our SEMINMF can obtain a more discriminative powerful representation space. Experimental results show the effectiveness of our proposed novel method in comparison to the state-of-the-art algorithms on several real world applications.

A nonnegative matrix factorization framework for semi-supervised document clustering with dual constraints

In this paper, we propose a new semi-supervised co-clustering algorithm Orthogonal Semi-Supervised Nonnegative Matrix Factorization (OSS-NMF) for document clustering. In this new approach, the clustering process is carried out by incorporating both prior domain knowledge of data points (documents) in the form of pair-wise constraints and category knowledge of features (words) into the NMF co-clustering framework. Under this framework, the clustering problem is formulated as the problem of finding the local minimizer of objective function, taking into account the dual prior knowledge. The update rules are derived, and an iterative algorithm is designed for the co-clustering process. Theoretically, we prove the correctness and convergence of our algorithm and demonstrate its mathematical rigorous. Our experimental evaluations show that the proposed document clustering model presents remarkable performance improvements with those constraints.

Semi-supervised Nonnegative Matrix Factorization for gene expression deconvolution: A case study

Heterogeneity in sample composition is an inherent issue in many gene expression studies and, in many cases, should be taken into account in the downstream analysis to enable correct interpretation of the underlying biological processes. Typical examples are infectious diseases or immunology-related studies using blood samples, where, for example, the proportions of lymphocyte sub-populations are expected to vary between cases and controls.Nonnegative Matrix Factorization (NMF) is an unsupervised learning technique that has been applied successfully in several fields, notably in bioinformatics where its ability to extract meaningful information from high-dimensional data such as gene expression microarrays has been demonstrated. Very recently, it has been applied to biomarker discovery and gene expression deconvolution in heterogeneous tissue samples.Being essentially unsupervised, standard NMF methods are not guaranteed to find components corresponding to the cell types of interest in the sample, which may jeopardize the correct estimation of cell proportions. We have investigated the use of prior knowledge, in the form of a set of marker genes, to improve gene expression deconvolution with NMF algorithms. We found that this improves the consistency with which both cell type proportions and cell type gene expression signatures are estimated. The proposed method was tested on a microarray dataset consisting of pure cell types mixed in known proportions. Pearson correlation coefficients between true and estimated cell type proportions improved substantially (typically from about 0.5 to approximately 0.8) with the semi-supervised (marker-guided) versions of commonly used NMF algorithms. Furthermore known marker genes associated with each cell type were assigned to the correct cell type more frequently for the guided versions. We conclude that the use of marker genes improves the accuracy of gene expression deconvolution using NMF and suggest modifications to how the marker gene information is used that may lead to further improvements.

Non-Negative Matrix Factorization for Semisupervised Heterogeneous Data Coclustering

Coclustering heterogeneous data has attracted extensive attention recently due to its high impact on various important applications, such us text mining, image retrieval, and bioinformatics. However, data coclustering without any prior knowledge or background information is still a challenging problem. In this paper, we propose a Semisupervised Non-negative Matrix Factorization (SS-NMF) framework for data coclustering. Specifically, our method computes new relational matrices by incorporating user provided constraints through simultaneous distance metric learning and modality selection. Using an iterative algorithm, we then perform trifactorizations of the new matrices to infer the clusters of different data types and their correspondence. Theoretically, we prove the convergence and correctness of SS-NMF coclustering and show the relationship between SS-NMF with other well-known coclustering models. Through extensive experiments conducted on publicly available text, gene expression, and image data sets, we demonstrate the superior performance of SS-NMF for heterogeneous data coclustering.

Semi-Supervised Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a popular method for low-rank approximation of nonnegative matrix, providing a useful tool for representation learning that is valuable for clustering and classification. When a portion of data are labeled, the performance of clustering or classification is improved if the information on class labels is incorporated into NMF. To this end, we present semi-supervised NMF (SSNMF), where we jointly incorporate the data matrix and the (partial) class label matrix into NMF. We develop multiplicative updates for SSNMF to minimize a sum of weighted residuals, each of which involves the nonnegative 2-factor decomposition of the data matrix or the label matrix, sharing a common factor matrix. Experiments on document datasets and EEG datasets in BCI competition confirm that our method improves clustering as well as classification performance, compared to the standard NMF, stressing that semi-supervised NMF yields semi-supervised feature extraction.

Non-negative matrix factorization for semi-supervised data clustering

Traditional clustering algorithms are inapplicable to many real-world problems where limited knowledge from domain experts is available. Incorporating the domain knowledge can guide a clustering algorithm, consequently improving the quality of clustering. In this paper, we propose SS-NMF: a semi-supervised non-negative matrix factorization framework for data clustering. In SS-NMF, users are able to provide supervision for clustering in terms of pairwise constraints on a few data objects specifying whether they “must” or “cannot” be clustered together. Through an iterative algorithm, we perform symmetric tri-factorization of the data similarity matrix to infer the clusters. Theoretically, we show the correctness and convergence of SS-NMF. Moveover, we show that SS-NMF provides a general framework for semi-supervised clustering. Existing approaches can be considered as special cases of it. Through extensive experiments conducted on publicly available datasets, we demonstrate the superior performance of SS-NMF for clustering.