Nonnegative Matrix Factorization Model Research Articles

Nonnegative Matrix Factorization models for knowledge extraction from biomedical and other real world data

AbstractInspect data for searching valuable information hidden in represents a key aspect in several fields. Fortunately, most of the available data presents an embedded mathematical structure which can be profitably exploited to better investigate latent patterns hidden in them.Dimensionality Reduction (DR) approaches represent one of the most suitable instrument to untangle latent information. These techniques aim to represent data under analysis onto a low‐dimensional space allowing to consider most of all of intrinsic knowledge as ideal sources (namely basis) of the process under consideration.In this work we consider Nonnegative Matrix Factorizations (NMFs), which prove to be the most effective among DR approaches in analyzing real‐life nonnegative data.NMF simulates the human part‐based learning process which states that parts are combined additively to form a whole. Some variants of NMF will be also presented as minimization tasks to which regularization terms can be added in accordance to some additional characteristics (such as sparsity or orthogonality).We investigate significant computational and interpretative aspects related to NMF according to different application domains, with a specific attention to the analysis of biological data. Moreover we present a new NMF model designed for microarray data analysis that incorporates specific biological proprieties as different constraints. Since NMF and its variants are daily used in several application domains, we conclude stressing how NMF and its constrained variants work in some real life applications, showing some original works related to the analysis of data from engineering field.

Spectral-Spatial Constrained Nonnegative Matrix Factorization for Spectral Mixture Analysis of Hyperspectral Images

Hyperspectral spectral mixture analysis (SMA), which intends to decompose mixed pixels into a collection of endmembers weighted by their corresponding fraction abundances, has been successfully used to tackle mixed-pixel problem in hyperspectral remote sensing applications. As an approach of decomposing a high-dimensional data matrix into the multiplication of two nonnegative matrices, nonnegative matrix factorization (NMF) has shown its advantages and been widely applied to SMA. Unfortunately, most of the NMF-based unmixing methods can easily lead to an unsuitable solution, due to inadequate mining of spatial and spectral information and the influence of outliers and noise. To overcome such limitations, a spatial constraint over abundance and a spectral constraint over endmembers are imposed over NMF-based unmixing model for spectral-spatial constrained unmixing. Specifically, a spatial neighborhood preserving constraint is proposed to preserve the local geometric structure of the hyperspectral data by assuming that pixels in a spatial neighborhood generally fall into a low-dimensional manifold, while a minimum spectral distance constraint is formulated to optimize endmember spectra as compact as possible. Furthermore, to handle non-Gaussian noises or outliers, an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm based loss function is ultimately adopted for the proposed spectral-spatial constrained nonnegative matrix factorization model and a projected gradient based optimization algorithm is designed for optimization. Experimental results over both synthetic and real-world datasets demonstrate that the proposed spatial and spectral constraints can certainly improve the performance of NMF-based unmixing and outperform state-of-the-art NMF-based unmixing algorithms.

SAC-NMF-Driven Graphical Feature Analysis and Applications

Feature analysis is a fundamental research area in computer graphics; meanwhile, meaningful and part-aware feature bases are always demanding. This paper proposes a framework for conducting feature analysis on a three-dimensional (3D) model by introducing modified Non-negative Matrix Factorization (NMF) model into the graphical feature space and push forward further applications. By analyzing and utilizing the intrinsic ideas behind NMF, we propose conducting the factorization on feature matrices constructed based on descriptors or graphs, which provides a simple but effective way to raise compressed and scale-aware descriptors. In order to enable part-aware model analysis, we modify the NMF model to be sparse and constrained regarding to both bases and encodings, which gives rise to Sparse and Constrained Non-negative Matrix Factorization (SAC-NMF). Subsequently, by adapting the analytical components (including hidden variables, bases, and encodings) to design descriptors, several applications have been easily but effectively realized. The extensive experimental results demonstrate that the proposed framework has many attractive advantages, such as being efficient, extendable, and so forth.

Multi-Stage Network Embedding for Exploring Heterogeneous Edges

The relationships between objects in a network are typically diverse and complex, leading to the heterogeneous edges with different semantic information. In this article, we focus on exploring the heterogeneous edges for network representation learning. By considering each relationship as a view that depicts a specific type of proximity between nodes, we propose a multi-stage non-negative matrix factorization (MNMF) model, committed to utilizing abundant information in multiple views to learn robust network representations. In fact, most existing network embedding methods are closely related to implicitly factorizing the complex proximity matrix. However, the approximation error is usually quite large, since a single low-rank matrix is insufficient to capture the original information. Through a multi-stage matrix factorization process motivated by gradient boosting, our MNMF model achieves lower approximation error. Meanwhile, the multi-stage structure of MNMF gives the feasibility of designing two kinds of non-negative matrix factorization (NMF) manners to preserve network information better. The united NMF aims to preserve the consensus information between different views, and the independent NMF aims to preserve unique information of each view. Concrete experimental results on realistic datasets indicate that our model outperforms three types of baselines in practical applications.

CLUSTERING AND INDEXING OF MULTIPLE DOCUMENTS USING FEATURE EXTRACTION THROUGH APACHE HADOOP ON BIG DATA

Bigdata is a challenging field in data processing since the information is retrieved from various search engines through internet. A number of large organizations, that use document clustering,fails in arranging the documents sequentially in their machines. Across the globe, advanced technologyhas contributed to the high speed internet access. But the consequences of useful yet unorganized information in machine files seemto be confused in the retrieval process. Manual ordering of files has its own complications. In this paper, application software like Apache Lucene and Hadoop have taken a lead towards text mining for indexing and parallel implementation of document clustering. In organizations, it identifies the structure of the text data in computer files and its arrangement from files to folders, folders to subfolders, and to higher folders. A deeper analysis of document clustering was performed by considering various efficient algorithms like LSI, SVD and was compared with the newly proposed updated model of Non-Negative Matrix Factorization. The parallel implementation of hadoopdevelopedautomatic clusters for similar documents. MapReduce framework enforced its approach using K-means algorithm for all the incoming documents. The final clusters were automatically organized in folders using Apache Lucene in machines. This model was tested by considering the dataset of Newsgroup20 text documents. Thus this paper determines the implementation of large scale documents using parallel performance of MapReduce and Lucenethat generate automatic arrangement of documents, which reduces the computational time and improves the quick retrieval of documents in any scenario.

Evolving Hierarchical and Tag Information via the Deeply Enhanced Weighted Non-Negative Matrix Factorization of Rating Predictions

Identifying the hidden features of items and users of a modern recommendation system, wherein features are represented as hierarchical structures, allows us to understand the association between the two entities. Moreover, when tag information that is added to items by users themselves is coupled with hierarchically structured features, the rating prediction efficiency and system personalization are improved. To this effect, we developed a novel model that acquires hidden-level hierarchical features of users and items and combines them with the tag information of items that regularizes the matrix factorization process of a basic weighted non-negative matrix factorization (WNMF) model to complete our prediction model. The idea behind the proposed approach was to deeply factorize a basic WNMF model to obtain hidden hierarchical features of user’s preferences and item characteristics that reveal a deep relationship between them by regularizing the process with tag information as an auxiliary parameter. Experiments were conducted on the MovieLens 100K dataset, and the empirical results confirmed the potential of the proposed approach and its superiority over models that use the primary features of users and items or tag information separately in the prediction process.

Semisupervised Community Preserving Network Embedding with Pairwise Constraints

Network embedding aims to learn the low-dimensional representations of nodes in networks. It preserves the structure and internal attributes of the networks while representing nodes as low-dimensional dense real-valued vectors. These vectors are used as inputs of machine learning algorithms for network analysis tasks such as node clustering, classification, link prediction, and network visualization. The network embedding algorithms, which considered the community structure, impose a higher level of constraint on the similarity of nodes, and they make the learned node embedding results more discriminative. However, the existing network representation learning algorithms are mostly unsupervised models; the pairwise constraint information, which represents community membership, is not effectively utilized to obtain node embedding results that are more consistent with prior knowledge. This paper proposes a semisupervised modularized nonnegative matrix factorization model, SMNMF, while preserving the community structure for network embedding; the pairwise constraints (must-link and cannot-link) information are effectively fused with the adjacency matrix and node similarity matrix of the network so that the node representations learned by the model are more interpretable. Experimental results on eight real network datasets show that, comparing with the representative network embedding methods, the node representations learned after incorporating the pairwise constraints can obtain higher accuracy in node clustering task and the results of link prediction, and network visualization tasks indicate that the semisupervised model SMNMF is more discriminative than unsupervised ones.

Structural Deep Nonnegative Matrix Factorization for community detection

Due to the important role in analyzing the topological structure of complex networks, community detection has attracted increasing attention recently. The network embedding methods have shown promising performances in community detection, aiming to learn low-dimensional representations of nodes in networks. Among them, Nonnegative Matrix Factorization (NMF) is proved to be an efficient approach. However, considering that the mapping between the original network and the community membership space contains rather complex hierarchical information, classic shallow-NMF approaches often fail to capture the complex underlying network structure, resulting in sub-optimal network representations. Therefore, how to find a method that is able to effectively capture the complex underlying network structure and preserve the global and local structure is an open and important topic. Inspired by the excellent ability of representation learning in deep autoencoder, we propose a Structural Deep Nonnegative Matrix Factorization model, named SDNMF, for community detection. Similar to deep autoencoder, the proposed multi-layer NMF-based model consists of a decoder module and an encoder module. Further, we propose to exploit the first-order similarity and second-order similarity jointly to preserve the structural information. The first-order similarity characterizes the local network information. Meanwhile, the global network information can be captured and the sparsity problem is alleviated by the second-order similarity. An efficient learning algorithm is developed to optimize the proposed DANMF model, which simultaneously optimizes the first-order and second-order similarity. The effectiveness of the proposed SDNMF is verified by comparing it with several state-of-the-art approaches for community detection on six real-word networks. The comparison is based on three performance metrics. Experimental results demonstrate that the proposed SDNMF can obtain a more accurate community membership matrix compared with the baselines. The convergence analysis is also performed to verify the efficiency of our approach.

A robust semi-supervised NMF model for single cell RNA-seq data.

BackgroundSingle-cell RNA-sequencing (scRNA-seq) technology is a powerful tool to study organism from a single cell perspective and explore the heterogeneity between cells. Clustering is a fundamental step in scRNA-seq data analysis and it is the key to understand cell function and constitutes the basis of other advanced analysis. Nonnegative Matrix Factorization (NMF) has been widely used in clustering analysis of transcriptome data and achieved good performance. However, the existing NMF model is unsupervised and ignores known gene functions in the process of clustering. Knowledges of cell markers genes (genes that only express in specific cells) in human and model organisms have been accumulated a lot, such as the Molecular Signatures Database (MSigDB), which can be used as prior information in the clustering analysis of scRNA-seq data. Because the same kind of cells is likely to have similar biological functions and specific gene expression patterns, the marker genes of cells can be utilized as prior knowledge in the clustering analysis.MethodsWe propose a robust and semi-supervised NMF (rssNMF) model, which introduces a new variable to absorb noises of data and incorporates marker genes as prior information into a graph regularization term. We use rssNMF to solve the clustering problem of scRNA-seq data.ResultsTwelve scRNA-seq datasets with true labels are used to test the model performance and the results illustrate that our model outperforms original NMF and other common methods such as KMeans and Hierarchical Clustering. Biological significance analysis shows that rssNMF can identify key subclasses and latent biological processes. To our knowledge, this study is the first method that incorporates prior knowledge into the clustering analysis of scRNA-seq data.

Statistical Analysis of Microarray Data Clustering using NMF, Spectral Clustering, Kmeans, and GMM.

In unsupervised learning literature, the study of clustering using microarray gene expression datasets has been extensively conducted with nonnegative matrix factorization (NMF), spectral clustering, kmeans, and gaussian mixture model (GMM)are some of the most used methods. However, there is still a limited number of works that utilize statistical analysis to measure the significances of performance differences between these methods. In this paper, statistical analysis of performance differences between ten NMF, six spectral clustering, four GMM, and the standard kmeans algorithms in clustering eleven publicly available microarray gene expression datasets with the number of clusters ranges from two to ten is presented. The experimental results show that statistically NMFs and kmeans have similar performances and outperform spectral clustering. As spectral clustering can be used to uncover hidden manifold structures, the underperformance of spectral methods leads us to question whether the datasets have manifold structures. Visual inspection using multidimensional scaling plots indicates that such structures do not exist. Moreover, as the plots indicate that clusters in some datasets have elliptical boundaries, GMM methods are also utilized. The experimental results show that GMM methods outperform the other methods to some degree, and thus imply that the datasets follow gaussian distributions.

Bayesian mean-parameterized nonnegative binary matrix factorization

Binary data matrices can represent many types of data such as social networks, votes, or gene expression. In some cases, the analysis of binary matrices can be tackled with nonnegative matrix factorization (NMF), where the observed data matrix is approximated by the product of two smaller nonnegative matrices. In this context, probabilistic NMF assumes a generative model where the data is usually Bernoulli-distributed. Often, a link function is used to map the factorization to the [0, 1] range, ensuring a valid Bernoulli mean parameter. However, link functions have the potential disadvantage to lead to uninterpretable models. Mean-parameterized NMF, on the contrary, overcomes this problem. We propose a unified framework for Bayesian mean-parameterized nonnegative binary matrix factorization models (NBMF). We analyze three models which correspond to three possible constraints that respect the mean-parameterization without the need for link functions. Furthermore, we derive a novel collapsed Gibbs sampler and a collapsed variational algorithm to infer the posterior distribution of the factors. Next, we extend the proposed models to a nonparametric setting where the number of used latent dimensions is automatically driven by the observed data. We analyze the performance of our NBMF methods in multiple datasets for different tasks such as dictionary learning and prediction of missing data. Experiments show that our methods provide similar or superior results than the state of the art, while automatically detecting the number of relevant components.

Topic Modeling Coherence: A Comparative Study between LDA and NMF Models using COVID’19 Corpus

Topic modeling is a method for finding abstract topics in a large collection of documents With it, it is possible to discover the mixture of hidden or “latent” topics that varies from document to document in a given corpus As an unsupervised machine learning approach, topic models are not easy to evaluate since there is no labelled “ground truth” data to compare with However, since topic modeling typically requires defining some parameters beforehand (first and foremost the number of topics k to be discovered), model evaluation is crucial in order to find an “optimal” set of parameters for the given data Latent Dirichlet allocation (LDA) and Non-Negative Matrix Factorization (NMF) are the two most popular topic modeling techniques LDA uses a probabilistic approach where as NMF uses matrix factorization approach In this paper we want to assess which most relevant technique for topic coherence using c_v measure, we have chosen citations’s Covid’19 Corpus for experimentations © 2020, World Academy of Research in Science and Engineering All rights reserved

A supervised non-negative matrix factorization model for speech emotion recognition

Feature representation plays a critical role in speech emotion recognition (SER). As a method of data dimensionality reduction, Non-negative Matrix Factorization (NMF) can obtain the low-dimensional representation of data by matrix decomposition, and make the data more distinguishable. In order to improve the recognition ability of NMF for SER, we conduct a potential study on NMF and propose a supervised NMF model, called joint discrimination ability and similarity constraint of NMF (DSNMF). This model incorporates the discriminative information and similarity information of samples into basic NMF as prior knowledge, so that the original data can be decomposed into more distinguished low-dimensional data. Specifically, on the one hand, the labels of the training set are used to improve the discriminative ability of the model; on the other hand, with the similarity of the training samples, the data of similar samples are more highly aggregated in the low-dimensional space. In addition, the convergence of DSNMF is proved theoretically and experimentally. Extensive experiments on EMODB and IEMOCAP corpora show that the proposed approach has a better classification effect on low-dimensional representation data than other NMF models.

Leveraging maximum entropy and correlation on latent factors for learning representations

Many tasks involve learning representations from matrices, and Non-negative Matrix Factorization (NMF) has been widely used due to its excellent interpretability. Through factorization, sample vectors are reconstructed as additive combinations of latent factors, which are represented as non-negative distributions over the raw input features. NMF models are significantly affected by latent factors’ distribution characteristics and the correlations among them. And NMF models are faced with the challenge of learning robust latent factor. To this end, we propose to learn representations with an awareness of the semantic quality evaluated from the aspects of intra- and inter-factors. On the one hand, a Maximum Entropy-based function is devised for the intra-factor semantic quality. On the other hand, the semantic uniqueness is evaluated via inter-factor correlation, which reinforces the aim of semantic compactness. Moreover, we present a novel non-linear NMF framework. The learning algorithm is presented and the convergence is theoretically analyzed and proved. Extensive experimental results on multiple datasets demonstrate that our method can be successfully applied to representative NMF models and boost performances over state-of-the-art models.

Topic modeling in short-text using non-negative matrix factorization based on deep reinforcement learning

Topic modeling for short texts is a challenging and interesting problem in the machine learning and knowledge discovery domains. Nowadays, millions of documents published on the internet from various sources. Internet websites are full of various topics and information, but there is a lot of simila rity between topics, contents, and total quality of sources, which causes data repetition and gives the user the same information. Another issue is data sparsity and ambiguity because the length of the short text is limited, which causes unsatisfactory results and give irrelevant results to end-users. All these mentioned issues in short texts made an interesting topic for researchers to use machine learning and knowledge discovery techniques to discover underlying topics from a massive amount of data. In this paper, we propose a combination of deep reinforcement learning (RL) and semantics-assisted non-negative matrix factorization model to extract meaningful and underlying topics from short document contents. The main objective of this work is to reduce the problem of repetitive information and data sparsity in short texts to help the users to get meaningful and relevant contents. Furthermore, our propose model reviews an issue of the Seq2Seq approach based on the reinforcement learning perspective and provides a combination of reinforcement learning and SeaNMF formulation using the block coordinate descent algorithm. Moreover, we compare different real-world datasets by using numerical calculation and present a couple of state-of-art models to get better performance on short text document topic modeling. Based on experimental results and comparative analysis, our propose model outperforms the state of art techniques in terms of short document topic modeling.

Self-Paced Nonnegative Matrix Factorization for Hyperspectral Unmixing

The presence of mixed pixels in the hyperspectral data makes unmixing to be a key step for many applications. Unsupervised unmixing needs to estimate the number of endmembers, their spectral signatures, and their abundances at each pixel. Since both endmember and abundance matrices are unknown, unsupervised unmixing can be considered as a blind source separation problem and can be solved by nonnegative matrix factorization (NMF). However, most of the existing NMF unmixing methods use a least-squares objective function that is sensitive to the noise and outliers. To deal with different types of noises in hyperspectral data, such as the noise in different bands (band noise), the noise in different pixels (pixel noise), and the noise in different elements of hyperspectral data matrix (element noise), we propose three self-paced learning based NMF (SpNMF) unmixing models in this article. The SpNMF models replace the least-squares loss in the standard NMF model with weighted least-squares losses and adopt a self-paced learning (SPL) strategy to learn the weights adaptively. In each iteration of SPL, atoms (bands or pixels or elements) with weight zero are considered as complex atoms and are excluded, while atoms with nonzero weights are considered as easy atoms and are included in the current unmixing model. By gradually enlarging the size of the current model set, SpNMF can select atoms from easy to complex. Usually, noisy or outlying atoms are complex atoms that are excluded from the unmixing model. Thus, SpNMF models are robust to noise and outliers. Experimental results on the simulated and two real hyperspectral data sets demonstrate that our proposed SpNMF methods are more accurate and robust than the existing NMF methods, especially in the case of heavy noise.

Answer Keyword Generation for Community Question Answering by Multiaspect Gamma–Poisson Matrix Completion

Community question answering (CQA) recommends appropriate answers to existing and new questions. Such answer recommendation is challenging since CQA data are often sparse and decentralized, and lacks sufficient information to generate suitable answers to existing questions. Matching answers to new questions is more challenging in modeling Q/A sparsity, generating answers to cold-start/novel questions, and integrating metadata about Q/A into models, etc. This article addresses these issues by a novel statistical model to automatically generate answer keywords in CQA with multiaspect Gamma-Poisson matrix completion (MAGIC). MAGIC is the first trial in CQA to model multiple aspects of Q/A sentence information in CQA by involving Q/A metadata, Q/A sparsity, and both lexical and semantic Q/A information in a hierarchical Gamma-Poisson model. MAGIC can efficiently generate answer keywords for both existing and new questions against nonnegative matrix factorization (MF), probability MF, and relevant Poisson factorization models w.r.t. recommending appropriate and informative answer keywords.

Robust non-negative matrix factorization with multiple correntropy-induced hypergraph regularizer

Non-negative matrix factorization (NMF) is a popular learning tool, which has widely used in computer vision and image processing. Many variants and extensions of NMF have been proposed, where the manifold regularized NMF methods have achieved promising performance due to the preservation of the geometric structures of the data. However, for many applications, the data is usually contaminated by complex noise. The noise leads the data to deviate from the intrinsic manifold, resulting in the degenerate performance. To make the NMF methods reflect the underlying manifold structure well, we propose a novel robust non-negative matrix factorization model. Our model proposes a novel ensemble manifold regularizer, which combines multiple robust hypergraphs to estimate the underlying manifold. Specifically, the correntropy-induced hypergraph is used as the initial manifold estimation. By incorporating the proposed regularizer into the original NMF framework, two novel manifold regularized NMF methods are proposed. The clustering results on the noisy image datasets demonstrate that our model is effective, which achieves the state-of-the-art performance.

Robust Manhattan non-negative matrix factorization for image recovery and representation

Existing robust non-negative matrix factorization methods fail to achieve data recovery and learn a robust representation. This is because these methods suppose that outliers and noise of the original data are the Gaussian distribution. In this paper, we propose a robust non-negative matrix model, called robust Manhattan non-negative matrix factorization, which can handle various noise (e.g. Gaussian noise, Salt and Pepper noise or Contiguous Occlusion). Different from previous robust non-negative matrix factorization models, we utilize mean filter and matrix completion as additional constraints to recover the corrupted data from normal data or neighbouring corrupted data, and achieve a robust low-dimensional representation by Manhattan non-negative matrix factorization. We theoretically compare the robustness of our proposed model with other non-negative matrix factorization models and theoretically prove the effectiveness of the proposed algorithm. Extensive experimental results on the image dataset containing noise and outliers validate the robustness and effectiveness of our proposed model for image recovery and representation.

Generalized Morphological Component Analysis for Hyperspectral Unmixing

Hyperspectral unmixing (HU) is an active research topic in the remote-sensing community. It aims at modeling mixed pixels using a collection of pure constituent materials ( endmembers ) weighted by their corresponding fractional abundances. Among existing unmixing schemes, nonnegative matrix factorization (NMF) has drawn significant attention due to its unsupervised nature, as well as its capacity to obtain both endmembers and fractional abundances simultaneously. In this article, we present a new blind unmixing method based on the generalized morphological component analysis (GMCA) framework, in which an additional constraint is introduced into the standard NMF model to represent the sparsity and morphological diversity of the abundance maps associated with each endmember. More specifically, we take into account the fact that different ground categories in a hyperspectral scene generally exhibit various spatial distributions and morphological characteristics. As a result, when providing a specific dictionary basis for these categories, their corresponding abundance maps (referred to as sources ) can be sparsely represented. In addition, due to the low correlation between different sources, their sparse representations will not share the same most significant coefficients. With this observation in mind, we can further promote source discrimination and separation in the unmixing process. Moreover, in order to obtain a stable solution of the involved optimization problem, we adopt an alternate iterative constrained algorithm with a threshold descent strategy. Our experiments, carried out on both synthetic and real hyperspectral scenes, reveal that our newly developed GMCA-based unmixing method obtains very promising results with fast convergence speed and requiring significantly less parameter tuning. This confirms the advantage of the proposed spatial morphological component approach for HU purposes.