Regularized Non-negative Matrix Factorization Research Articles

Learning a Subspace and Clustering Simultaneously with Manifold Regularized Nonnegative Matrix Factorization

With the incredible growth of high-dimensional data such as microarray gene expression data and web blogs from internet, the researchers are desirable to develop new clustering techniques to address the critical problem created by irrelevant dimensions. Properties of Nonnegative Matrix Factorization (NMF) as a clustering method were studied by relating its formulation to other methods such as K-means clustering. In this paper, by introducing clustering indicator constraints on NMF and incorporating manifold regularization to preserve geometric structures, we propose a novel manifold regularized NMF method that can simultaneously learn subspace and do clustering. As a result, our clustering results can directly assign cluster label to data points. Extensive experimental results show that our method outperforms related other methods.

Inter- and intra-hypergraph regularized nonnegative matrix factorization with hybrid constraints

The accurate low rank representation of high-dimensional data learned by the manifold regularized nonnegative matrix factorization framework is effective in data clustering. In previous work, this work has mainly been solved in the way of similarity matrix induction. To further increase the efficacy of low-rank representations, we propose a novel semi-supervised non-negative matrix factorization (NMF) model in this study called inter- and intra-hypergraph regularized non-negative matrix factorization with hybrid constraints (IGNMFC). IGNMFC constructs intra-hypergraph regularization and intra-hypergraph regularization by hypergraph learning, which can precisely induce high-dimensional data to map toward low-dimensional. Moreover, hybrid constraints are introduced to improve the exclusivity and sparsity of low-dimensional representations, and the result accounts for the benefit of this method in learning distinguishable subspace representations. Finally, IGNMFC is transformed into an optimal problem and an efficient iteration rule is proposed. Experiments on several datasets demonstrate that the proposed method outperforms the state-of-the-art NMF algorithms, and can achieve at least 7.9%∼15% accuracy improvement in most cases.

Dual regularized subspace learning using adaptive graph learning and rank constraint: Unsupervised feature selection on gene expression microarray datasets

High-dimensional problems have increasingly drawn attention in gene selection and analysis. To add insult to injury, usually the number of features is greater than number of samples in microarray gene dataset which leads to an ill-posed underdetermined equation system. Poor performance and high computational time for learning algorithms are consequences of redundant features in high-dimensional data. Feature selection is a noteworthy pre-processing method to ameliorate the curse of dimensionality with aim of maximum relevancy and minimum redundancy information preservation. Likewise, unsupervised feature selection has been important since collecting labels for data is expensive. In this paper, we develop a novel robust unsupervised feature selection to select discriminative subset of features for unlabeled data based on rank constrained and dual regularized nonnegative matrix factorization. The major focus of the proposed technique is to discard redundant features while keeping the informative features. Proposed feature selection technique consists of nonnegative matrix factorization to decompose the data into feature weight matrix and representation matrix, inner product norm as regularization for both feature weight matrix and representation matrix, adaptive structure learning to preserve local information and Schatten-p norm as rank constraint. To demonstrate the effectiveness of the proposed method, numerical studies are conducted on six benchmark microarray datasets. The results show that the proposed technique outperforms eight state-of-art unsupervised feature selection techniques in terms of clustering accuracy and normalized mutual information.

Subspace learning using structure learning and non-convex regularization: Hybrid technique with mushroom reproduction optimization in gene selection

Gene selection as a problem with high dimensions has drawn considerable attention in machine learning and computational biology over the past decade. In the field of gene selection in cancer datasets, different types of feature selection techniques in terms of strategy (filter, wrapper and embedded) and label information (supervised, unsupervised, and semi-supervised) have been developed. However, using hybrid feature selection can still improve the performance. In this paper, we propose a hybrid feature selection based on filter and wrapper strategies. In the filter-phase, we develop an unsupervised features selection based on non-convex regularized non-negative matrix factorization and structure learning, which we deem NCNMFSL. In the wrapper-phase, for the first time, mushroom reproduction optimization (MRO) is leveraged to obtain the most informative features subset. In this hybrid feature selection method, irrelevant features are filtered-out through NCNMFSL, and most discriminative features are selected by MRO. To show the effectiveness and proficiency of the proposed method, numerical experiments are conducted on Breast, Heart, Colon, Leukemia, Prostate, Tox-171 and GLI-85 benchmark datasets. SVM and decision tree classifiers are leveraged to analyze proposed technique and top accuracy are 0.97, 0.84, 0.98, 0.95, 0.98, 0.87 and 0.85 for Breast, Heart, Colon, Leukemia, Prostate, Tox-171 and GLI-85, respectively. The computational results show the effectiveness of the proposed method in comparison with state-of-art feature selection techniques.

Multi-view clustering via label-embedded regularized NMF with dual-graph constraints

Nonnegative matrix factorization (NMF) methods have achieved remarkable performances in multi-view clustering due to their effectiveness and efficiency. To better obtain a low-dimensional common representation, the limited labels and the geometric structure of the multi-view data should be fully utilized in clustering. In this work, we introduce a novel multi-view learning approach, dubbed label-embedded regularized NMF with dual-graph constraints (LeNMF-DC), for clustering. Our proposed LeNMF-DC approach mainly utilizes matrix factorization to obtain a low-dimensional common representation of the multi-view data, in which the prior knowledge hidden in data can be fully explored. Specifically, we construct three graph regularization terms to preserve the manifold structure in the data, feature and label space, respectively. Moreover, we take advantage of the labels of the labeled samples without additional parameters. In addition, we develop an alternate iterative optimization scheme to solve the model of LeNMF-DC and then show its convergence rate. Compared with traditional multi-view clustering approaches, the labels of unlabeled samples in our proposed LeNMF-DC approach are assigned by the label constraint matrix rather than the clustering algorithm, and thus it avoids performance loss during the clustering. Experimental results on four benchmark datasets manifest that our LeNMF-DC approach can achieve superior performances than several state-of-the-art approaches in multi-view clustering.

RRNMF-MAGL: Robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning for dimensionality reduction

In this paper, a new unsupervised dimensionality reduction method named robust regularization non-negative matrix factorization with multi-constraint adaptive graph learning (RRNMF-MAGL) is developed. Compared with the existing methods, RRNMF-MAGL can extract robust low-dimensional features and adaptively learn a manifold structure to well reflect the data distribution. Specifically, to alleviate the influence of outliers and noises, a robust non-negative matrix factorization (RNMF) approach with L21-norm loss function is first constructed. Then, a multi-constraint adaptive graph learning (MAGL) model is designed based on low-dimensional features to reduce the computational complexity and avoid the adverse influence of redundant information. Moreover, multiple constraints (i.e., sparsity and locality) are incorporated into graph learning for enhancing the discrimination ability of graph structure. Next, to make the learned low-dimensional features well maintain the local geometric structure information of data, a graph Laplacian regularization (GLR) is designed. Finally, an iterative update strategy is proposed to optimize RRNMF-MAGL and its convergence is verified by numerical experiments. The superior effectiveness and robustness of RRNMF-MAGL is demonstrated by extensive experiments on eight publicly available datasets.

Robust Exponential Graph Regularization Non-Negative Matrix Factorization Technology for Feature Extraction

Graph regularized non-negative matrix factorization (GNMF) is widely used in feature extraction. In the process of dimensionality reduction, GNMF can retain the internal manifold structure of data by adding a regularizer to non-negative matrix factorization (NMF). Because Ga NMF regularizer is implemented by local preserving projections (LPP), there are small sample size problems (SSS). In view of the above problems, a new algorithm named robust exponential graph regularized non-negative matrix factorization (REGNMF) is proposed in this paper. By adding a matrix exponent to the regularizer of GNMF, the possible existing singular matrix will change into a non-singular matrix. This model successfully solves the problems in the above algorithm. For the optimization problem of the REGNMF algorithm, we use a multiplicative non-negative updating rule to iteratively solve the REGNMF method. Finally, this method is applied to AR, COIL database, Yale noise set, and AR occlusion dataset for performance test, and the experimental results are compared with some existing methods. The results indicate that the proposed method is more significant.

Graph Regularized Nonnegative Matrix Factorization for Community Detection in Attributed Networks

Community detection has become an important research topic in machine learning due to the proliferation of network data. However, most existing methods have been developed based on only exploiting the topology structures of the network, which can result in missing the advantage of using the nodes' attribute information. As a result, it is expected that much valuable information that could be used to improve the quality of discovered communities will be ignored. To solve this limitation, we propose a novel Augment Graph Regularization Nonnegative Matrix Factorization for Attributed Networks (AGNMF-AN) method, which is simple yet effective. Firstly, Augment Attributed Graph (AAG) is applied to combine both the topological structure and attributed nodes of the network. Secondly, we introduced an effective framework to update the affinity matrix, in which the affinity matrix's weight in each iteration is modified adaptively instead of using a fixed affinity matrix in the classical graph regularization-based nonnegative matrix factorization methods. Thirdly, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2,1}$</tex-math></inline-formula> -norm is utilized to reduce the effect of random noise and outliers in the quality of structure community. Experimental results show that our method performs unexpectedly well in comparison to existing state-of-the-art methods in attributed networks.

Consensus and complementary regularized non-negative matrix factorization for multi-view image clustering

Multi-view clustering (MVC) aims to classify objects using multi-view data. Many existing MVC methods explore the consensus agreement among views or enhance the complementary features from multiple views in order to improve the clustering quality. However, multi-view data, especially multi-view image data, contains much more complex and meaningful information, which not only describes intrinsic common properties of one object, but also implies rich unique features from different views. Both of them are important and useful for MVC. Though the widely-used non-negative matrix factorization (NMF) based MVC methods can address these two kinds of information, the qualities of learned representations are not well, which limits the clustering performance. To solve this problem, motivated by the ideas of feature selection and graph regularization, a novel NMF based unified representation learning framework is presented with the integration of two specifically designed graph regularization terms in order to obtain a high-quality representation for multi-view data: An unified NMF based optimization problem is formulated to learn the consensus and complementary representations simultaneously and an alternating optimization algorithm is designed to solve this non-convex optimization problem. Then, these learned representations are efficiently fused to create an integrated graph representation for the final clustering. Extensive experiments on six real image datasets demonstrate the superior performance of the proposed method compared with other eleven competitive MVC methods.

Semi-supervised dual-graph regularization non-negative matrix factorization with local coordinate and orthogonal constraints for image clustering

Due to significant performance in the representation of data points, non-negative matrix factorization (NMF) has been widely applied in machine-learning fields, such as dimension reduction, image representation, feature extraction, data mining, and so on. However, classical NMF suffers from a common issue, low efficiency in representing the internal geometric structure of data and sparsity limitation. To circumvent this problem, we innovatively propose a semi-supervised NMF algorithm called semi-supervised dual-graph regularization non-negative matrix factorization (LOSDNMF), into which dual-graph and bi-orthogonal constraints are embedded to reduce the inconsistency between the original matrix and the basic vectors while maintaining the manifold structures of the data and feature spaces. This strategy can fully explore the potential geometry information of the data, which is extremely beneficial to enhance the learning ability of the model. In addition, the local coordinate constraints are introduced to ensure good sparsity of the coefficient matrix and simplify the calculation. Furthermore, an iterative updating scheme for the optimization problem of LOSDNMF and its convergence proofs are also provided in detail. The effectiveness of the proposed method is verified on eight benchmark datasets. Experimental results show that our method can effectively improve clustering performance.

Robust graph regularization nonnegative matrix factorization for link prediction in attributed networks

Robust graph regularization nonnegative matrix factorization for link prediction in attributed networks

M6Adecom: Analysis of m6A profile matrix based on graph regularized non-negative matrix factorization.

Epitranscriptomic m6A methylation is shown to mediate extensive regulations under the context of various RNA binding protein (RBP) readers. With m6A methylation data has reached a sizable scale, the functional context-aware analysis of m6A profiles is becoming more feasible and demanded. In this study, we employed graph regularized non-negative matrix factorization (GNMF) for m6A profile analysis and comparison, where the RBP binding preference of m6A sites were incorporated as the functional context-based graph constraint term. Compared to the baseline non-negative matrix factorization (NMF) method, this GNMF-based method could better capture the distinctions in multiple functional characteristics between different group of m6A sites, including but not limited to the associated biological pathways and disease genes. We further established m6Adecom, an online tool that can be used for correlation and enrichment analysis of m6A profiles using the matrix decomposition result from GNMF, and gene set enrichment analysis based on the high-score m6A sites. m6Adecom is freely accessible at http://www.rnanut.net/m6adecom.

Shared and Cell-Type-Specific Gene Expression Patterns Associated With Autism Revealed by Integrative Regularized Non-Negative Matrix Factorization.

Human brain-related disorders, such as autism spectrum disorder (ASD), are often characterized by cell heterogeneity, as the cell atlas of brains consists of diverse cell types. There are commonality and specificity in gene expression among different cell types of brains; hence, there may also be commonality and specificity in dysregulated gene expression affected by ASD among brain cells. Moreover, as genes interact together, it is important to identify shared and cell-type-specific ASD-related gene modules for studying the cell heterogeneity of ASD. To this end, we propose integrative regularized non-negative matrix factorization (iRNMF) by imposing a new regularization based on integrative non-negative matrix factorization. Using iRNMF, we analyze gene expression data of multiple cell types of the human brain to obtain shared and cell-type-specific gene modules. Based on ASD risk genes, we identify shared and cell-type-specific ASD-associated gene modules. By analyzing these gene modules, we study the commonality and specificity among different cell types in dysregulated gene expression affected by ASD. The shared ASD-associated gene modules are mostly relevant to the functioning of synapses, while in different cell types, different kinds of gene functions may be specifically dysregulated in ASD, such as inhibitory extracellular ligand-gated ion channel activity in GABAergic interneurons and excitatory postsynaptic potential and ionotropic glutamate receptor signaling pathway in glutamatergic neurons. Our results provide new insights into the molecular mechanism and pathogenesis of ASD. The identification of shared and cell-type-specific ASD-related gene modules can facilitate the development of more targeted biomarkers and treatments for ASD.

NNDSVD-GRMF: A Graph Dual Regularization Matrix Factorization Method Using Non-Negative Initialization for Predicting Drug-Target Interactions

Accurate drug-target interactions (DTIs) prediction can significantly speed up the process of new drug design and development. Recently, many matrix factorization methods have been used to predict DTIs. However, most of them use heuristic and iterative strategies, and their convergence and performance can not be guaranteed. In order to accurately predict DTIs, we propose a new algorithm, NNDSVD-GRMF, our method is based on graph dual regularization non-negative matrix factorization (GDNMF) and non-negative double singular value decomposition (NNDSVD), which considers both the initialization stage of the non-negative matrix factorization and the structural information of the data and features. At the same time, in order to improve the adaptability of the algorithm, the extension of the NNDSVD-GRMF (NNDSVD-WGRMF) is also proposed. Extensive experimental results show that our methods have better performance than other state-of-the-art methods. In the case studies, among the 10 highest-scoring drugs predicted to interact with androgen receptor, 9 drugs have been validated, and among the 10 highest-scoring target proteins predicted to be targeted by the drug nicotine bitartrate, 9 targets have been validated.

Community Detection in Partially Observable Social Networks

The discovery of community structures in social networks has gained significant attention since it is a fundamental problem in understanding the networks’ topology and functions. However, most social network data are collected from partially observable networks with both missing nodes and edges . In this article, we address a new problem of detecting overlapping community structures in the context of such an incomplete network, where communities in the network are allowed to overlap since nodes belong to multiple communities at once. To solve this problem, we introduce KroMFac , a new framework that conducts community detection via regularized nonnegative matrix factorization (NMF) based on the Kronecker graph model. Specifically, from an inferred Kronecker generative parameter matrix, we first estimate the missing part of the network. As our major contribution to the proposed framework, to improve community detection accuracy, we then characterize and select influential nodes (which tend to have high degrees) by ranking, and add them to the existing graph. Finally, we uncover the community structures by solving the regularized NMF-aided optimization problem in terms of maximizing the likelihood of the underlying graph. Furthermore, adopting normalized mutual information (NMI), we empirically show superiority of our KroMFac approach over two baseline schemes by using both synthetic and real-world networks.

Affinity Regularized Non-Negative Matrix Factorization for Lifelong Topic Modeling

Lifelong topic model (LTM), an emerging paradigm for never-ending topic learning, aims to yield higher-quality topics as time passes through knowledge accumulated from the past yet learned for the future. In this paper, we propose a novel lifelong topic model based on non-negative matrix factorization (NMF), called Affinity Regularized NMF for LTM (NMF-LTM), which to our best knowledge is distinctive from the popular LDA-based LTMs. NMF-LTM achieves lifelong learning by introducing word-word graph Laplacian as semantic affinity regularization. Other priors such as sparsity, diversity, and between-class affinity are incorporated as well for better performance, and a theoretical guarantee is provided for the algorithmic convergence to a local minimum. Extensive experiments on various public corpora demonstrate the effectiveness of NMF-LTM, particularly its human-like behaviors in two carefully designed learning tasks and the ability in topic modeling of big data. A further exploration of semantic relatedness in knowledge graphs and a case study on a large-scale real-world corpus exhibit the strength of NMF-LTM in discovering high-quality topics in an efficient and robust way.

Rank-constrained nonnegative matrix factorization for data representation

Graph-based regularized nonnegative matrix factorization (NMF) methods performed well in many real-world applications. However, it is still an open problem to construct an optimal graph to effectively discover the intrinsic geometric structure of data. In this paper, we propose a new data representation framework, called rank-constrained nonnegative matrix factorization (RCNMF). We impose the rank constraint on the Laplacian matrix of the learned graph, so it can ensure that the number of connected components is consistent with the number of sample categories. Instead of a fixed graph-based regularization, the proposed framework can adaptively adjust the weight of the affinity matrix in each iteration. We develop two versions of RCNMF based on the l1 and l2 norms, and introduce their optimization schemes. In addition, their convergence and the complexity analyses are also provided. Experimental results on four benchmark datasets show that our methods outperform state-of-the-art methods in clustering.

Completion of multiview missing data based on multi-manifold regularised non-negative matrix factorisation

In multi-source data analysis, the absence of data values or attributes is inevitably brought about by various influencing factors including environment, which results in the loss of knowledge to be conveyed by data. To solve the problem of missing data in multi-source data analysis, completion method for multiview missing data based on multi-manifold regularized non-negative matrix factorization was proposed in this paper. This method was based on the assumption of consistency of the multiview data and an algorithm of multi-manifold regularized non-negative matrix factorization is adopted to obtain homogeneous manifold and global clustering. On this basis, a multiview synergistic discrimination model is built of the non-missing view that referred to the Gaussian mixture model to pre-mark the clustering that the incremental missing data belonged to. Using the consistency of each view in the low-dimensional space, a prediction model of missing data at the specified view is established using the multiple linear regression technique to achieve accurate data completion under conditions of missing multi-attributes. Through the establishment of data filling model with three handling methods for missing values, namely CMMD-MNMF, FIMUS and Hot deck, the completion performance, clustering performance and classification performance of data sets including UCI, Flower17 and Flower102 are analyzed by simulation experiments. As shown in the results, the method of multi-view data missing completion is verified to be effective.

PTL-LTM model for complex action recognition using local-weighted NMF and deep dual-manifold regularized NMF with sparsity constraint

Complex action recognition possesses significant academic research value, potential commercial value and broad market application prospect. For improving its performance, a local-weighted nonnegative matrix factorization with rank regularization constraint (LWNMF_RC) is firstly presented, which removes complex background and then obtains motion salient regions. Secondly, a dual-manifold regularized nonnegative matrix factorization with sparsity constraint (DMNMF_SC) is proposed, which not only considers the short-term and middle-term temporal dependencies implied in data manifold, but also mines the geometric structure hidden in feature manifold. In addition, the introduction of sparsity constraint makes features possess better discriminativeness. Thirdly, a deep DMNMF_SC method is constructed, which acquires more hierarchical and discriminative features. Finally, a long-term temporal memory model with probability transfer learning (PTL-LTM) is proposed, which accurately memorizes the long-term temporal dependency among multiple simple action segments and, meanwhile, makes full use of the probability features of rich labeled simple actions and then applies the knowledge learned from simple actions for complex action recognition. Consequently, the performance is effectively improved.

Sparse Graph Regularization Non-Negative Matrix Factorization Based on Huber Loss Model for Cancer Data Analysis.

Non-negative matrix factorization (NMF) is a matrix decomposition method based on the square loss function. To exploit cancer information, cancer gene expression data often uses the NMF method to reduce dimensionality. Gene expression data usually have some noise and outliers, while the original NMF loss function is very sensitive to non-Gaussian noise. To improve the robustness and clustering performance of the algorithm, we propose a sparse graph regularization NMF based on Huber loss model for cancer data analysis (Huber-SGNMF). Huber loss is a function between L 1-norm and L 2-norm that can effectively handle non-Gaussian noise and outliers. Taking into account the sparsity matrix and data geometry information, sparse penalty and graph regularization terms are introduced into the model to enhance matrix sparsity and capture data manifold structure. Before the experiment, we first analyzed the robustness of Huber-SGNMF and other models. Experiments on The Cancer Genome Atlas (TCGA) data have shown that Huber-SGNMF performs better than other most advanced methods in sample clustering and differentially expressed gene selection.