Nonnegative Matrix Tri-factorization Research Articles

Semi-supervised non-negative matrix tri-factorization with adaptive neighbors and block-diagonal learning

Graph-regularized non-negative matrix factorization (GNMF) is proved to be effective for the clustering of nonlinear separable data. Existing GNMF variants commonly improve model performance by adding different additional constraints or refining the model factorization form, which can lead to problems such as increased algorithm complexity or insufficient performance release. In this paper, we propose semi-supervised non-negative matrix tri-factorization with adaptive neighbors and block-diagonal (ABNMTF). Different from existing methods, in ABNMTF the similarity graph matrix is learned from the original data by adaptive neighbors k-nearest model, and a block diagonal matrix is constructed based on a few labeled data to update the similarity matrix. Our approach reconstructs the block diagonal structure into the adaptive similarity matrix, which enables simultaneous learning of the similarity matrix and label binding during factorization, engendering a distinguishable subspace representation matrix and therefore improving the clustering performance without significantly increasing the complexity of the algorithm. We also represent an optimization method to solve the ABNMTF and provide analyses of convergence and computational complexity. Extensive experiments on 8 real image datasets show that the proposed algorithm reports superior performance against several state-of-the-art approaches. Code has been made available at:https://github.com/LstinWh/ABNMTF.

Symmetric nonnegative matrix trifactorization

The Symmetric Nonnegative Matrix Trifactorization (SN-Trifactorization) is a factorization of an n×n nonnegative symmetric matrix A of the form BCBT, where C is a k×k symmetric matrix, and both B and C are required to be nonnegative. This work introduces the SNT-rank of A, as the minimal k, for which such factorization exists. After a list of basic properties and an exploration of SNT-rank of low rank matrices, the class of nonnegative symmetric matrices with SNT-rank equal to rank is studied. The paper concludes with a completion problem, that asks for matrices with the smallest possible SNT-rank among all nonnegative symmetric matrices with given diagonal blocks.

Robust capped norm dual hyper-graph regularized non-negative matrix tri-factorization.

Non-negative matrix factorization (NMF) has been widely used in machine learning and data mining fields. As an extension of NMF, non-negative matrix tri-factorization (NMTF) provides more degrees of freedom than NMF. However, standard NMTF algorithm utilizes Frobenius norm to calculate residual error, which can be dramatically affected by noise and outliers. Moreover, the hidden geometric information in feature manifold and sample manifold is rarely learned. Hence, a novel robust capped norm dual hyper-graph regularized non-negative matrix tri-factorization (RCHNMTF) is proposed. First, a robust capped norm is adopted to handle extreme outliers. Second, dual hyper-graph regularization is considered to exploit intrinsic geometric information in feature manifold and sample manifold. Third, orthogonality constraints are added to learn unique data presentation and improve clustering performance. The experiments on seven datasets testify the robustness and superiority of RCHNMTF.

NMTF-DTI: A Nonnegative Matrix Tri-factorization Approach With Multiple Kernel Fusion for Drug-Target Interaction Prediction.

Prediction of drug-target interactions (DTIs) plays a significant role in drug development and drug discovery. Although this task requires a large investment in terms of time and cost, especially when it is performed experimentally, the results are not necessarily significant. Computational DTI prediction is a shortcut to reduce the risks of experimental methods. In this study, we propose an effective approach of nonnegative matrix tri-factorization, referred to as NMTF-DTI, to predict the interaction scores between drugs and targets. NMTF-DTI utilizes multiple kernels (similarity measures) for drugs and targets and Laplacian regularization to boost the prediction performance. The performance of NMTF-DTI is evaluated via cross-validation and is compared with existing DTI prediction methods in terms of the area under the receiver operating characteristic (ROC) curve (AUC) and the area under the precision and recall curve (AUPR). We evaluate our method on four gold standard datasets, comparing to other state-of-the-art methods. Cross-validation and a separate, manually created dataset are used to set parameters. The results show that NMTF-DTI outperforms other competing methods. Moreover, the results of a case study also confirm the superiority of NMTF-DTI.

Meta Path-Aware Recommendation Method Based on Non-Negative Matrix Factorization in LBSN

Location-based social networks (LBSN) is a new type of heterogeneous information network (HIN). The check-in data usually has the characteristics of a large amount of data and high sparsity. It is a problem worth studying how to effectively discover its complex community structure and accurately recommend it to users. Most HIN-based recommendation methods rely on path-based similarity, which cannot fully mine latent structure features of LBSN users and items. This paper proposes a meta-path-aware common clustering recommendation method MPNMF (meta-path-aware non-negative matrix factorization), based on non-negative matrix tri-factorization. By establishing the objective function based on non-negative matrix tri-factorization and second-order meta-path method, LBSN users and points of interest are integrated with their multi-dimensional heterogeneous relationships. The interrelated user clusters and interest point clusters can be obtained, effectively alleviating the influence of data sparsity. To solve the initial value problem of the model, this method uses the spectral cluster method, which provides a good initial value for the construction of the prediction model. It improves the operational efficiency of the model and the precision of model recommendations. Experiments on the real LBSN datasets show that the proposed method has a high recommendation precision and recall.

Dual-Space Transfer Learning Based on an Indirect Mutual Promotion Strategy

Transfer learning is designed to leverage knowledge in the source domain with labels to help build classification models in the target domain where labels are scarce or even unavailable. Previous studies have shown that high-level concepts extracted from original features are more suitable for cross-domain classification tasks, so many transfer learning methods transfer knowledge by modeling high-level concepts on the original feature space. However, there are two limitations to this method: First, learning high-level concepts directly on the original feature space will reduce the proportion of shared information contained in common features in the process of knowledge transfer bridge construction. Second, only learning multiple high-level concepts on the original feature space, the latent shared information contained in the domain-specific features cannot be targeted learned, so the latent shared information in the domain-specific features cannot be effectively used. To overcome these limitations, this paper proposes a novel method named Dual-Space Transfer Learning based on an Indirect Mutual Promotion Strategy (DSTL). The DSTL method is formalized as an optimization problem based on non-negative matrix tri-factorization. DSTL first extracts the common features between domains and constructs the common feature space. Then, the learning of the high-level concepts of the common feature space and the original feature space is integrated through an indirect promotion strategy, which can enhance the learning effect of common features and domain-specific features through the mutual help of the two feature spaces. The system test on benchmark data sets shows the superiority of the DSTL method.

Identifying cellular cancer mechanisms through pathway-driven data integration.

MotivationCancer is a genetic disease in which accumulated mutations of driver genes induce a functional reorganization of the cell by reprogramming cellular pathways. Current approaches identify cancer pathways as those most internally perturbed by gene expression changes. However, driver genes characteristically perform hub roles between pathways. Therefore, we hypothesize that cancer pathways should be identified by changes in their pathway–pathway relationships.ResultsTo learn an embedding space that captures the relationships between pathways in a healthy cell, we propose pathway-driven non-negative matrix tri-factorization. In this space, we determine condition-specific (i.e. diseased and healthy) embeddings of pathways and genes. Based on these embeddings, we define our ‘NMTF centrality’ to measure a pathway’s or gene’s functional importance, and our ‘moving distance’, to measure the change in its functional relationships. We combine both measures to predict 15 genes and pathways involved in four major cancers, predicting 60 gene–cancer associations in total, covering 28 unique genes. To further exploit driver genes’ tendency to perform hub roles, we model our network data using graphlet adjacency, which considers nodes adjacent if their interaction patterns form specific shapes (e.g. paths or triangles). We find that the predicted genes rewire pathway–pathway interactions in the immune system and provide literary evidence that many are druggable (15/28) and implicated in the associated cancers (47/60). We predict six druggable cancer-specific drug targets.Availability and implementationThe code and data are available at: https://gitlab.bsc.es/swindels/pathway_driven_nmtfSupplementary information Supplementary data are available at Bioinformatics online.

Predicting Drug Synergism by Means of Non-Negative Matrix Tri-Factorization.

Traditional drug experiments to find synergistic drug pairs are time-consuming and expensive due to the numerous possible combinations of drugs that have to be examined. Thus, computational methods that can give suggestions for synergistic drug investigations are of great interest. Here, we propose a Non-negative Matrix Tri-Factorization (NMTF) based approach that leverages the integration of different data types for predicting synergistic drug pairs in multiple specific cell lines. Our computational framework relies on a network-based representation of available data about drug synergism, which also allows integrating genomic information about cell lines. We computationally evaluate the performances of our method in finding missing relationships between synergistic drug pairs and cell lines, and in computing synergy scores between drug pairs in a specific cell line, as well as we estimate the benefit of adding cell line genomic data to the network. Our approach obtains very good performance (Average Precision Score equal to 0.937, Pearson's correlation coefficient equal to 0.760) when cell line genomic data and rich data about synergistic drugs in a cell line are considered. Finally, we systematically searched our top-scored predictions in the available literature and in the NCI ALMANAC, a well-known database of drug combination experiments, proving the goodness of our findings.

Robust collaborative clustering approach with adaptive local structure learning

Different from one-sided clustering technique based on sample similarity, collaborative clustering captures the sample clustering structure and feature clustering structure by mining the duality between samples and features. Non-negative Matrix Tri-Factorization (NMTF) has become one of the most commonly used frameworks for collaborative clustering due to its excellent geometric significance. Unfortunately, NMTF-based approaches generally suffer from the following drawbacks: (1) Based on a fixed and fallible affinity matrix to explore the spatial intrinsic geometry structure, which limits the spatial exploration ability of the model. (2) Based on the L2 loss function to measure the reconstruction error, which weakens the robustness of the model. (3) Based on strict constraints to preserve the orthogonality of the factor matrix, which increases the burden of model solving. To overcome the aforementioned shortcomings, we propose a novel NMTF-based unsupervised collaborative clustering approach. In particular, we first propose a new adaptive local structure learning strategy to enhance the spatial exploration ability of the model. Secondly, we adopt the L1 loss function to improve the robustness of the model. Thirdly, we relax the orthogonal constraints of the factor matrix as a regularization term to reduce the burden of model solving. Finally, we propose a special four-step iterative algorithm to solve the corresponding model, and prove the convergence of the algorithm. The numerical performance on 3 synthetic data sets and 8 benchmark databases demonstrates that the proposed approach outperforms the state-of-the-art collaborative clustering algorithm.

Nonnegative matrix tri-factorization based clustering in a heterogeneous information network with star network schema

Heterogeneous Information Networks (HINs) contain multiple types of nodes and edges; therefore, they can preserve the semantic information and structure information. Cluster analysis using an HIN has obvious advantages over a transformation into a homogenous information network, which can promote the clustering results of different types of nodes. In our study, we applied a Nonnegative Matrix Tri-Factorization (NMTF) in a cluster analysis of multiple metapaths in HIN. Unlike the parameter estimation method of the probability distribution in previous studies, NMTF can obtain several dependent latent variables simultaneously, and each latent variable in NMTF is associated with the cluster of the corresponding node in the HIN. The method is suited to co-clustering leveraging multiple metapaths in HIN, because NMTF is employed for multiple nonnegative matrix factorizations simultaneously in our study. Experimental results on the real dataset show that the validity and correctness of our method, and the clustering result are better than that of the existing similar clustering algorithm.

Parallel Non-negative Matrix Tri-Factorization for Text Data Co-clustering

As a novel paradigm for data mining and dimensionality reduction, Non-negative Matrix Tri-Factorization (NMTF) has attracted much attention due to its notable performance and elegant mathematical derivation, and it has been applied to a plethora of real-world applications, such as text data co-clustering. However, the existing NMTF-based methods usually involve intensive matrix multiplications, which exhibits a major limitation of high computational complexity. With the explosion at both the size and the feature dimension of texts, there is a growing need to develop a parallel and scalable NMTF-based algorithm for text data co-clustering. To this end, we first show in this paper how to theoretically derive the original optimization problem of NMTF by introducing the Lagrangian multipliers. Then, we propose to solve the Lagrange dual objective function in parallel through an efficient distributed implementation. Extensive experiments on five benchmark corpora validate the effectiveness, efficiency, and scalability of our distributed parallel update algorithm for an NMTF-based text data co-clustering method.

Predicting deleterious missense genetic variants via integrative supervised nonnegative matrix tri-factorization

Among an assortment of genetic variations, Missense are major ones which a small subset of them may led to the upset of the protein function and ultimately end in human diseases. Various machine learning methods were declared to differentiate deleterious and benign missense variants by means of a large number of features, including structure, sequence, interaction networks, gene disease associations as well as phenotypes. However, development of a reliable and accurate algorithm for merging heterogeneous information is highly needed as it could be captured all information of complex interactions on network that genes participate in. In this study we proposed a new method based on the non-negative matrix tri-factorization clustering method. We outlined two versions of the proposed method: two-source and three-source algorithms. Two-source algorithm aggregates individual deleteriousness prediction methods and PPI network, and three-source algorithm incorporates gene disease associations into the other sources already mentioned. Four benchmark datasets were employed for internally and externally validation of both algorithms of our predictor. The results at all datasets confirmed that, our method outperforms most state of the art variant prediction tools. Two key features of our variant effect prediction method are worth mentioning. Firstly, despite the fact that the incorporation of gene disease information at three-source algorithm can improve prediction performance by comparison with two-source algorithm, our method did not hinder by type 2 circularity error unlike some recent ensemble-based prediction methods. Type 2 circularity error occurs when the predictor annotates variants on the basis of the genes located on. Secondly, the performance of our predictor is superior over other ensemble-based methods for variants positioned on genes in which we do not have enough information about their pathogenicity.

A convergent algorithm for bi-orthogonal nonnegative matrix tri-factorization

A convergent algorithm for nonnegative matrix factorization (NMF) with orthogonality constraints imposed on both basis and coefficient matrices is proposed in this paper. This factorization concept was first introduced by Ding et al. (Proceedings of 12th ACM SIGKDD international conference on knowledge discovery and data mining, pp 126–135, 2006) with intent to further improve clustering capability of NMF. However, as the original algorithm was developed based on multiplicative update rules, the convergence of the algorithm cannot be guaranteed. In this paper, we utilize the technique presented in our previous work Mirzal (J Comput Appl Math 260:149–166, 2014a; Proceedings of the first international conference on advanced data and information engineering (DaEng-2013). Springer, pp 177–184, 2014b; IEEE/ACM Trans Comput Biol Bioinform 11(6):1208–1217, 2014c) to develop a convergent algorithm for this problem and prove that it converges to a stationary point inside the solution space. As it is very hard to numerically show the convergence of an NMF algorithm due to the slow convergence and numerical precision issues, experiments are instead performed to evaluate whether the algorithm has the nonincreasing property (a necessary condition for the convergence) where it is shown that the algorithm has this property. Further, clustering capability of the algorithm is also inspected by using Reuters-21578 data corpus.

TCD-CF: Triple cross-domain collaborative filtering recommendation

• A triple cross-domain collaborative filtering recommendation method is proposed. • An extended codebook construction algorithm is designed. • A transfer learning method based on extended codebook is presented. • Empirical results show the proposed method is superior to state-of-the-art methods. Recently, data sparsity is still one of the critical problems faced by recommendation systems. Although many existing methods based on cross-domain can alleviate it to a certain extent, these methods only use the information of single-domain (e.g., user-side, item-side and rating-side) or dual-domain (e.g., user-rating-side, user-item-side and item-rating-side) to make recommendations, which results in performance degradation . In this paper, we propose a triple cross-domain collaborative filtering method to alleviate data sparsity, named TCD-CF. In TCD-CF method, the triple-side intrinsic characteristics are first obtained by using the joint nonnegative matrix factorization to integrate the user-side, item-side and rating-side domain knowledge. Then the extended codebook (as knowledge to transfer) based on these intrinsic characteristics is constructed by using the orthogonal nonnegative matrix tri-factorization. Finally, the codebook-based transfer method for cross-system CF is applied into the source domain and target domain to predict the missing ratings and perform recommendation in the target domain. Extensive experiments on two real-world datasets demonstrate that the proposed method outperforms the state-of-the-art methods for the cross-domain recommendation task.

Four algorithms to solve symmetric multi-type non-negative matrix tri-factorization problem

In this paper, we consider the symmetric multi-type non-negative matrix tri-factorization problem (SNMTF), which attempts to factorize several symmetric non-negative matrices simultaneously. This can be considered as a generalization of the classical non-negative matrix tri-factorization problem and includes a non-convex objective function which is a multivariate sixth degree polynomial and a has convex feasibility set. It has a special importance in data science, since it serves as a mathematical model for the fusion of different data sources in data clustering. We develop four methods to solve the SNMTF. They are based on four theoretical approaches known from the literature: the fixed point method (FPM), the block-coordinate descent with projected gradient (BCD), the gradient method with exact line search (GM-ELS) and the adaptive moment estimation method (ADAM). For each of these methods we offer a software implementation: for the former two methods we use Matlab and for the latter Python with the TensorFlow library. We test these methods on three data-sets: the synthetic data-set we generated, while the others represent real-life similarities between different objects. Extensive numerical results show that with sufficient computing time all four methods perform satisfactorily and ADAM most often yields the best mean square error (MSE). However, if the computation time is limited, FPM gives the best MSE because it shows the fastest convergence at the beginning. All data-sets and codes are publicly available on our GitLab profile.

Identifying Protein Complexes With Clear Module Structure Using Pairwise Constraints in Protein Interaction Networks

The protein-protein interaction (PPI) networks can be regarded as powerful platforms to elucidate the principle and mechanism of cellular organization. Uncovering protein complexes from PPI networks will lead to a better understanding of the science of biological function in cellular systems. In recent decades, numerous computational algorithms have been developed to identify protein complexes. However, the majority of them primarily concern the topological structure of PPI networks and lack of the consideration for the native organized structure among protein complexes. The PPI networks generated by high-throughput technology include a fraction of false protein interactions which make it difficult to identify protein complexes efficiently. To tackle these challenges, we propose a novel semi-supervised protein complex detection model based on non-negative matrix tri-factorization, which not only considers topological structure of a PPI network but also makes full use of available high quality known protein pairs with must-link constraints. We propose non-overlapping (NSSNMTF) and overlapping (OSSNMTF) protein complex detection algorithms to identify the significant protein complexes with clear module structures from PPI networks. In addition, the proposed two protein complex detection algorithms outperform a diverse range of state-of-the-art protein complex identification algorithms on both synthetic networks and human related PPI networks.

A Novel Method for Identifying Essential Proteins Based on Non-negative Matrix Tri-Factorization.

Identification of essential proteins is very important for understanding the basic requirements to sustain a living organism. In recent years, there has been an increasing interest in using computational methods to predict essential proteins based on protein–protein interaction (PPI) networks or fusing multiple biological information. However, it has been observed that existing PPI data have false-negative and false-positive data. The fusion of multiple biological information can reduce the influence of false data in PPI, but inevitably more noise data will be produced at the same time. In this article, we proposed a novel non-negative matrix tri-factorization (NMTF)-based model (NTMEP) to predict essential proteins. Firstly, a weighted PPI network is established only using the topology features of the network, so as to avoid more noise. To reduce the influence of false data (existing in PPI network) on performance of identify essential proteins, the NMTF technique, as a widely used recommendation algorithm, is performed to reconstruct a most optimized PPI network with more potential protein–protein interactions. Then, we use the PageRank algorithm to compute the final ranking score of each protein, in which subcellular localization and homologous information of proteins were used to calculate the initial scores. In addition, extensive experiments are performed on the publicly available datasets and the results indicate that our NTMEP model has better performance in predicting essential proteins against the start-of-the-art method. In this investigation, we demonstrated that the introduction of non-negative matrix tri-factorization technology can effectively improve the condition of the protein–protein interaction network, so as to reduce the negative impact of noise on the prediction. At the same time, this finding provides a more novel angle of view for other applications based on protein–protein interaction networks.

Inferring Sentiment from Web Images with Joint Inference on Visual and Social Cues: A Regulated Matrix Factorization Approach

In this paper, we study the problem of understanding human sentiments from large scale collection of Internet images based on both image features and contextual social network information (such as friend comments and user description). Despite the great strides in analyzing user sentiment based on text information, the analysis of sentiment behind the image content has largely been ignored. Thus, we extend the significant advances in text-based sentiment prediction tasks to the higher-level challenge of predicting the underlying sentiments behind the images. We show that neither visual features nor the textual features are by themselves sufficient for accurate sentiment labeling. Thus, we provide a way of using both of them. We leverage the low-level visual features and mid-level attributes of an image, and formulate sentiment prediction problem as a non-negative matrix tri-factorization framework, which has the flexibility to incorporate multiple modalities of information and the capability to learn from heterogeneous features jointly. We develop an optimization algorithm for finding a local-optima solution under the proposed framework. With experiments on two large-scale datasets, we show that the proposed method improves significantly over existing state-of-the-art methods.

Linear functional organization of the omic embedding space.

MotivationWe are increasingly accumulating complex omics data that capture different aspects of cellular functioning. A key challenge is to untangle their complexity and effectively mine them for new biomedical information. To decipher this new information, we introduce algorithms based on network embeddings. Such algorithms represent biological macromolecules as vectors in d-dimensional space, in which topologically similar molecules are embedded close in space and knowledge is extracted directly by vector operations. Recently, it has been shown that neural networks used to obtain vectorial representations (embeddings) are implicitly factorizing a mutual information matrix, called Positive Pointwise Mutual Information (PPMI) matrix. Thus, we propose the use of the PPMI matrix to represent the human protein–protein interaction (PPI) network and also introduce the graphlet degree vector PPMI matrix of the PPI network to capture different topological (structural) similarities of the nodes in the molecular network.ResultsWe generate the embeddings by decomposing these matrices with Nonnegative Matrix Tri-Factorization. We demonstrate that genes that are embedded close in these spaces have similar biological functions, so we can extract new biomedical knowledge directly by doing linear operations on their embedding vector representations. We exploit this property to predict new genes participating in protein complexes and to identify new cancer-related genes based on the cosine similarities between the vector representations of the genes. We validate 80% of our novel cancer-related gene predictions in the literature and also by patient survival curves that demonstrating that 93.3% of them have a potential clinical relevance as biomarkers of cancer.Availability and implementationCode and data are available online at https://gitlab.bsc.es/axenos/embedded-omics-data-geometry/.Supplementary information Supplementary data are available at Bioinformatics online.

A Block Inertial Bregman Proximal Algorithm for Nonsmooth Nonconvex Problems with Application to Symmetric Nonnegative Matrix Tri-Factorization

We propose BIBPA, a block inertial Bregman proximal algorithm for minimizing the sum of a block relatively smooth function (that is, relatively smooth concerning each block) and block separable nonsmooth nonconvex functions. We show that the cluster points of the sequence generated by BIBPA are critical points of the objective under standard assumptions, and this sequence converges globally when a regularization of the objective function satisfies the Kurdyka-Łojasiewicz (KL) property. We also provide the convergence rate when a regularization of the objective function satisfies the Łojasiewicz inequality. We apply BIBPA to the symmetric nonnegative matrix tri-factorization (SymTriNMF) problem, where we propose kernel functions for SymTriNMF and provide closed-form solutions for subproblems of BIBPA.