Matrix Tri-factorization Research Articles

Feature co-shrinking for co-clustering

Many real-world applications require multi-way feature selection rather than single-way feature selection. Multi-way feature selection is more challenging compared to single-way feature selection due to the presence of inter-correlation among the multi-way features. To address this challenge, we propose a novel non-negative matrix tri-factorization model based on co-sparsity regularization to facilitate feature co-shrinking for co-clustering. The basic idea is to learn the inter-correlation among the multi-way features while shrinking the irrelevant ones by encouraging the co-sparsity of the model parameters. The objective is to simultaneously minimize the loss function for the matrix tri-factorization, and the co-sparsity regularization imposed on the model. Furthermore, we develop an efficient and convergence-guaranteed algorithm to solve the non-smooth optimization problem, which works in an iteratively update fashion. The experimental results on various data sets demonstrate the effectiveness of the proposed approach.

Matrix factorization-based data fusion for the prediction of lncRNA-disease associations.

Long non-coding RNAs (lncRNAs) play crucial roles in complex disease diagnosis, prognosis, prevention and treatment, but only a small portion of lncRNA-disease associations have been experimentally verified. Various computational models have been proposed to identify lncRNA-disease associations by integrating heterogeneous data sources. However, existing models generally ignore the intrinsic structure of data sources or treat them as equally relevant, while they may not be. To accurately identify lncRNA-disease associations, we propose a Matrix Factorization based LncRNA-Disease Association prediction model (MFLDA in short). MFLDA decomposes data matrices of heterogeneous data sources into low-rank matrices via matrix tri-factorization to explore and exploit their intrinsic and shared structure. MFLDA can select and integrate the data sources by assigning different weights to them. An iterative solution is further introduced to simultaneously optimize the weights and low-rank matrices. Next, MFLDA uses the optimized low-rank matrices to reconstruct the lncRNA-disease association matrix and thus to identify potential associations. In 5-fold cross validation experiments to identify verified lncRNA-disease associations, MFLDA achieves an area under the receiver operating characteristic curve (AUC) of 0.7408, at least 3% higher than those given by state-of-the-art data fusion based computational models. An empirical study on identifying masked lncRNA-disease associations again shows that MFLDA can identify potential associations more accurately than competing models. A case study on identifying lncRNAs associated with breast, lung and stomach cancers show that 38 out of 45 (84%) associations predicted by MFLDA are supported by recent biomedical literature and further proves the capability of MFLDA in identifying novel lncRNA-disease associations. MFLDA is a general data fusion framework, and as such it can be adopted to predict associations between other biological entities. The source code for MFLDA is available at: http://mlda.swu.edu.cn/codes.php? name = MFLDA. gxyu@swu.edu.cn. Supplementary data are available at Bioinformatics online.

Scalable non-negative matrix tri-factorization

BackgroundMatrix factorization is a well established pattern discovery tool that has seen numerous applications in biomedical data analytics, such as gene expression co-clustering, patient stratification, and gene-disease association mining. Matrix factorization learns a latent data model that takes a data matrix and transforms it into a latent feature space enabling generalization, noise removal and feature discovery. However, factorization algorithms are numerically intensive, and hence there is a pressing challenge to scale current algorithms to work with large datasets. Our focus in this paper is matrix tri-factorization, a popular method that is not limited by the assumption of standard matrix factorization about data residing in one latent space. Matrix tri-factorization solves this by inferring a separate latent space for each dimension in a data matrix, and a latent mapping of interactions between the inferred spaces, making the approach particularly suitable for biomedical data mining.ResultsWe developed a block-wise approach for latent factor learning in matrix tri-factorization. The approach partitions a data matrix into disjoint submatrices that are treated independently and fed into a parallel factorization system. An appealing property of the proposed approach is its mathematical equivalence with serial matrix tri-factorization. In a study on large biomedical datasets we show that our approach scales well on multi-processor and multi-GPU architectures. On a four-GPU system we demonstrate that our approach can be more than 100-times faster than its single-processor counterpart.ConclusionsA general approach for scaling non-negative matrix tri-factorization is proposed. The approach is especially useful parallel matrix factorization implemented in a multi-GPU environment. We expect the new approach will be useful in emerging procedures for latent factor analysis, notably for data integration, where many large data matrices need to be collectively factorized.

Robust co-clustering via dual local learning and high-order matrix factorization

Co-clustering is to group features and samples simultaneously and has received increasing attention in data mining and machine learning, particularly in text document categorization and gene expression. In this paper, two effective co-clustering algorithms are proposed to exploit the joint advantages of local learning and matrix factorization. First, the co-clustering problem is formulated as a form of matrix tri-factorization which embeds local structure learning and orthogonality constraints for clustering indicators. Using high-order matrix factorization, an effective algorithm is proposed for co-clustering problems and its convergence is proved. Second, symmetric co-clustering problems are studied, where the sample affinity matrix serves as the input matrix. Analogous high-order matrix factorization is used to develop an effective convergent algorithm for that problem. Finally, the two proposed algorithms are validated in eight publicly available real-world datasets from machine learning repository. Extensive experiments demonstrate that the proposed algorithms achieve competitive performance over existing state-of-the-art co-clustering methods in all tested datasets.

Community Detection Based on Regularized Semi-Nonnegative Matrix Tri-Factorization in Signed Networks

Community detection is a fundamental task in the social network analysis field, which is beneficial for many real-world applications such as recommendation systems and telephone fraud detection. Community detection in unsigned networks has been extensively studied, however, few works focus on community detection in signed networks. Under this background, we propose a framework based on regularized semi-nonnegative matrix tri-factorization which maps the signed network from high-dimensional space to low-dimensional space, such that the communities of the signed network can be derived. In addition, to improve the detection accuracy, we introduce a graph regularization to distribute the pair of nodes which are connected with negative links into different communities. The experimental results on both synthetic datasets and real-world datasets verify the effectiveness of the proposed method.

Penalized nonnegative matrix tri-factorization for co-clustering

Nonnegative matrix factorization has been widely used in co-clustering tasks which group data points and features simultaneously. In recent years, several proposed co-clustering algorithms have shown their superiorities over traditional one-side clustering, especially in text clustering and gene expression. Due to the NP-completeness of the co-clustering problems, most existing methods relaxed the orthogonality constraint as nonnegativity, which often deteriorates performance and robustness as a result. In this paper, penalized nonnegative matrix tri-factorization is proposed for co-clustering problems, where three penalty terms are introduced to guarantee the near orthogonality of the clustering indicator matrices. An iterative updating algorithm is proposed and its convergence is proved. Furthermore, the high-order nonnegative matrix tri-factorization technique is provided for symmetric co-clustering tasks and a corresponding algorithm with proved convergence is also developed. Finally, extensive experiments in six real-world datasets demonstrate that the proposed algorithms outperform the compared state-of-the-art co-clustering methods.

Fast smooth rank function approximation based on matrix tri-factorization

Recently, Smooth Rank Function (SRF) is proposed for matrix completion problem. The main idea of this algorithm is based on a continuous and differentiable approximation of the rank function. However, it need to deal with singular value decomposition of matrix in each iteration, which consumes much time for large matrix. In this paper, by utilizing the tri-factorization of matrix, a fast matrix completion method based on SRF is proposed. Then, based on our fast matrix completion method, a rank adaptive smooth rank function approximation is presented with appropriate rank estimation. We mathematically prove the convergence of the proposed method. Experimental results show that our proposed method improves the running time significantly. Furthermore, our proposed method outperforms other existing matrix completion approaches in most cases.

Multi-Task Multi-View Clustering

Multi-task clustering and multi-view clustering have severally found wide applications and received much attention in recent years. Nevertheless, there are many clustering problems that involve both multi-task clustering and multi-view clustering, i.e., the tasks are closely related and each task can be analyzed from multiple views. In this paper, we introduce a multi-task multi-view clustering framework which integrates within-view-task clustering, multi-view relationship learning, and multi-task relationship learning. Under this framework, we propose two multi-task multi-view clustering algorithms, the bipartite graph based multi-task multi-view clustering algorithm, and the semi-nonnegative matrix tri-factorization based multi-task multi-view clustering algorithm. The former one can deal with the multi-task multi-view clustering of nonnegative data, the latter one is a general multi-task multi-view clustering method, i.e., it can deal with the data with negative feature values. Experimental results on publicly available data sets in web page mining and image mining show the superiority of the proposed multi-task multi-view clustering algorithms over either multi-task clustering algorithms or multi-view clustering algorithms for multi-task clustering of multi-view data.

Exploration of Web Page Structural Patterns Based on Request Dependency Graph Decomposition

This article first proposed a Bipartite Request Dependency Graph (BRDG) that describes the object-level interrelationships between user click requests and embedded web object requests. These two kinds of requests are classified from HTTP data by an identification algorithm. The interrelationships between user click requests and embedded web object reflect the web page structural, which contain latent web information. Exploring structural patterns is crucial for many aspects like web security analysis and web information visualization. Accordingly, the article also proposed a novel graph decomposition method called orthogonal nonnegative matrix tri-factorization (tNMF) to the BRDG. Compared to traditional web graph analysis focus on statistical and structural properties of the whole graph, the proposed method is dedicated to mine latent web structural patterns. Decomposition results demonstrate that several interesting structures exist in the BRDG. The article aims at classifying these subgraphs as several structural patterns and shedding light on the causes of these patterns.

A Network-Based Data Integration Approach to Support Drug Repurposing and Multi-Target Therapies in Triple Negative Breast Cancer.

The integration of data and knowledge from heterogeneous sources can be a key success factor in drug design, drug repurposing and multi-target therapies. In this context, biological networks provide a useful instrument to highlight the relationships and to model the phenomena underlying therapeutic action in cancer. In our work, we applied network-based modeling within a novel bioinformatics pipeline to identify promising multi-target drugs. Given a certain tumor type/subtype, we derive a disease-specific Protein-Protein Interaction (PPI) network by combining different data-bases and knowledge repositories. Next, the application of suitable graph-based algorithms allows selecting a set of potentially interesting combinations of drug targets. A list of drug candidates is then extracted by applying a recent data fusion approach based on matrix tri-factorization. Available knowledge about selected drugs mechanisms of action is finally exploited to identify the most promising candidates for planning in vitro studies. We applied this approach to the case of Triple Negative Breast Cancer (TNBC), a subtype of breast cancer whose biology is poorly understood and that lacks of specific molecular targets. Our “in-silico” findings have been confirmed by a number of in vitro experiments, whose results demonstrated the ability of the method to select candidates for drug repurposing.

Inferring Crohn's disease association from exome sequences by integrating biological knowledge.

BackgroundExome sequencing has been emerged as a primary method to identify detailed sequence variants associated with complex diseases including Crohn’s disease in the protein-coding regions of human genome. However, constructing an interpretable model for exome sequencing data is challenging because of the huge diversity of genomic variation. In addition, it has been known that utilizing biologically relevant information in a rigorous manner is essential for effectively extracting disease-associated information.ResultsIn this paper, we incorporate three different types of biological knowledge such as predicted pathogenicity, disease gene annotation, and functional interaction network of human genes, and integrate them with exome sequence data in non-negative matrix tri-factorization framework. Based on the proposed method, we successfully identified Crohn’s disease patients from exome sequencing data and achieved the area under the receiver operating characteristics curve (AUC) of 0.816, while other clustering methods not using biological information achieved the AUC of 0.786. Moreover, the disease association score derived from our method showed higher correlation with Crohn’s disease genes than other unrelated genes.ConclusionsAs a consequence, by integrating biological information across multiple levels such as variant, gene, and systems, our method could be useful for identifying disease susceptibility and its associated genes from exome sequencing data.

Multi-bridge transfer learning

Transfer learning, which aims to exploit the knowledge in the source domains to promote the learning tasks in the target domains, has attracted extensive research interests recently. The general idea of the previous approaches is to model the shared structure in one latent space as the bridge across domains by reducing the distribution divergences. However, there exist some latent factors in the other latent spaces, which can also be utilized to draw the corresponding distributions closer for establishing the bridges. In this paper, we propose a novel transfer learning method, referred to as Multi-Bridge Transfer Learning (MBTL), to learn the distributions in the different latent spaces together. Therefore, more latent factors shared can be utilized to transfer knowledge. Additionally, an iterative algorithm with convergence guarantee based on non-negative matrix tri-factorization techniques is proposed to solve the optimization problem. Comprehensive experiments demonstrate that MBTL can significantly outperform state-of-the-art learning methods on the topic and sentiment classification tasks.

Domain adaptation via Multi-Layer Transfer Learning

Transfer learning, which leverages labeled data in a source domain to train an accurate classifier for classification tasks in a target domain, has attracted extensive research interests recently for its effectiveness proven by many studies. Previous approaches adopt a common strategy that models the shared structure as a bridge across different domains by reducing distribution divergences. However, those approaches totally ignore specific latent spaces, which can be utilized to learn non-shared concepts. Only specific latent spaces contain specific latent factors, lacking which will lead to ineffective distinct concept learning. Additionally, only learning latent factors in one latent feature space layer may ignore those in the other layers. The missing latent factors may also help us to model the latent structure shared as the bridge. This paper proposes a novel transfer learning method Multi-Layer Transfer Learning (MLTL). MLTL first generates specific latent feature spaces. Second, it combines these specific latent feature spaces with common latent feature space into one latent feature space layer. Third, it generates multiple layers to learn the corresponding distributions on different layers with their pluralism simultaneously. Specifically, the pluralism of the distributions on different layers means that learning the distributions on one layer can help us to learn the distributions on the others. Furthermore, an iterative algorithm based on Non-Negative Matrix Tri-Factorization is proposed to solve the optimization problem. Comprehensive experiments demonstrate that MLTL can significantly outperform the state-of-the-art learning methods on topic and sentiment classification tasks.

Analysis on a Nonnegative Matrix Factorization and Its Applications

In this work we perform some mathematical analysis on a special nonnegative matrix trifactorization (NMF) and apply this NMF to some imaging and inverse problems. We will propose a sparse low-rank approximation of positive data and images in terms of tensor products of positive vectors and investigate its effectiveness in terms of the number of tensor products to be used in the approximation. A new multilevel analysis (MLA) framework is suggested to extract major components in the matrix representing structures of different resolutions but still preserve the positivity of the basis and sparsity of the approximation. We will also propose and formulate a semismooth Newton method based on primal-dual active sets for the nonnegative factorization. Numerical results are given to demonstrate the effectiveness of the proposed method at capturing features in images and structures of inverse problems under no a priori assumption on the underlying structure in the data as well as to provide a sparse low-rank representation of the data.

Sparse topical analysis of dyadic data using matrix tri-factorization

Many applications involve dyadic data, where associations between one pair of domain entities, such as documents, words and associations between another pair, such as documents, users are completely observed. We motivate the analysis of such dyadic data introducing an additional discrete dimension, which we call topics, and explore sparse relationships between the domain entities and the topic, such as user-topic and document-topic relationships. For this problem of sparse topical analysis of dyadic data, we propose a formulation using sparse matrix tri-factorization. This formulation requires sparsity constraints, not only on the individual factor matrices, but also on the product of two of the factors. To the best of our knowledge, this problem of sparse matrix tri-factorization has not been studied before. We propose a solution that introduces a surrogate for the product of factors and enforce sparsity on this surrogate as well as on the individual factors through L1-regularization. The resulting optimization problem is efficiently solvable in an alternating minimization framework over sub-problems involving individual factors using the well known FISTA algorithm. For the sub-problems that are constrained, we use a projected variant of the FISTA algorithm. We also show that our formulation leads to independent sub-problems towards solving a factor matrix, thereby supporting parallel implementation leading to scalable solution. We perform experiments over bibliographic and product review data to show that the proposed framework based on sparse tri-factorization formulation results in better generalization ability and factorization accuracy compared to baselines that use sparse bi-factorization.

Fuse: multiple network alignment via data fusion.

Discovering patterns in networks of protein-protein interactions (PPIs) is a central problem in systems biology. Alignments between these networks aid functional understanding as they uncover important information, such as evolutionary conserved pathways, protein complexes and functional orthologs. However, the complexity of the multiple network alignment problem grows exponentially with the number of networks being aligned and designing a multiple network aligner that is both scalable and that produces biologically relevant alignments is a challenging task that has not been fully addressed. The objective of multiple network alignment is to create clusters of nodes that are evolutionarily and functionally conserved across all networks. Unfortunately, the alignment methods proposed thus far do not meet this objective as they are guided by pairwise scores that do not utilize the entire functional and evolutionary information across all networks. To overcome this weakness, we propose Fuse, a new multiple network alignment algorithm that works in two steps. First, it computes our novel protein functional similarity scores by fusing information from wiring patterns of all aligned PPI networks and sequence similarities between their proteins. This is in contrast with the previous tools that are all based on protein similarities in pairs of networks being aligned. Our comprehensive new protein similarity scores are computed by Non-negative Matrix Tri-Factorization (NMTF) method that predicts associations between proteins whose homology (from sequences) and functioning similarity (from wiring patterns) are supported by all networks. Using the five largest and most complete PPI networks from BioGRID, we show that NMTF predicts a large number protein pairs that are biologically consistent. Second, to identify clusters of aligned proteins over all networks, Fuse uses our novel maximum weight k-partite matching approximation algorithm. We compare Fuse with the state of the art multiple network aligners and show that (i) by using only sequence alignment scores, Fuse already outperforms other aligners and produces a larger number of biologically consistent clusters that cover all aligned PPI networks and (ii) using both sequence alignments and topological NMTF-predicted scores leads to the best multiple network alignments thus far. Our dataset and software are freely available from the web site: http://bio-nets.doc.ic.ac.uk/Fuse/ natasha@imperial.ac.uk Supplementary data are available at Bioinformatics online.

Co-ClusterD: A Distributed Framework for Data Co-Clustering with Sequential Updates

Co-clustering has emerged to be a powerful data mining tool for two-dimensional co-occurrence and dyadic data. However, co-clustering algorithms often require significant computational resources and have been dismissed as impractical for large data sets. Existing studies have provided strong empirical evidence that expectation-maximization (EM) algorithms (e.g., k-means algorithm) with sequential updates can significantly reduce the computational cost without degrading the resulting solution. Motivated by this observation, we introduce sequential updates for alternate minimization co-clustering (AMCC) algorithms which are variants of EM algorithms, and also show that AMCC algorithms with sequential updates converge. We then propose two approaches to parallelize AMCC algorithms with sequential updates in a distributed environment. Both approaches are proved to maintain the convergence properties of AMCC algorithms. Based on these two approaches, we present a new distributed framework, Co-ClusterD, which supports efficient implementations of AMCC algorithms with sequential updates. We design and implement Co-ClusterD, and show its efficiency through two AMCC algorithms: fast nonnegative matrix tri-factorization (FNMTF) and information theoretic co-clustering (ITCC). We evaluate our framework on both a local cluster of machines and the Amazon EC2 cloud. Empirical results show that AMCC algorithms implemented in Co-ClusterD can achieve a much faster convergence and often obtain better results than their traditional concurrent counterparts.

Quadruple Transfer Learning: Exploiting both shared and non-shared concepts for text classification

Transfer learning focuses on leveraging the knowledge in source domains to complete the learning tasks in target domains, where the data distributions of the source and target domains are related but different in accordance with original features. To tackle the challenge of different data distributions, previous methods mine the high-level concepts (e.g., feature clusters) from original features, which shows to be suitable for the classification. The general strategies of the previous approaches are to utilize the identical concepts, the synonymous concepts or both of them as shared concepts to establish the bridge between the source and target domains. Besides the shared concepts, some methods use the different concepts for training model. Specifically, these methods assume that the identical concepts (e.g., feature clusters) in different domains can be mapped to the same example classes. However, some ambiguous concepts may exist in different domains and result in misleading classification in the target domains. Therefore, we need a general transfer learning framework, which can exploit four kinds of concepts including the identical concepts, the synonymous concepts, the different concepts and the ambiguous concepts simultaneously, for cross-domain classification.In this paper, we present a novel method, Quadruple Transfer Learning (QTL), which models these four kinds of concepts together to fit different situations on the data distributions. In addition, an iterative algorithm with convergence guarantee based on non-negative matrix tri-factorization techniques is presented to solve the optimization problem. Finally, systematic experiments demonstrate that QTL is more effective than all the compared baselines.

Anomaly Detection in Microblogging via Co-Clustering

Traditional anomaly detection on microblogging mostly focuses on individual anomalous users or messages. Since anomalous users employ advanced intelligent means, the anomaly detection is greatly poor in performance. In this paper, we propose an innovative framework of anomaly detection based on bipartite graph and co-clustering. A bipartite graph between users and messages is built to model the homogeneous and heterogeneous interactions. The proposed co-clustering algorithm based on nonnegative matrix tri-factorization can detect anomalous users and messages simultaneously. The homogeneous relations modeled by the bipartite graph are used as constraints to improve the accuracy of the co-clustering algorithm. Experimental results show that the proposed scheme can detect individual and group anomalies with high accuracy on a Sina Weibo dataset.

Large-Scale Cross-Language Web Page Classification via Dual Knowledge Transfer Using Fast Nonnegative Matrix Trifactorization

With the rapid growth of modern technologies, Internet has reached almost every corner of the world. As a result, it becomes more and more important to manage and mine information contained in Web pages in different languages. Traditional supervised learning methods usually require a large amount of training data to obtain accurate and robust classification models. However, labeled Web pages did not increase as fast as the growth of Internet. The lack of sufficient training Web pages in many languages, especially for those in uncommonly used languages, makes it a challenge for traditional classification algorithms to achieve satisfactory performance. To address this, we observe that Web pages for a same topic from different languages usually share some common semantic patterns, though in different representation forms. In addition, we also observe that the associations between word clusters and Web page classes are another type of reliable carriers to transfer knowledge across languages. With these recognitions, in this article we propose a novel joint nonnegative matrix trifactorization (NMTF) based Dual Knowledge Transfer (DKT) approach for cross-language Web page classification. Our approach transfers knowledge from the auxiliary language, in which abundant labeled Web pages are available, to the target languages, in which we want to classify Web pages, through two different paths: word cluster approximation and the associations between word clusters and Web page classes. With the reinforcement between these two different knowledge transfer paths, our approach can achieve better classification accuracy. In order to deal with the large-scale real world data, we further develop the proposed DKT approach by constraining the factor matrices of NMTF to be cluster indicator matrices. Due to the nature of cluster indicator matrices, we can decouple the proposed optimization objective and the resulted subproblems are of much smaller sizes involving much less matrix multiplications, which make our new approach much more computationally efficient. We evaluate the proposed approach in extensive experiments using a real world cross-language Web page data set. Promising results have demonstrated the effectiveness of our approach that are consistent with our theoretical analyses.