Co-clustering Of Data Research Articles

CO-CLUSTERING OF SPATIALLY RESOLVED TRANSCRIPTOMIC DATA.

Spatial transcriptomics is a groundbreaking technology that allows the measurement of the activity of thousands of genes in a tissue sample and maps where the activity occurs. This technology has enabled the study of the spatial variation of the genes across the tissue. Comprehending gene functions and interactions in different areas of the tissue is of great scientific interest, as it might lead to a deeper understanding of several key biological mechanisms, such as cell-cell communication or tumor-microenvironment interaction. To do so, one can group cells of the same type and genes that exhibit similar expression patterns. However, adequate statistical tools that exploit the previously unavailable spatial information to more coherently group cells and genes are still lacking. In this work, we introduce SpaRTaCo, a new statistical model that clusters the spatial expression profiles of the genes according to a partition of the tissue. This is accomplished by performing a co-clustering, i.e., inferring the latent block structure of the data and inducing two types of clustering: of the genes, using their expression across the tissue, and of the image areas, using the gene expression in the spots where the RNA is collected. Our proposed methodology is validated with a series of simulation experiments and its usefulness in responding to specific biological questions is illustrated with an application to a human brain tissue sample processed with the 10X-Visium protocol.

Parallel Non-negative Matrix Tri-Factorization for Text Data Co-clustering

As a novel paradigm for data mining and dimensionality reduction, Non-negative Matrix Tri-Factorization (NMTF) has attracted much attention due to its notable performance and elegant mathematical derivation, and it has been applied to a plethora of real-world applications, such as text data co-clustering. However, the existing NMTF-based methods usually involve intensive matrix multiplications, which exhibits a major limitation of high computational complexity. With the explosion at both the size and the feature dimension of texts, there is a growing need to develop a parallel and scalable NMTF-based algorithm for text data co-clustering. To this end, we first show in this paper how to theoretically derive the original optimization problem of NMTF by introducing the Lagrangian multipliers. Then, we propose to solve the Lagrange dual objective function in parallel through an efficient distributed implementation. Extensive experiments on five benchmark corpora validate the effectiveness, efficiency, and scalability of our distributed parallel update algorithm for an NMTF-based text data co-clustering method.

Co-clustering algorithms for distributional data with automated variable weighting

This paper is concerned with the co-clustering of distribution-valued data, that is, the simultaneous partitioning of rows and columns of an input data table, the elements of which are distributions (or histograms) representing aggregate data. The first proposed method extends the double k-means algorithm to distributional data. The L2 Wasserstein distance, also known as Mallow’s distance, is used to compare distributions. To consider the different relevance of the variables characterizing the clusters, four variants of adaptive distributional double k-means are proposed. Accordingly, in the co-clustering procedure, an additional step is introduced to compute the relevance weights associated with the variables. In particular, each of the four algorithms provides i) a set of weights for the variables; ii) different sets of weights for the variables, one for each cluster (cluster-wise); iii) a double set of weights for the variables according to the decomposition of the L2 Wasserstein distance into two components; iv) different double sets of weights for the variables and distance components, one for each cluster (cluster-wise). Applications using simulated and real data demonstrate the effectiveness of the proposed algorithms and the contribution of the relevance weights to the co-clustering procedure according to the structure of the data.

Co-clustering optimization using Artificial Bee Colony (ABC) algorithm

This paper presents an Artificial Bee Colony (ABC) optimization based algorithm for co-clustering of high-dimensional data. The ABC algorithm is used for optimization problems including data clustering. We incorporate aspects of co-clustering by embedding it into the objective function used for clustering by the ABC algorithm. Instead of a linear metric, such as the Euclidean distance, we propose the use of higher order correlations to build similarity between rows and columns, each based on the other. This measure uses co-evolving similarities which when embedded into the objective function results in optimizing the co-clusters. The search space is also explored in the vicinity of the solutions produced by the ABC algorithm using three local search methods — the first is a heuristic based on computing the cluster means; the second uses the analytical gradient of the objective with respect to a centroid to find lower cost solutions in the vicinity; and, the third is a hybrid of the first two methods. Numerical experiments show significant improvement in the search for optimal clustering by incorporating new similarity metric and optimized local search method. Finally, the algorithm is shown to be highly scalable for parallel architectures for both distributed and shared memory systems. Theoretically, the best iso-efficiency function of Θ (p log p) for fully connected network with p processors is also computed for the parallel algorithm.

A Cloud-Edge-Aided Incremental High-Order Possibilistic c-Means Algorithm for Medical Data Clustering

Medical Internet of Things are generating a big volume of data to enable smart medicine that tries to offer computer-aided medical and healthcare services with artificial intelligence techniques like deep learning and clustering. However, it is a challenging issue for deep learning, and clustering algorithms to analyze large medical data because of their high computational complexity, thus hindering the progress of smart medicine. In this article, we present an incremental high-order possibilistic c-means algorithm (IHoPCM) on a cloud-edge computing system to achieve medical data coclustering of multiple hospitals in different locations. Specifically, each hospital employs the deep computation model to learn a feature tensor of each medical data object on the local edge computing system, and then uploads the feature tensors to the cloud computing platform. The high-order possibilistic c-means algorithm is performed on the cloud system for medical data clustering on uploaded feature tensors. Once the new medical data feature tensors are arriving at the cloud computing platform, the incremental high-order possibilistic c-means algorithm (IHoPCM) is performed on the combination of the new feature tensors and the previous clustering centers to obtain clustering results for the feature tensors received to date. In this way, repeated clustering on the previous feature tensors is avoided to improve the clustering efficiency. In the experiments, we compare different algorithms on two medical datasets regarding clustering accuracy and clustering efficiency. Results show that the presented IHoPCM method achieves great improvements over the compared algorithms in clustering accuracy and efficiency.

Co-Clustering of Ordinal Data via Latent Continuous Random Variables and Not Missing at Random Entries

ABSTRACTThis article is about the co-clustering of ordinal data. Such data are very common on e-commerce platforms where customers rank the products/services they bought. In more detail, we focus on arrays of ordinal (possibly missing) data involving two disjoint sets of individuals/objects corresponding to the rows/columns of the arrays. Typically, an observed entry (i, j) in the array is an ordinal score assigned by the individual/row i to the object/column j. A new generative model for arrays of ordinal data is introduced along with an inference algorithm for parameters estimation. The model accounts for not missing at random data and relies on latent continuous random variables. The fitting allows to simultaneously co-cluster the rows and columns of an array. The estimation of the model parameters is performed via a classification expectation maximization algorithm. A model selection criterion is formally obtained to select the number of row and column clusters. To show that our approach reaches and often outperforms the state of the art, we carry out numerical experiments on synthetic data. Finally, applications on real datasets highlight the model capacity to deal with very sparse arrays. Supplementary materials for this article are available online.

Word Co-Occurrence Regularized Non-Negative Matrix Tri-Factorization for Text Data Co-Clustering

Text data co-clustering is the process of partitioning the documents and words simultaneously. This approach has proven to be more useful than traditional one-sided clustering when dealing with sparsity. Among the wide range of co-clustering approaches, Non-Negative Matrix Tri-Factorization (NMTF) is recognized for its high performance, flexibility and theoretical foundations. One important aspect when dealing with text data, is to capture the semantic relationships between words since documents that are about the same topic may not necessarily use exactly the same vocabulary. However, this aspect has been overlooked by previous co-clustering models, including NMTF. To address this issue, we rely on the distributional hypothesis stating that words which co-occur frequently within the same context, e.g., a document or sentence, are likely to have similar meanings. We then propose a new NMTF model that maps frequently co-occurring words roughly to the same direction in the latent space to reflect the relationships between them. To infer the factor matrices, we derive a scalable alternating optimization algorithm, whose convergence is guaranteed. Extensive experiments, on several real-world datasets, provide strong evidence for the effectiveness of the proposed approach, in terms of co-clustering.

Multi-manifold matrix decomposition for data co-clustering

We propose a novel Multi-Manifold Matrix Decomposition for Co-clustering (M3DC) algorithm that considers the geometric structures of both the sample manifold and the feature manifold simultaneously. Specifically, multiple candidate manifolds are constructed separately to take local invariance into account. Then, we employ multi-manifold learning to approximate the optimal intrinsic manifold, which better reflects the local geometrical structure, by linearly combining these candidate manifolds. In M3DC, the candidate manifolds are obtained using various manifold-based dimensionality reduction methods. These methods are based on different rationales and use different metrics for data distances. Experimental results on several real data sets demonstrate the effectiveness of our proposed M3DC.

Large-Scale Heterogeneous Data Co-Clustering Based on Nonnegative Matrix Factorization

Large-Scale Heterogeneous Data Co-Clustering Based on Nonnegative Matrix Factorization

Co-ClusterD: A Distributed Framework for Data Co-Clustering with Sequential Updates

Co-clustering has emerged to be a powerful data mining tool for two-dimensional co-occurrence and dyadic data. However, co-clustering algorithms often require significant computational resources and have been dismissed as impractical for large data sets. Existing studies have provided strong empirical evidence that expectation-maximization (EM) algorithms (e.g., k-means algorithm) with sequential updates can significantly reduce the computational cost without degrading the resulting solution. Motivated by this observation, we introduce sequential updates for alternate minimization co-clustering (AMCC) algorithms which are variants of EM algorithms, and also show that AMCC algorithms with sequential updates converge. We then propose two approaches to parallelize AMCC algorithms with sequential updates in a distributed environment. Both approaches are proved to maintain the convergence properties of AMCC algorithms. Based on these two approaches, we present a new distributed framework, Co-ClusterD, which supports efficient implementations of AMCC algorithms with sequential updates. We design and implement Co-ClusterD, and show its efficiency through two AMCC algorithms: fast nonnegative matrix tri-factorization (FNMTF) and information theoretic co-clustering (ITCC). We evaluate our framework on both a local cluster of machines and the Amazon EC2 cloud. Empirical results show that AMCC algorithms implemented in Co-ClusterD can achieve a much faster convergence and often obtain better results than their traditional concurrent counterparts.

Semi-Supervised Heterogeneous Fusion for Multimedia Data Co-Clustering

Co-clustering is a commonly used technique for tapping the rich meta-information of multimedia web documents, including category, annotation, and description, for associative discovery. However, most co-clustering methods proposed for heterogeneous data do not consider the representation problem of short and noisy text and their performance is limited by the empirical weighting of the multi-modal features. In this paper, we propose a generalized form of Heterogeneous Fusion Adaptive Resonance Theory, called GHF-ART, for co-clustering of large-scale web multimedia documents. By extending the two-channel Heterogeneous Fusion ART (HF-ART) to multiple channels, GHF-ART is designed to handle multimedia data with an arbitrarily rich level of meta-information. For handling short and noisy text, GHF-ART does not learn directly from the textual features. Instead, it identifies key tags by learning the probabilistic distribution of tag occurrences. More importantly, GHF-ART incorporates an adaptive method for effective fusion of multi- modal features, which weights the features of multiple data sources by incrementally measuring the importance of feature modalities through the intra-cluster scatters. Extensive experiments on two web image data sets and one text document set have shown that GHF-ART achieves significantly better clustering performance and is much faster than many existing state-of-the-art algorithms.

Evolutionary soft co-clustering: formulations, algorithms, and applications

We consider the co-clustering of time-varying data using evolutionary co-clustering methods. Existing approaches are based on the spectral learning framework, thus lacking a probabilistic interpretation. We overcome this limitation by developing a probabilistic model in this paper. The proposed model assumes that the observed data are generated via a two-step process that depends on the historic co-clusters. This allows us to capture the temporal smoothness in a probabilistically principled manner. To perform maximum likelihood parameter estimation, we present an EM-based algorithm. We also establish the convergence of the proposed EM algorithm. An appealing feature of the proposed model is that it leads to soft co-clustering assignments naturally. We evaluate the proposed method on both synthetic and real-world data sets. Experimental results show that our method consistently outperforms prior approaches based on spectral method. To fully exploit the real-world impact of our methods, we further perform a systematic application study on the analysis of Drosophila gene expression pattern images. We encode the spatial gene expression information at a particular developmental time point into a data matrix using a mesh-generation pipeline. We then co-cluster the embryonic domains and the genes simultaneously for multiple time points using our evolutionary co-clustering method. Results show that the co-clusters of gene and embryonic domains reflect the underlying biology.

Parameter-less co-clustering for star-structured heterogeneous data

The availability of data represented with multiple features coming from heterogeneous domains is getting more and more common in real world applications. Such data represent objects of a certain type, connected to other types of data, the features, so that the overall data schema forms a star structure of inter-relationships. Co-clustering these data involves the specification of many parameters, such as the number of clusters for the object dimension and for all the features domains. In this paper we present a novel co-clustering algorithm for heterogeneous star-structured data that is parameter-less. This means that it does not require either the number of row clusters or the number of column clusters for the given feature spaces. Our approach optimizes the Goodman---Kruskal's ?, a measure for cross-association in contingency tables that evaluates the strength of the relationship between two categorical variables. We extend ? to evaluate co-clustering solutions and in particular we apply it in a higher dimensional setting. We propose the algorithm CoStar which optimizes ? by a local search approach. We assess the performance of CoStar on publicly available datasets from the textual and image domains using objective external criteria. The results show that our approach outperforms state-of-the-art methods for the co-clustering of heterogeneous data, while it remains computationally efficient.

Aggregate two-way co-clustering of ads and user data for online advertisements

Clustering plays an important role in data mining, as it is used by many applications as a preprocessing step for data analysis. Traditional clustering focuses on grouping similar objects, while two-way co-clustering can group dyadic data (objects as well as their attributes) simultaneously. In this research, we apply two-way co-clustering to the analysis of online advertising where both ads and users need to be clustered. However, in addition to the ad-user link matrix that denotes the ads which a user has linked, we also have two additional matrices, which represent extra information about users and ads. In this paper, we proposed a 3-staged clustering method that makes use of the three data matrices to enhance clustering performance. In addition, an Iterative Cross Co-Clustering (ICCC) algorithm is also proposed for two-way co-clustering. The experiment is performed using the advertisement and user data from Morgenstern, a financial social website that focuses on the agency of advertisements. The result shows that iterative cross co-clustering provides better performance than traditional clustering and completes the task more efficiently.

Constrained co-clustering with non-negative matrix factorisation

Co-clustering refers to the problem of deriving sub-matrices of the data matrix by simultaneously clustering the rows (data instances) and columns (features) of the matrix. While very effective in discovering useful knowledge, many of the co-clustering algorithms adopt a completely unsupervised approach. Integration of domain knowledge can guide the co-clustering process and greatly enhance the overall performance. We propose a semi-supervised Non-negative Matrix-factorisation (SS-NMF) based framework to integrate domain knowledge in the form of must-link and cannot-link constraints. Specifically, we augment the data matrix by integrating the constraints using metric learning and then perform NMF to obtain co-clustering. Under the proposed framework, we present two approaches to integrate domain knowledge, viz. a distance metric learning approach and an information theoretic metric learning approach. Through experiments performed on real-world web service data and publicly available text datasets, we demonstrate the performance of the proposed SS-NMF based approach for data co-clustering.

Non-Negative Matrix Factorization for Semisupervised Heterogeneous Data Coclustering

Coclustering heterogeneous data has attracted extensive attention recently due to its high impact on various important applications, such us text mining, image retrieval, and bioinformatics. However, data coclustering without any prior knowledge or background information is still a challenging problem. In this paper, we propose a Semisupervised Non-negative Matrix Factorization (SS-NMF) framework for data coclustering. Specifically, our method computes new relational matrices by incorporating user provided constraints through simultaneous distance metric learning and modality selection. Using an iterative algorithm, we then perform trifactorizations of the new matrices to infer the clusters of different data types and their correspondence. Theoretically, we prove the convergence and correctness of SS-NMF coclustering and show the relationship between SS-NMF with other well-known coclustering models. Through extensive experiments conducted on publicly available text, gene expression, and image data sets, we demonstrate the superior performance of SS-NMF for heterogeneous data coclustering.

Bipartite isoperimetric graph partitioning for data co-clustering

Data co-clustering refers to the problem of simultaneous clustering of two data types. Typically, the data is stored in a contingency or co-occurrence matrix C where rows and columns of the matrix represent the data types to be co-clustered. An entry C ij of the matrix signifies the relation between the data type represented by row i and column j. Co-clustering is the problem of deriving sub-matrices from the larger data matrix by simultaneously clustering rows and columns of the data matrix. In this paper, we present a novel graph theoretic approach to data co-clustering. The two data types are modeled as the two sets of vertices of a weighted bipartite graph. We then propose Isoperimetric Co-clustering Algorithm (ICA)--a new method for partitioning the bipartite graph. ICA requires a simple solution to a sparse system of linear equations instead of the eigenvalue or SVD problem in the popular spectral co-clustering approach. Our theoretical analysis and extensive experiments performed on publicly available datasets demonstrate the advantages of ICA over other approaches in terms of the quality, efficiency and stability in partitioning the bipartite graph.

Joint co-clustering: Co-clustering of genomic and clinical bioimaging data

For better understanding the genetic mechanisms underlying clinical observations, and better defining a group of potential candidates for protein-family-inhibiting therapy, it is interesting to determine the correlations between genomic, clinical data and data coming from high resolution and fluorescent microscopy. We introduce a computational method, called joint co-clustering , that can find co-clusters or groups of genes, bioimaging parameters and clinical traits that are believed to be closely related to each other based on the given empirical information. As bioimaging parameters, we quantify the expression of growth factor receptor EGFR/erb-B family in non-small cell lung carcinoma (NSCLC) through a fully-automated computer-aided analysis approach. This immunohistochemical analysis is usually performed by pathologists via visual inspection of tissue samples images. Our fully-automated techniques streamlines this error-prone and time-consuming process, thereby facilitating analysis and diagnosis. Experimental results for several real-life datasets demonstrate the high quantitative precision of our approach. The joint co-clustering method was tested with the receptor EGFR/erb-B family data on non-small cell lung carcinoma (NSCLC) tissue and identified statistically significant co-clusters of genes, receptor protein expression and clinical traits. The validation of our results with the literature suggest that the proposed method can provide biologically meaningful co-clusters of genes and traits and that it is a very promising approach to analyse large-scale biological data and to study multi-factorial genetic pathologies through their genetic alterations.