Co-clustering Algorithms Research Articles

T-Distributed Stochastic Neighbor Embedding for Co-Representation Learning

Co-clustering is the simultaneous clustering of the samples and attributes of a data matrix that provides deeper insight into data than traditional clustering. However, there is a lack of representation learning algorithms that serve this mechanism of co-clustering, and the current representation learning algorithms are limited to the sample perspective and lack the use of information in the attribute perspective. To solve this problem, in this article, ctSNE , a co-representation learning model based on t-distributed stochastic neighbor embedding, is proposed for unsupervised co-clustering, where ctSNE makes the dataset representation outputted more discriminative of row and column clusters (i.e. co-discrimination). On the basis of t-distributed stochastic neighbor embedding retaining the sample data distribution and local data structure, the philosophy of collaboration is introduced (i.e., row and column hidden relationship information) so that the ctSNE model is equipped with co-representation learning capability, which can effectively improve the performance of co-clustering. To prove the effectiveness of the ctSNE model, several classic co-clustering algorithms are used to check the co-representation performance of ctSNE, and a novel internal index based on an internal clustering index, known as total inertia, is proposed to demonstrate the effect of co-clustering. The numerous experimental results show that ctSNE has tremendous co-representation capability and can significantly improve the performance of co-clustering algorithms.

Fast parameterless prototype-based co-clustering

Tensor co-clustering algorithms have been proven useful in many application scenarios, such as recommender systems, biological data analysis and the analysis of complex and evolving networks. However, they are significantly affected by wrong parameter configurations, since, at the very least, they require the cluster number to be set for each mode of the matrix/tensor, although they typically have other algorithm-specific hyper-parameters that need to be fine-tuned. Among the few known objective functions that can be optimized without setting these parameters, the Goodman–Kruskal τ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ au $$\\end{document}—a statistical association measure that estimates the strength of the link between two or more discrete random variables—has proven its effectiveness in complex matrix and tensor co-clustering applications. However, its optimization in a co-clustering setting is tricky and, so far, has leaded to very slow and, at least in some specific but not unfrequent cases, inaccurate algorithms, due to its normalization term. In this paper, we investigate some interesting mathematical properties of τ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ au $$\\end{document}, and propose a new simplified objective function with the ability of discovering an arbitrary and a priori unspecified number of good-quality co-clusters. Additionally, the new objective function definition allows for a novel prototype-based optimization strategy that enables the fast execution of matrix and higher-order tensor co-clustering. We show experimentally that the new algorithm preserves or even improves the quality of the discovered co-clusters by outperforming state-of-the-art competing approaches, while reducing the execution time by at least two orders of magnitude.

Orthogonal parametric non-negative matrix tri-factorization with α-divergence for co-clustering

Co-clustering algorithms can seek homogeneous sub-matrices into a dyadic data matrix, such as a document-word matrix. Algorithms for co-clustering can be expressed as a non-negative matrix tri-factorization problem such that X≈FSG⊤, which is associated with the non-negativity conditions on all matrices and the orthogonality of F (row-coefficient) and G (column-coefficient) matrices. Most algorithms are based on Euclidean distance and Kullback–Leibler divergence without parameters to control orthogonality. We propose to apply the orthogonality of parameters by adding two penalty terms based on the α-divergence objective function. Orthogonal parametric non-negative matrix tri-factorization uses orthogonal parameters for row and column space, separately. Finally, we compare the proposed algorithms with other algorithms on six real text datasets.

Robust weighted co-clustering with global and local discrimination

In the past few decades, the clustering problem has made considerable progress, and co-clustering algorithms have attracted more attention. Compared with one-side clustering, co-clustering not only groups samples according to the distribution of features but also groups features according to the distribution of samples at the same time. This duality helps to explore the structural information of data, such as genes and texts. In this paper, a new co-clustering algorithm is proposed to simultaneously consider feature weights, data noise, local manifolds, and global scatter, named robust weighted co-clustering with global and local discrimination. Furthermore, an alternate update rule is put forward to optimize objective, theoretically proven to converge. Then, the algorithm’s duality, robustness, and effectiveness have been verified on synthetic, corrupted, and real datasets, respectively. The runtime and parameter sensitivity of the algorithm are also analyzed. Finally, sufficient experiments clarify the competitiveness of our algorithm compared to other ones.

Learning an Optimal Bipartite Graph for Subspace Clustering via Constrained Laplacian Rank.

In this article, we focus on utilizing the idea of co-clustering algorithms to address the subspace clustering problem. In recent years, co-clustering methods have been developed greatly with many important applications, such as document clustering and gene expression analysis. Different from the traditional graph-based methods, co-clustering can utilize the bipartite graph to extract the duality relationship between samples and features. It means that the bipartite graph can obtain more information than other traditional graph methods. Therefore, we proposed a novel method to handle the subspace clustering problem by combining dictionary learning with a bipartite graph under the constraint of the (normalized) Laplacian rank. Besides, to avoid the effect of redundant information hiding in the data, the original data matrix is not used as the static dictionary in our model. By updating the dictionary matrix under the sparse constraint, we can obtain a better coefficient matrix to construct the bipartite graph. Based on Theorem 2 and Lemma 1, we further speed up our algorithm. Experimental results on both synthetic and benchmark datasets demonstrate the superior effectiveness and stability of our model.

Multi-objective genetic model for co-clustering ensemble

Co-clustering ensemble establishes a consensus co-clustering over the data, and the ensemble process can be described as an optimization problem that can be solved by genetic algorithms. However, co-clustering ensemble methods based on genetic models are very few, in which fuzzy clustering and hard clustering is not combined. In this paper, a multi-objective genetic model for co-clustering ensemble (GMCCE) is proposed, and the corresponding objective function is designed. First, to process fuzzy samples and general samples more appropriately, bilateral fuzzy clustering and hard co-clustering are combined organically. Then, chromosomes are encoded as the membership of rows and columns, and after evolution process, the best chromosome is the consensus result. Finally, the proposed model is used to design a GMCCE algorithm. To evaluate the potential of GMCCE, extensive experiments are carried out, including comparison with base co-clustering algorithms and state-of-the-art algorithms. The results demonstrate that the GMCCE algorithm outperforms other algorithms.

The Exploration of Restaurant Recommender System

The exploitation of Recommender Systems (RS) isstill a challenge, hence it is important to explore the three correlatedattributes, such as restaurant, food, and service ratings. Therefore, thisstudy provides an in-depth review of these attribute ratings using theCollaborative Filtering (CF) technique. Experiments were performed with k-NN,SVD, Slope One, and Co-Clustering algorithms, while RMSE, MSE, MAE, and FCPwere used as evaluation metrics. The results showed that the service restaurantrating predictions produced the best average MSE and RMSE accuracy in 5 and10-fold cross-validation. Furthermore, the best hyperparameter of algorithmsusing Grid Search was achieved in restaurant rating prediction. In conclusion,SVD surpasses other algorithms in MSE and RMSE for all scenarios.

TensorClus: A python library for tensor (Co)-clustering

Tensor data analysis is the evolutionary step of data analysis to more than two dimensions. Dealing with tensor data is often based on tensor decomposition methods. The present paper focuses on unsupervised learning and provides a python package referred to as TensorClus including novel co-clustering algorithms of three-way data. All proposed algorithms are based on the latent block models and suitable to different types of data, sparse or not. They are successfully evaluated on challenges in text mining, recommender systems, and hyperspectral image clustering. TensorClus is an open-source Python package that allows easy interaction with other python packages such as NumPy and TensorFlow; it also offers an interface with some tensor decomposition packages namely Tensorly and TensorD on the one hand, and on the other, the co-clustering package Coclust. Finally, it provides CPU and GPU compatibility. The TensorClus library is available at https://pypi.org/project/TensorClus/.

Feature-reduction fuzzy co-clustering approach for hyper-spectral image analysis

Fuzzy co-clustering algorithms are the effective techniques for multi-dimensional clustering in which all features are considered of equal importance (relevance). In fact, the features’ importance could be different, even several of them could be considered redundant. The removal of the redundant features has formed the idea of feature-reduction in problems of the big data processing. In this paper, we propose a new unsupervised learning scheme by incorporating the feature-weighted entropy into the objective function of fuzzy co-clustering, called the Feature-Reduction Fuzzy Co-Clustering Algorithm (FRFCoC). First, a new objective function is formed on the basis of the original fuzzy co-clustering objective function which adds parameters representing the entropy weight of the different features. Next, a feature-reduction and clustering automatic schema are adjusted based on FCoC’s original learning schema which calculates new parameters and conditions to eliminate irrelevant feature components. FRFCoC algorithm can be mathematically shown to converge after a finite number of iterations. The experiment results were conducted on some many-features data sets and hyperspectral images that have demonstrated the outstanding performance of FRFCoC algorithm compared with some previously proposed algorithms.

Co-clustering algorithms for distributional data with automated variable weighting

This paper is concerned with the co-clustering of distribution-valued data, that is, the simultaneous partitioning of rows and columns of an input data table, the elements of which are distributions (or histograms) representing aggregate data. The first proposed method extends the double k-means algorithm to distributional data. The L2 Wasserstein distance, also known as Mallow’s distance, is used to compare distributions. To consider the different relevance of the variables characterizing the clusters, four variants of adaptive distributional double k-means are proposed. Accordingly, in the co-clustering procedure, an additional step is introduced to compute the relevance weights associated with the variables. In particular, each of the four algorithms provides i) a set of weights for the variables; ii) different sets of weights for the variables, one for each cluster (cluster-wise); iii) a double set of weights for the variables according to the decomposition of the L2 Wasserstein distance into two components; iv) different double sets of weights for the variables and distance components, one for each cluster (cluster-wise). Applications using simulated and real data demonstrate the effectiveness of the proposed algorithms and the contribution of the relevance weights to the co-clustering procedure according to the structure of the data.

A dual reformulation and solution framework for regularized convex clustering problems

Clustering techniques are powerful tools commonly used in statistical learning and data analytics. Most of the past research formulates clustering tasks as a non-convex problem, where a global optimum often cannot be found. Recent studies show that hierarchical clustering and k-means clustering can be relaxed and analyzed as a convex problem. Moreover, sparse convex clustering algorithms are proposed to extend the convex clustering framework to high-dimensional space by introducing an adaptive group-Lasso penalty term. Due to the non-smoothness nature of the associated objective functions, there are still no efficient fast-convergent algorithms for clustering problems even with convexity. In this paper, we first review the structure of convex clustering problems and prove the differentiability of their dual problems. We then show that such reformulated dual problems can be efficiently solved by the accelerated first-order methods with the feasibility projection. Furthermore, we present a general framework for convex clustering with regularization terms and discuss a specific implementation of this framework using L1,1-norm. We also derive the dual form for the regularized convex clustering problems and show that it can be efficiently solved by embedding a projection operator and a proximal operator in the accelerated gradient method. Finally, we compare our approach with several other co-clustering algorithms using a number of example clustering problems. Numerical results show that our models and solution methods outperform all the compared algorithms for both convex clustering and convex co-clustering.

An Overview of Co-Clustering via Matrix Factorization

Co-clustering algorithms have been widely used for text clustering and gene expression through matrix factorization. In recent years, diverse co-clustering algorithms which group data points and features synchronously have shown their advantages over traditional one-side clustering. In order to solve the co-clustering problems, most existing methods relaxed constraints via matrix factorization. In this paper, we provide a detailed understanding of six co-clustering algorithms with different performance and robustness. We conduct comprehensive experiments in eight real-world datasets to compare and evaluate these co-clustering methods based on four evaluation metrics including clustering accuracy, normalized mutual information, adjusted rand index, and purity. Our findings demonstrate the strengths and weaknesses of these methods and provide insights to motivate further exploration of co-clustering methods and matrix factorization.

CoClust: A Python Package for Co-Clustering

Co-clustering (also known as biclustering), is an important extension of cluster analysis since it allows to simultaneously group objects and features in a matrix, resulting in row and column clusters that are both more accurate and easier to interpret. This paper presents the theory underlying several effective diagonal and non-diagonal co-clustering algorithms, and describes CoClust, a package which provides implementations for these algorithms. The quality of the results produced by the implemented algorithms is demonstrated through extensive tests performed on datasets of various size and balance. CoClust has been designed to complete and easily interface with popular Python machine learning libraries such as scikit-learn.

A Fuzzy Co-Clustering Algorithm via Modularity Maximization

In this paper we propose a fuzzy co-clustering algorithm via modularity maximization, named MMFCC. In its objective function, we use the modularity measure as the criterion for co-clustering object-feature matrices. After converting into a constrained optimization problem, it is solved by an iterative alternative optimization procedure via modularity maximization. This algorithm offers some advantages such as directly producing a block diagonal matrix and interpretable description of resulting co-clusters, automatically determining the appropriate number of final co-clusters. The experimental studies on several benchmark datasets demonstrate that this algorithm can yield higher quality co-clusters than such competitors as some fuzzy co-clustering algorithms and crisp block-diagonal co-clustering algorithms, in terms of accuracy.

CCGA: Co-similarity based Co-clustering using genetic algorithm

Co-clustering refers to the simultaneous clustering of objects and their features. It is used as a clustering technique when the data exhibit similarities only in a subset of features instead of the whole feature set. Clustering (and co-clustering) has been proven to be an optimization problem which makes evolutionary algorithms a suitable candidate for optimizing the cluster labels. Genetic algorithms have been used in the literature for data clustering by optimizing cluster labels to reduce mean distance from cluster centers. Using only genetic operators and Euclidean distances, however, have resulted in limited success. In this paper, we propose to use a Genetic Algorithm framework for co-clustering data. What makes this contribution significant and distinctly unique is that we propose the use of a co-similarity objective function that uses multiple objective functions to seamlessly integrate the co-clustering framework into the optimization problem. Co-similarity matrices are intertwined row and column similarity matrices that are computed on the basis of each other. To the best of our knowledge, we are the first to propose the use of Genetic Algorithm to optimize co-similarity matrices for the co-clustering task. We conduct several experiments to analyse the performance of our proposed approach and compare them with numerous state-of-the-art clustering and co-clustering algorithms, on a variety of real world datasets. Our results show that the proposed approach significantly outperforms other clustering and co-clustering algorithms on all the datasets tested.

A hierarchical co-clustering approach for entity exploration over Linked Data

With the increasing amount of Linked Data on the Web, large numbers of linked entities often make it difficult for users to find the entities of interest quickly for further exploration. Clustering as a fundamental approach, has been adopted to organize entities into meaningful groups. In general, link and entity class are semantically labelled and can be used to group linked entities. However, entities are usually associated with many links and classes. To avoid information overload, we propose a novel hierarchical co-clustering approach to simultaneously group links and entity classes. In our approach, we define a measure of intra-link similarity and intra-class similarity respectively, and then incorporate them into co-clustering. Our proposed approach is implemented in a Linked Data browser called CoClus. We compare it with other three browsers by conducting a task-based user study and the experimental results show that our approach provides useful support for entity exploration. We also compare our algorithm with three baseline co-clustering algorithms and the experimental results indicate that it outperforms baselines in terms of the Clustering Index score.

TWCC: Automated Two-way Subspace Weighting Partitional Co-Clustering

A two-way subspace weighting partitional co-clustering method TWCC is proposed. In this method, two types of subspace weights are introduced to simultaneously weight the data in two ways, i.e., columns on row clusters and rows on column clusters. An objective function that uses the two types of weights in the distance function to determine the co-clusters of data is defined, and an iterative TWCC co-clustering algorithm to optimize the objective function is proposed, in which the two types of subspace weights are automatically computed. A series of experiments on both synthetic and real-life data were conducted to investigate the properties of TWCC, compare the two-way clustering results of TWCC with those of eight co-clustering algorithms, and compare one-way clustering results of TWCC with those of six clustering algorithms. The results have shown that TWCC is robust and effective for large high-dimensional data.

Robust co-clustering via dual local learning and high-order matrix factorization

Co-clustering is to group features and samples simultaneously and has received increasing attention in data mining and machine learning, particularly in text document categorization and gene expression. In this paper, two effective co-clustering algorithms are proposed to exploit the joint advantages of local learning and matrix factorization. First, the co-clustering problem is formulated as a form of matrix tri-factorization which embeds local structure learning and orthogonality constraints for clustering indicators. Using high-order matrix factorization, an effective algorithm is proposed for co-clustering problems and its convergence is proved. Second, symmetric co-clustering problems are studied, where the sample affinity matrix serves as the input matrix. Analogous high-order matrix factorization is used to develop an effective convergent algorithm for that problem. Finally, the two proposed algorithms are validated in eight publicly available real-world datasets from machine learning repository. Extensive experiments demonstrate that the proposed algorithms achieve competitive performance over existing state-of-the-art co-clustering methods in all tested datasets.

Model-based co-clustering for the effective handling of sparse data

With the exponential growth of text documents on the web, there is a genuine need for techniques that organize terms and documents, simultaneously, into meaningful clusters, thereby making large datasets easier to handle and interpret. Several block diagonal clustering algorithms have proven successful in identifying co-clusters of documents and words. However, despite their effectiveness, most of the existing methods do not provide a parameterizable model for tackling the problem of block diagonal identification. In this paper, we rely on mixture models, which offer strong theoretical foundations and considerable flexibility. More precisely, we propose a parsimonious latent block model based on the mixture of Poisson distributions and tailored for sparse high dimensional data such as document-term matrices. In order to efficiently estimate the model parameters, we derive two scalable co-clustering algorithms based on variational inference. Empirical results obtained on several real-world text datasets highlight the advantages of the proposed model and the corresponding co-clustering algorithms.

Coclustering of Multidimensional Big Data: A Useful Tool for Genomic, Financial, and Other Data Analysis

The analysis of a multidimensional data array is necessary in many applications. Although a data set can be very large, it is possible that meaningful and coherent patterns embedded in the data array are much smaller in size. For example, in genomic data, we may want to find a subset of genes that coexpress under a subset of conditions. In this article, I will explain coclustering algorithms for solving the coherent pattern-detection problem. In these methods, a coherent pattern corresponds to a low-rank matrix or tensor and can be represented as an intersection of hyperplanes in a high-dimensional space. We can then extract coherent patterns from the large data array by detecting hyperplanes. Examples will be provided to demonstrate the effectiveness of the coclustering algorithms for solving unsupervised pattern classification problems.