Parallel Hierarchical Clustering Research Articles

Big data analysis using a parallel ensemble clustering architecture and an unsupervised feature selection approach

Ensemble clustering is known as a challenging research direction in data mining. The results of several individual clustering methods are combined to produce higher quality final clusters. This study introduces a parallel hierarchical clustering approach based on the divide-and-conquer strategy, which is an attempt to realize faster and more efficient ensemble clustering. Here, we propose a cluster consensus selection approach that selects a subset of meriting primary clusters to participate in the final consensus. Considering the sample-cluster and cluster–cluster similarity on the selected primary clusters, we form the final clusters based on the clusters clustering technique as a consensus function. In addition, the proposed scheme is equipped with an unsupervised feature selection approach to remove features that do not contribute significantly to clustering. Extensive evaluations have been performed on datasets of different dimensions from the University of California Irvine (UCI) machine learning repository. The simulation results guarantee the efficiency of the proposed scheme and improves the average performance between 6% and 24% compared to the state-of-the-art clustering methods.

ParChain

This paper studies the hierarchical clustering problem, where the goal is to produce a dendrogram that represents clusters at varying scales of a data set. We propose the ParChain framework for designing parallel hierarchical agglomerative clustering (HAC) algorithms, and using the framework we obtain novel parallel algorithms for the complete linkage, average linkage, and Ward's linkage criteria. Compared to most previous parallel HAC algorithms, which require quadratic memory, our new algorithms require only linear memory, and are scalable to large data sets. ParChain is based on our parallelization of the nearest-neighbor chain algorithm, and enables multiple clusters to be merged on every round. We introduce two key optimizations that are critical for efficiency: a range query optimization that reduces the number of distance computations required when finding nearest neighbors of clusters, and a caching optimization that stores a subset of previously computed distances, which are likely to be reused. Experimentally, we show that our highly-optimized implementations using 48 cores with two-way hyper-threading achieve 5.8--110.1x speedup over state-of-the-art parallel HAC algorithms and achieve 13.75--54.23x self-relative speedup. Compared to state-of-the-art algorithms, our algorithms require up to 237.3x less space. Our algorithms are able to scale to data set sizes with tens of millions of points, which existing algorithms are not able to handle.

ESPRIT-Forest: Parallel clustering of massive amplicon sequence data in subquadratic time.

The rapid development of sequencing technology has led to an explosive accumulation of genomic sequence data. Clustering is often the first step to perform in sequence analysis, and hierarchical clustering is one of the most commonly used approaches for this purpose. However, it is currently computationally expensive to perform hierarchical clustering of extremely large sequence datasets due to its quadratic time and space complexities. In this paper we developed a new algorithm called ESPRIT-Forest for parallel hierarchical clustering of sequences. The algorithm achieves subquadratic time and space complexity and maintains a high clustering accuracy comparable to the standard method. The basic idea is to organize sequences into a pseudo-metric based partitioning tree for sub-linear time searching of nearest neighbors, and then use a new multiple-pair merging criterion to construct clusters in parallel using multiple threads. The new algorithm was tested on the human microbiome project (HMP) dataset, currently one of the largest published microbial 16S rRNA sequence dataset. Our experiment demonstrated that with the power of parallel computing it is now compu- tationally feasible to perform hierarchical clustering analysis of tens of millions of sequences. The software is available at http://www.acsu.buffalo.edu/∼yijunsun/lab/ESPRIT-Forest.html.

Shortest-linkage-based parallel hierarchical clustering on main-belt moving objects of the solar system

Data clustering is an important data preparation process in many scientific analysis researches. In astronomy, although the distributed environments and modern observation techniques enable users to collect and access huge amounts of data, the corresponding clustering process may become very costly. One of the challenges is that the sequential clustering algorithms, that can be applied to cluster hundreds of thousand main-belt asteroids to reason about the origins of the main-belt asteroids, may not be used in the distributed environment directly. Therefore, this study focuses on the problem of parallelizing the traditional hierarchical agglomerative clustering algorithm using shortest-linkage. We propose a new parallel hierarchical agglomerative clustering algorithm based on the master–worker model. The master process divides the whole computation into several small tasks, and distributes the tasks to the worker processes for parallel processing. Then, the master process merges the results from the worker processes to form a hierarchical data structure. The proposed algorithm uses a pruning threshold to reduce the execution time and the storage requirement during the computation. It also supports fast incremental update that merges new data items into a constructed hierarchical tree in seconds, given a tree of about 550,000 data items. To evaluate the performance of our algorithm, this study has conducted several experiments using the MPCORB dataset and a dataset from the DVO database. The results confirm the efficiency of our proposed methodology. Compared with prior similar studies, the proposed algorithm is more flexible and practical in the problem of distributed hierarchical agglomerative clustering.

Reverse Engineering and Analysis of Genome-Wide Gene Regulatory Networks from Gene Expression Profiles Using High-Performance Computing

Regulation of gene expression is a carefully regulated phenomenon in the cell. “Reverse-engineering” algorithms try to reconstruct the regulatory interactions among genes from genome-scale measurements of gene expression profiles (microarrays). Mammalian cells express tens of thousands of genes; hence, hundreds of gene expression profiles are necessary in order to have acceptable statistical evidence of interactions between genes. As the number of profiles to be analyzed increases, so do computational costs and memory requirements. In this work, we designed and developed a parallel computing algorithm to reverse-engineer genome-scale gene regulatory networks from thousands of gene expression profiles. The algorithm is based on computing pairwise Mutual Information between each gene-pair. We successfully tested it to reverse engineer the Mus Musculus (mouse) gene regulatory network in liver from gene expression profiles collected from a public repository. A parallel hierarchical clustering algorithm was implemented to discover “communities” within the gene network. Network communities are enriched for genes involved in the same biological functions. The inferred network was used to identify two mitochondrial proteins.

STUDY OF HIERARCHICAL CLUSTERING PARALLEL COMPUTATION ON PRAM MODEL

An adaptive parallel algorithm for hierarchical clustering based on PRAM model was presented. The following approaches were devised to produce the optimized clustered data set, including the data preprocessing based on "90-10" rule to decrease the size of the data set, progressively the parallel algorithm to create Euclid minimum spanning trees on absolute graph, and the algorithm that determined the split strategies and dealt with the memory conflicts. The data set was clustered based on the noncollision memory, the lowest cost, and weakest PRAM-EREW model. N data sets were clustered in O((λn)2/p) time (0.1 ≤ λ ≤ 0.3) by performing this algorithm using p processors (1 ≤ p ≤ n/ log (n)). The parallel hierarchical clustering algorithm based on PRAM model was adaptive, and of noncollision memory. The computing time could be significantly reduced after original inputting data was effectually preprocessed through the improved preprocessing methods presented in this paper.

In this issue

In this issue

Exploiting parallelism to support scalable hierarchical clustering

AbstractA distributed memory parallel version of the group average hierarchical agglomerative clustering algorithm is proposed to enable scaling the document clustering problem to large collections. Using standard message passing operations reduces interprocess communication while maintaining efficient load balancing. In a series of experiments using a subset of a standard Text REtrieval Conference (TREC) test collection, our parallel hierarchical clustering algorithm is shown to be scalable in terms of processors efficiently used and the collection size. Results show that our algorithm performs close to the expected O(n2/p) time on p processors rather than the worst‐case O(n3/p) time. Furthermore, the O(n2/p) memory complexity per node allows larger collections to be clustered as the number of nodes increases. While partitioning algorithms such as k‐means are trivially parallelizable, our results confirm those of other studies which showed that hierarchical algorithms produce significantly tighter clusters in the document clustering task. Finally, we show how our parallel hierarchical agglomerative clustering algorithm can be used as the clustering subroutine for a parallel version of the buckshot algorithm to cluster the complete TREC collection at near theoretical runtime expectations.

A parallel hierarchical clustering algorithm for PCs cluster system

The efficiency of clustering algorithms is strongly needed with very large databases and high-dimensional data types. As a solution, parallel algorithms can be used to provide powerful computing ability. PCs cluster system is one of low-cost general-purpose parallel computing systems. In this paper, we first theoretically analyze the idea of adopting data parallelism when designing a parallel clustering algorithm for PCs cluster systems, including analysis of speedup and selection of communication schemes. We then present a parallel hierarchical clustering algorithm called PARC. Experiment results demonstrate the correctness of the theoretical analysis and show that in general, PARC obtains as good quality of clustering as linear clustering algorithms, while communication time is considerably improved.

Efficient Parallel Algorithms for Hierarchical Clustering on Arrays with Reconfigurable Optical Buses

Clustering is a basic operation in image processing and computer vision, and it plays an important role in unsupervised pattern recognition and image segmentation. While there are many methods for clustering, the single-link hierarchical clustering is one of the most popular techniques. In this paper, with the advantages of both optical transmission and electronic computation, we design efficient parallel hierarchical clustering algorithms on the arrays with reconfigurable optical buses (AROB). We first design three efficient basic operations which include the matrix multiplication of two N×N matrices, finding the minimum spanning tree of a graph with N vertices, and identifying the connected component containing a specified vertex. Based on these three data operations, an O(logN) time parallel hierarchical clustering algorithm is proposed using N3 processors. Furthermore, if the connectivity of the AROB with four-port connection is allowed, two constant time clustering algorithms can be also derived using N4 and N3 processors, respectively. These results improve on previously known algorithms developed on various parallel computational models.

Parallel hierarchical clustering algorithms on processor arrays with a reconfigurable bus system

Clustering techniques are usually used in pattern recognition, image segmentation and object detection. Let N be the number of patterns and M be the number of features of each pattern and N _ ̆ M . In this paper, we first design two O(1) time basic operations for concentrating all nonempty data of size N and computing the proximity matrix using N × N and N × N × M processors, respectively. Then, based on these two operations, a constant time parallel hierarchical clustering algorithm is proposed on a 3-D processor array with reconfigurable bus system using N 4 processors. Then, by reducing the number of processors by a factor of N, an O(log 2 N) time algorithm for this problem is also derived. Note that no one had ever obtained a constant time algorithm for this problem on the existing parallel computation models.