Classical Clustering Algorithms Research Articles

A Survey of Parallel Clustering Algorithms Based on Spark

Clustering is one of the most important unsupervised machine learning tasks, which is widely used in information retrieval, social network analysis, image processing, and other fields. With the explosive growth of data, the classical clustering algorithms cannot meet the requirements of clustering for big data. Spark is one of the most popular parallel processing platforms for big data, and many researchers have proposed many parallel clustering algorithms based on Spark. In this paper, the existing parallel clustering algorithms based on Spark are classified and summarized, the parallel design framework of each kind of algorithms is discussed, and after comparing different kinds of algorithms, the direction of the future research is discussed.

RECURSIVE HIERARCHICAL CLUSTERING FOR HYPERSPECTRAL IMAGES

Abstract. Partition based clustering techniques are widely used in data mining and also to analyze hyperspectral images. Unsupervised clustering only depends on data, without any external knowledge. It creates a complete partition of the image with many classes. And so, sparse labeled samples may be used to label each cluster, and so simplify the supervised step. Each clustering algorithm has its own advantages, drawbacks (initialization, training complexity). We propose in this paper to use a recursive hierarchical clustering based on standard clustering strategies such as K-Means or Fuzzy-C-Means. The recursive hierarchical approach reduces the algorithm complexity, in order to process large amount of input pixels, and also to produce a clustering with a high number of clusters. Moreover, in hyperspectral images, a classical question is related to the high dimensionality and also to the distance that shall be used. Classical clustering algorithms usually use the Euclidean distance to compute distance between samples and centroids. We propose to implement the spectral angle distance instead and evaluate its performance. It better fits the pixel spectrums and is less sensitive to illumination change or spectrum variability inside a semantic class. Different scenes are processed with this method in order to demonstrate its potential.

An effective multi-level synchronization clustering method based on a linear weighted Vicsek model

To conquer the shortcoming that general clustering methods cannot process big data in the main memory, this paper presents an effective multi-level synchronization clustering (MLSynC) method by using a framework of “divide and collect” and a linear weighted Vicsek model. We also introduce two concrete implementations of MLSynC method, a two-level framework algorithm and a recursive algorithm. MLSynC method has a different process with SynC algorithm, ESynC algorithm and SSynC algorithm. By the theoretic analysis, we find the time complexity of MLSynC method is less than SSynC. Simulation and experimental study on multi-kinds of data sets validate that MLSynC method not only gets better local synchronization effect but also needs less iterative times and time cost than SynC algorithm. Moreover, we observe that MLSynC method not only needs less time cost than ESynC and SSynC, but also almost gets the same local synchronization effect as ESynC and SSynC if the partition of the data set is proper. Further comparison experiments with some classical clustering algorithms demonstrate the clustering effect of MLSynC method.

Spectral clustering via ensemble deep autoencoder learning (SC-EDAE)

Several works have studied clustering strategies that combine classical clustering algorithms and deep learning methods. These strategies generally improve clustering performance, however deep autoencoder setting issues impede the robustness of these approaches. To alleviate the impact of hyperparameters setting, we propose a model which combines spectral clustering and deep autoencoder strengths in an ensemble framework. Our proposal does not require any pretraining and includes the three following steps: generating various deep embeddings from the original data, constructing a sparse and low-dimensional ensemble affinity matrix based on anchors strategy and applying spectral clustering to obtain the common space shared by multiple deep representations. While the anchors strategy ensures an efficient merging of the encodings, the fusion of various deep representations enables to mitigate the deep networks setting issues. Experiments on various benchmark datasets demonstrate the potential and robustness of our approach compared to state-of-the-art deep clustering methods.

Urban Hotspots Mining Based on Improved FDBSCAN Algorithm

Enormous human activities happen in modern cities nowadays. These activities generate trajectory information, which can play a big role in urban planning, transportation improvement, behavior analysis, and many location-based services. Currently, GPS devices are installed in many cities’ taxies. These positioning devices are the ideal source of trajectory data. Urban hotspots can be extracted by applying spatial clustering algorithm to the taxi trajectory data. Since the classical DBSCAN clustering algorithm is short of execution efficiency and its variant FDBSCAN clustering algorithm fails to make fine-grained clustering on the taxi trajectory data, this paper proposes an improved FDBSCAN clustering algorithm, O-FDBSCAN clustering algorithm. The core of the O-FDBSCAN clustering algorithm is, it adds some coordinate offsets to the original trajectory data point based on some background geographic region knowledge. The background knowledge contains the collected functional domain information in the nearby area. By adding this kind of weight to the trajectory data point, the clustering result will be closer to the real target place and therefore give a more fine-grained clustering result. In this paper, we extract the urban hotspots and their spatiotemporal pattern by using the O-FDBSCAN clustering algorithm. The comparison result shows that, the proposed algorithm has better fine-grained clustering results than FDBSCAN. be taken in constructing both.

A Novel Improved K-Means Algorithm Based on Parameter Adaptive Selection

As a classical clustering algorithm, K-means has been widely applied due to its features of simple mathematical thinking, fast convergence rate, less complexity, and easy to implementation. However, K-means algorithm always requires users to set the desired number of clusters in advance, and the initial cluster centers are usually generated in a random way. When dealing with unknown datasets that users do not have enough domain-assisted knowledge, such parameters setting strategies not only increases the burden on users, but also makes clustering quality difficult to guarantee. Therefore, in view of the high sensitivity of K-means clustering process to initial parameters, this paper propose an improved DDWK-means (Distance-Density-Weight K-means) algorithm. Based on the distance-density feature and the method of inertia weight of particle swarm optimization algorithm, the optimal initial cluster centers not only can be determined adaptively according to the structural characteristics of the dataset itself without introducing artificial parameters, but also can be adjusted dynamically due to the threshold change of clustering quality metric. We make an experimental study with five standard datasets from UCI (University of California Irvine), and the results indicate that the DDWK-means algorithm exhibits a significantly improvement in clustering efficiency and stability.

Design of a reservoir for cloud-enabled echo state network with high clustering coefficient

Reservoir computing (RC) is considered as a suitable alternative for descending gradient methods in recursive neural networks (RNNs) training. The echo state network (ESN) is a platform for RC and nonlinear system simulation in the cloud environment with many external users. In the past researches, the highest eigenvalue of reservoir connection weight (spectral radius) was used to predict reservoir dynamics. Some researchers have illustrated; the characteristics of scale-free and small-world can improve the approximation capability in echo state networks; however, recent studies have shown importance of the infrastructures such as clusters and the stability criteria of these reservoirs as altered. In this research, we suggest a high clustered ESN called HCESN that its internal neurons are interconnected in form of clusters. Each of the clusters contains one backbone and a number of local nodes. We implemented a classical clustering algorithm, called K-means, and three optimization algorithms including genetic algorithm (GA), differential evolution (DE), and particle swarm optimization (PSO) to improve the clustering efficiency of the new reservoir and compared them with each other. For investigating the spectral radius and predictive power of the resulting reservoirs, we also applied them to the laser time series and the Mackey-Glass dynamical system. It is demonstrated that new clustered reservoirs have some specifications of biologic neural systems and complex networks like average short path length, high clustering coefficient, and power-law distribution. The empirical results illustrated that the ESN based on PSO could strikingly enhance echo state property (ESP) and obtains less chaotic time series prediction error compared with other works and the original version of ESN. Therefore, it can approximate nonlinear dynamical systems and predict the chaotic time series.

Structured block diagonal representation for subspace clustering

The aim of the subspace clustering is to segment the high-dimensional data into the corresponding subspace. The structured sparse subspace clustering and the block diagonal representation clustering are quite advanced spectral-type subspace clustering algorithms when handling to the linear subspaces. In this paper, the respective advantages of these two algorithms are fully exploited, and the structured block diagonal representation (SBDR) subspace clustering is proposed. In many classical spectral-type subspace clustering algorithms, the affinity matrix which obeys the block diagonal property can not necessarily bring satisfying clustering results. However, the k-block diagonal regularizer of the SBDR algorithm directly pursues the block diagonal matrix, and this regularizer is obviously more effective. On the other hand, the general procedure of the spectral-type subspace clustering algorithm is to get the affinity matrix firstly and next perform the spectral clustering. The SBDR algorithm considers the intrinsic relationship of the two seemingly separate steps, the subspace structure matrix obtained by the spectral clustering is used iteratively to facilitate a better initialization for the representation matrix. The experimental results on the synthetic dataset and the real dataset have demonstrated the superior performance of the proposed algorithm over other prevalent subspace clustering algorithms.

A Novel Segmentation Method for Brain MRI Using a Block-Based Integrated Fuzzy C-Means Clustering Algorithm

Accurate segmentation of brain tissue has important guiding significance and practical application value for the diagnosis of brain diseases. Brain magnetic resonance imaging (MRI) has the characteristics of high dimensionality and large sample size. Such datasets create considerable computational complexity in image processing. To efficiently process large sample data, this article integrates the proposed block clustering strategy with the classic fuzzy C-means clustering (FCM) algorithm and proposes a block-based integrated FCM clustering algorithm (BI-FCM). The algorithm first performs block processing on each image and then clusters each subimage using the FCM algorithm. The cluster centers for all subimages are again clustered using FCM to obtain the final cluster center. Finally, the distance from each pixel to the final cluster center is obtained, and the corresponding division is performed according to the distance. The dataset used in this experiment is the Simulated Brain Database (SBD). The results show that the BI-FCM algorithm addresses the large sample processing problem well, and the theory is simple and effective.

An investigation on support vector clustering for big data in quantum paradigm

The support vector clustering algorithm is a well-known clustering algorithm based on support vector machines using Gaussian or polynomial kernels. The classical support vector clustering algorithm works well in general, but its performance degrades when applied on big data. In this paper, we have investigated the performance of support vector clustering algorithm implemented in a quantum paradigm for possible run-time improvements. We have developed and analyzed a quantum version of the support vector clustering algorithm. The proposed approach is based on the quantum support vector machine and quantum kernels (i.e., Gaussian and polynomial). The proposed quantum version of the SVM clustering method demonstrates a significant speed-up gain on the overall run-time complexity as compared to the classical counterpart.

A novel mixing matrix estimation algorithm in instantaneous underdetermined blind source separation

Due to the lack of sufficient prior information, how to estimate the mixing matrix in multiple underdetermined blind source separation (UBSS) models is a difficult problem. This study proposes an algorithm, which is used for the estimation of mixing matrix in instantaneous UBSS. Firstly, we propose an efficient single-source-points detection criterion with the transformation for the mixed signal vector, which is used as the basis for the clustering process. There are some shortcomings in the classical clustering algorithms, including the dependence on input parameters, restrictions on the data dimension, the requirement of the prior knowledge of the source signals and high complexity. To overcome these drawbacks, the modified density peaks clustering algorithm is used for the estimation of the initial clustering centers to adapt to different circumstances. Based on the idea of mean clustering, the single source points near each initial cluster center are processed, respectively, and the final estimation results of the mixing matrix are obtained. A variety of simulation experiments demonstrate the universality and validity of the proposed algorithm. The proposed method also has excellent performance even under the circumstance of low signal-to-noise ratio.

Human Activity Recognition Based on Parallel Approximation Kernel K-Means Algorithm

Recently, owing to the capability of mobile and wearable devices to sense daily human activity, human activity recognition (HAR) datasets have become a large-scale data resource. Due to the heterogeneity and nonlinearly separable nature of the data recorded by these sensors, the datasets generated require special techniques to accurately predict human activity and mitigate the considerable heterogeneity. Consequently, classic clustering algorithms do not work well with these data. Hence, kernelization, which converts the data into a new feature vector representation, is performed on nonlinearly separable data. This study aims to present a robust method to perform HAR data clustering to mitigate heterogeneity in data with minimal resource consumption. Therefore, we propose a parallel approximated clustering approach to handle the computational cost of big data by addressing noise, heterogeneity, and nonlinearity in data using data reduction, filtering, and approximated clustering methods on parallel computing environments that have not been previously addressed. Our key contribution is to treat HAR as big data implemented by approximation kernel K-means approaches and fill the gap between the HAR clustering cost and parallel computing fields. We implemented our approach on Google cloud on a parallel spark cluster, which helped us to process large-scale HAR data across multiple machines of clusters. The normalized mutual information is used as validation metric to assess the quality of the clustering algorithm. Additionally, the precision, recall, f-score metrics values are obtained somehow to compare the results with a classification technique. The experimental results of our clustering approach prove its effectiveness compared with a classification technique and can efficiently detect physical activity and mitigate the heterogeneity of the datasets.

MR-BIRCH: A scalable MapReduce-based BIRCH clustering algorithm

Many classical clustering algorithms have been fitted into MapReduce, which provides a novel solution for clustering big data. However, several iterations are required to reach an acceptable result in most of the algorithms. For each iteration, a new MapReduce job must be executed to load the dataset into main memory, which results in high I/O overhead and poor efficiency. BIRCH algorithm stores only the statistical information of objects with CF entries and CF tree to cluster big data, but with the increase of the tree nodes, the main memory will be insufficient to contain more objects. Hence, BIRCH has to reduce the tree, which will degrade the clustering quality and decelerate the whole execution efficiency. To deal with the problem, BIRCH was fitted into MapReduce called MR-BIRCH in this paper. In contrast to a great number of MapReduce-based algorithms, MR-BIRCH loads dataset only once, and the dataset is processed parallel in several machines. The complexity and scalability were analyzed to evaluate the quality of MR-BIRCH, and MR-BIRCH was compared with Python sklearn BIRCH and Apache Mahout k-means on real-world and synthetic datasets. Experimental results show, most of the time, MR-BIRCH was better or equal to sklearn BIRCH, and it was competitive to Mahout k-means.

Improved K-means Algorithm Based on Threshold Value Radius

K-means algorithm is a classical clustering algorithm, which needs to specify K value artificially and chooses the initial clustering center randomly. However, it is easy to fall into local optimal solution. In order to overcome the shortcomings of K-means algorithm, an improved algorithm based on threshold value radius is proposed in this paper. The initial clustering center and threshold radius are determined automatically by the average value of Euclidean distance between data. The experimental results show that the method proposed can effectively and accurately overcome the shortcomings of the traditional K-means algorithm.

An Unequal Clustering Algorithm for Wireless Sensor Networks Based on Interval Type-2 TSK Fuzzy Logic Theory

Clustering is an effective method for reducing energy consumption in wireless sensor networks (WSNs). In a multihop clustered network scenario, each sensor node transmits data to its own cluster head (CH), and the CH aggregates the data from its member nodes and forwards it to the base station (BS) via other CHs. However, the “hot spot” problem is prone to occur in clustered WSNs because CHs closer to the BS have heavier intercluster forwarding tasks. To address this problem, this paper proposes an unequal clustering algorithm based on interval type-2 TSK fuzzy logic theory (UCT2TSK). The relative distance to the BS (RDB), residual energy (RE), and node density (ND) are considered as the inputs of an interval type-2 fuzzy logic system (FLS). Through fuzzy reasoning, outputs are acquired that can be used to optimize the CHs and determine the cluster sizes. Simulation results verify that UCT2TSK can effectively balance energy consumption and enhance energy efficiency because it has better performance in network lifetime and network throughput than other classical and recent clustering algorithms.

Overview of clustering analysis algorithms in unknown protocol recognition

In the process of identifying unknown protocol of bit stream, the clustering of data sets of bit stream in the protocol is the basis of further identifying unknown protocol. Therefore, on the one hand, this paper analyses the classical clustering algorithms used in unknown protocol recognition from three perspectives: the whole process of clustering analysis, similarity measurement and clustering result evaluation. On the other hand, the development trend of clustering algorithm in unknown protocol recognition is summarized, and other problems in unknown protocol recognition can be solved by clustering algorithm according to the characteristics of bit stream data set, which can provide reference for future research work. Finally, the challenges faced by the Research Institute and the prospects for future work are given.

Fast Incremental Spectral Clustering in Titanate Application via Graph Fourier Transform

In numerous applications the rapid increase of data size with time makes classical clustering algorithms too slow because of the high computational cost. In the present contribution, a novel method for online spectral clustering algorithm is introduced, which can be applied for simulation of structure and properties of titanate in chemical engineering. The proposed algorithm uses the recent results in the emerging field of graph signal processing. It avoids the costly computation of the eigenvectors by filtering random signals on the graph, and only a few derivative operations are needed to update the clustering result when a new data arrives. Initial simulations are presented using both simulated and real data sets, illustrating the relevance of the proposed algorithm.

A novel Chinese herbal medicine clustering algorithm via artificial bee colony optimization.

Traditional Chinese medicine (TCM) has become popular and been viewed as an effective clinical treatment across the world. Accordingly, there is an ever-increasing interest in performing data analysis over TCM data. Aiming to cope with the problem of excessively depending on empirical values when selecting cluster centers by traditional clustering algorithms, an improved artificial bee colony algorithm is proposed by which to automatically select cluster centers and apply it to aggregate Chinese herbal medicines. The proposed method integrates the following new techniques: (1) improving the artificial bee colony algorithm by applying a new searching strategy of neighbour nectar, (2) employing the improved artificial bee colony algorithm to optimize the parameters of the cutoff distance dc, the local density ρi and the minimum distance δi between the element i and any other element with higher density in the cluster algorithm by fast search and finding of density peaks (called DP algorithm) to find the optimal cluster centers, in order to clustering herbal medicines in an accurate fashion with the guarantee of runtime performance. Extensive experiments were conducted on the UCI benchmark datasets and the TCM datasets and the results verify the effectiveness of the proposed method by comparing it with classical clustering algorithms including K-means, K-mediods and DBSCAN in multiple evaluation metrics, that is, Silhouette Coefficient, Entropy, Purity, Precision, Recall and F1-Measure. The results show that the IABC-DP algorithm outperforms other approaches with good clustering quality and accuracy on the UCI and the TCM datasets as well. In addition, it can be found that the improved artificial bee colony algorithm can effectively reduce the number of iterations when compared to the traditional bee colony algorithm. In particular, the IABC-DP algorithm is applied to cluster multi-dimensional Chinese herbal medicines and the result shows that it outperforms other clustering algorithms in clustering Chinese herbal medicines, which can contribute to a larger effort targeted at advancing the study of discovering composition rules of traditional Chinese prescriptions.

A Novel Breast Cancer Detection Technology Using an Advanced Transfer Maximal Entropy Clustering Algorithm

The initial diagnosis of breast cancer involves analyzing the relevant examination report of the patient to determine whether the tumor is benign or malignant. Unsupervised clustering algorithms can be used with this type of problem. In a cluster analysis of a patient's examination data, the clustering results and the preliminary diagnosis results are obtained. However, due to the high cost of detection, medical datasets often have a small sample size or lack information. The traditional clustering technique usually has poor clustering effects in such scenarios. To solve this problem, this paper proposes an advanced transfer learning mechanism based on the classic maximum entropy clustering algorithm and proposes an advanced transfer maximal entropy clustering (AT-MEC) algorithm. A simulation experiment using the Wisconsin Breast Cancer Dataset is performed. This paper verifies that the proposed AT-MEC algorithm has a better clustering effect than other clustering algorithms in the Wisconsin Breast Cancer Dataset.

A review and proposal of (fuzzy) clustering for nonlinearly separable data

In many practical situations data may be characterized by nonlinearly separable clusters. Classical (hard or fuzzy) clustering algorithms produce a partition of objects by computing the Euclidean distance. As such, they are based on the linearity assumption and, therefore, do not identify properly clusters characterized by nonlinear structures. To overcome this limitation, several approaches can be followed: density-, kernel-, graph- or manifold-based clustering. A review of these approaches is offered and some new fuzzy manifold-based clustering algorithms, involving the so-called geodesic distance, are proposed. The effectiveness of such algorithms is shown by synthetic, benchmark and real data.