Classical Clustering Algorithms Research Articles

Fuzzy Clustering in Cascaded Feature Space

The success of fuzzy clustering heavily relies on the proper feature space constructed by the input data. For nonspherical and overlapped clusters, kernel fuzzy clustering is more effective because it finds more proper feature space compared to conventional fuzzy clustering. Unfortunately, poor scalability of kernel fuzzy clustering is induced by the construction of a kernel matrix. To solve the problem, random feature-based method was presented to approximate the kernel function. More interestingly, these features exposed in the approximate feature space are directly manipulable. Inspired by the architecture of functional-link neural network, to exploit more information from both the original data space and the approximate kernel space, a new feature space called cascaded feature space (CF) is constructed in this paper. By performing classical fuzzy c-means (FCM) in CF space, a new fuzzy clustering framework called FCM-CF is developed. To reduce computational complexity and perform FCM in CF space well, dimension reduction methods are adopted to generate two variants of FCM-CF. The experiment results of our proposed algorithms verify their superiority in comparison of other classical fuzzy clustering algorithms.

Weighted adjacent matrix for K-means clustering

K-means clustering is one of the most popular clustering algorithms and has been embedded in other clustering algorithms, e.g. the last step of spectral clustering. In this paper, we propose two techniques to improve previous k-means clustering algorithm by designing two different adjacent matrices. Extensive experiments on public UCI datasets showed the clustering results of our proposed algorithms significantly outperform three classical clustering algorithms in terms of different evaluation metrics.

An intelligent clustering algorithm for high-dimensional multiview data in big data applications

There are many high-dimensional multiview data in various big data applications. It is very difficult to deal with those high-dimensional multiview data for the classic clustering algorithms, which consider all features of data with equal relevance. To tackle this challenging problem, this paper aims at proposing a novel intelligent weighting k-means clustering (IWKM) algorithm based on swarm intelligence. Firstly, the degree of coupling between clusters is presented in the model of clustering to enlarge the dissimilarity of clusters. Various weights of views and features are used in the weighting distance function to determine the clusters of objects. Secondly, to eliminate the sensitivity of initial cluster centers, swarm intelligence is utilized to find initial cluster centers, weights of views, and weights of features by a global search. Lastly, a precise perturbation is proposed to improve optimization performance of swarm intelligence. To verify the performance of clustering for high-dimensional multiview data, the experiments were performed by the evaluation metrics of Rand Index, Jaccard Coefficient and Folkes Russe in five big data applications on the two different computational platforms of apache spark and single node. The experimental results show that IWKM is effective and efficient in clustering of high-dimensional multiview data, and can obtain better performance than the other 5 kinds of approaches in these complicated data sets with more views and higher dimensions on apache spark and single node.

Nearest-Neighbour-Induced Isolation Similarity and Its Impact on Density-Based Clustering

A recent proposal of data dependent similarity called Isolation Kernel/Similarity has enabled SVM to produce better classification accuracy. We identify shortcomings of using a tree method to implement Isolation Similarity; and propose a nearest neighbour method instead. We formally prove the characteristic of Isolation Similarity with the use of the proposed method. The impact of Isolation Similarity on densitybased clustering is studied here. We show for the first time that the clustering performance of the classic density-based clustering algorithm DBSCAN can be significantly uplifted to surpass that of the recent density-peak clustering algorithm DP. This is achieved by simply replacing the distance measure with the proposed nearest-neighbour-induced Isolation Similarity in DBSCAN, leaving the rest of the procedure unchanged. A new type of clusters called mass-connected clusters is formally defined. We show that DBSCAN, which detects density-connected clusters, becomes one which detects mass-connected clusters, when the distance measure is replaced with the proposed similarity. We also provide the condition under which mass-connected clusters can be detected, while density-connected clusters cannot.

OPEC: Daily Load Data Analysis Based on Optimized Evolutionary Clustering

Customers’ electricity consumption behavior can be studied from daily load data. Studying the daily load data for user behavior pattern analysis is an emerging research area in smart grid. Traditionally, the daily load data can be clustered into different clusters, to reveal the different categories of consumption. However, as user’s electricity consumption behavior changes over time, classical clustering algorithms are not suitable for tracing the changes, as they rebuild the clusters when clustering at any timestamp but never consider the relationship with the clusters in the previous state. To understand the changes of consumption behavior, we proposed an optimized evolutionary clustering (OPEC) algorithm, which optimized the existing evolutionary clustering algorithm by joining the Proper Restart (PR) Framework. OPEC relied on the basic fact that user’s energy consumption behavior would not abruptly change significantly, so the clusters would change progressively and remain similar in adjacent periods, except for an emergency. The newly added PR framework can deal with a situation where data changes dramatically in a short period of time, and where the former frameworks of evolutionary clustering do not work well. We evaluated the OPEC based on daily load data from Shanghai, China and the power load diagram data from UCI machine learning repository. We also carefully discussed the adjustment of the parameter in the optimized algorithm and gave an optimal value for reference. OPEC can be implemented to adapt to this situation and improve clustering quality. By understanding the changes of the users’ power consumption modes, we can detect abnormal power consumption behaviors, and also analyze the changing trend to improve the operations of the power system. This is significant for the regulation of peak load in the power grid. In addition, it can bring certain economic benefits to the operation of the power grid.

Classification of postprandial glycemic patterns in type 1 diabetes subjects under closed-loop control: an in silico study.

In this contribution we explore some alternatives in order to obtain filtered and low dimension CGM data to provide well processed CGM data to AP systems. The presented approach explores the possible association of certain patient behaviors with certain glucose patterns. We compare the classical clustering algorithms (K-means, and fuzzy C-means), which has shown some limitations for CGM data processing, with a new clustering algorithm (K-means ellipsoid algorithm) more suited to CGM data. We test this new algorithm in a variety of complex scenarios including variabilty in the amount of ingested carbohydrates, absorption time and intrapatient parameters. The new algorithm overcomes the perceived problems and is able to discriminate between normoglycaemic, moderate and severe hyperglycaemic post-prandial behaviour, even with similar amounts of carbohydrates contained in a meal.

A Genetic XK-Means Algorithm with Empty Cluster Reassignment

K-Means is a well known and widely used classical clustering algorithm. It is easy to fall into local optimum and it is sensitive to the initial choice of cluster centers. XK-Means (eXploratory K-Means) has been introduced in the literature by adding an exploratory disturbance onto the vector of cluster centers, so as to jump out of the local optimum and reduce the sensitivity to the initial centers. However, empty clusters may appear during the iteration of XK-Means, causing damage to the efficiency of the algorithm. The aim of this paper is to introduce an empty-cluster-reassignment technique and use it to modify XK-Means, resulting in an EXK-Means clustering algorithm. Furthermore, we combine the EXK-Means with genetic mechanism to form a genetic XK-Means algorithm with empty-cluster-reassignment, referred to as GEXK-Means clustering algorithm. The convergence of GEXK-Means to the global optimum is theoretically proved. Numerical experiments on a few real world clustering problems are carried out, showing the advantage of EXK-Means over XK-Means, and the advantage of GEXK-Means over EXK-Means, XK-Means, K-Means and GXK-Means (genetic XK-Means).

Hybridization of Chaos and Flower Pollination Algorithm over K-Means for data clustering

Classical clustering algorithms like K-means often converge to local optima and have slow convergence rates for larger datasets. To overcome such situations in clustering, swarm based algorithms have been proposed. Swarm based approaches attempt to achieve the optimal solution for such problems in reasonable time. Many swarm based algorithms such as Flower Pollination Algorithm (FPA), Cuckoo Search Algorithm (CSA), Black Hole Algorithm (BHA), Bat Algorithm (BA) Particle Swarm Optimization (PSO), Firefly Algorithm (FFA), Artificial Bee Colony (ABC) etc have been successfully applied to many non-linear optimization problems. In this paper, an algorithm is proposed which hybridizes Chaos Optimization and Flower Pollination over K-means to improve the efficiency of minimizing the cluster integrity. The proposed algorithm referred as Chaotic FPA (CFPA) is compared with FPA, CSA, BHA, BA, FFA, and PSO over K-Means for data clustering problem. Experiments are conducted on sixteen benchmark datasets. Algorithms are compared on four different performance parameters — cluster integrity, execution time, number of iterations to converge (NIC) and stability. Results obtained are analyzed statistically using Non-parametric Friedman test. If Friedman test rejects the Null hypothesis then pair wise comparison is done using Nemenyi test. Experimental Result demonstrates the following: (a) CFPA and BHA have better performance on the basis of cluster integrity as compared to other algorithms; (b) Prove the superiority of CFPA and CSA over others on the basis of execution time; (c) CFPA and FPA converges earlier than other algorithms to evaluate optimal cluster integrity; (d) CFPA and BHA produce more stable results than other algorithms.

PSO+K-means Algorithm for Anomaly Detection in Big Data

The use of clustering methods in anomaly detection is considered as an effective approach. The choice of the cluster primary center and the finding of local optimum in the well-known k-means and other classic clustering algorithms are considered as one of the major problems and do not allow to get accurate results in anomaly detection. In this paper to improve the accuracy of anomaly detection based on the combination of PSO (particle swarm optimization) and k-means algorithms, the new weighted clustering method is proposed. The proposed method is tested on Yahoo! S5 dataset and a comparative analysis of the obtained results with the k-means algorithm is performed. The results of experiments show that compared to the k-means algorithm the proposed method is more robust and allows to get more accurate results.

NS-DBSCAN: A Density-Based Clustering Algorithm in Network Space

Spatial clustering analysis is an important spatial data mining technique. It divides objects into clusters according to their similarities in both location and attribute aspects. It plays an essential role in density distribution identification, hot-spot detection, and trend discovery. Spatial clustering algorithms in the Euclidean space are relatively mature, while those in the network space are less well researched. This study aimed to present a well-known clustering algorithm, named density-based spatial clustering of applications with noise (DBSCAN), to network space and proposed a new clustering algorithm named network space DBSCAN (NS-DBSCAN). Basically, the NS-DBSCAN algorithm used a strategy similar to the DBSCAN algorithm. Furthermore, it provided a new technique for visualizing the density distribution and indicating the intrinsic clustering structure. Tested by the points of interest (POI) in Hanyang district, Wuhan, China, the NS-DBSCAN algorithm was able to accurately detect the high-density regions. The NS-DBSCAN algorithm was compared with the classical hierarchical clustering algorithm and the recently proposed density-based clustering algorithm with network-constraint Delaunay triangulation (NC_DT) in terms of their effectiveness. The hierarchical clustering algorithm was effective only when the cluster number was well specified, otherwise it might separate a natural cluster into several parts. The NC_DT method excessively gathered most objects into a huge cluster. Quantitative evaluation using four indicators, including the silhouette, the R-squared index, the Davis–Bouldin index, and the clustering scheme quality index, indicated that the NS-DBSCAN algorithm was superior to the hierarchical clustering and NC_DT algorithms.

An Interpretable Fuzzy DBN-Based Classifier for Indoor User Movement Prediction in Ambient Assisted Living Applications

In this paper, an interpretable fuzzy deep belief network (DBN)-based classifier called deep belief networks-based Takagi-Sugeno-Kang fuzzy classifier (DBN-TSK-FC) is created for indoor user movement prediction in ambient assisted living applications. With its promising classification performance, DBN-TSK-FC features sharing both the powerful neural representation ability of a DBN and the strong uncertainty-handling capability of an interpretable fuzzy representation. On the one hand, DBN-TSK-FC builds its interpretable fuzzy representation in a hierarchical way by applying the classical fuzzy clustering algorithm FCM to obtain fuzzy partitions on the training dataset. Then, it forms interpretable antecedent parts of fuzzy rules as the corresponding fuzzy representation. On the other hand, DBN-TSK-FC builds its DBN-based neural representation in the other hierarchical way. That is, it applies the existing unsupervised DBN pretraining on the training dataset and then takes the neural representation of all the hidden nodes in the top layer of the corresponding DBN as the set of consequent variables of fuzzy rules. In this approach, both the interpretable fuzzy representation and the DBN-based neural representation are further fused to form the corresponding fuzzy rules quickly by using the least learning machine (LLM) on both the fuzzy rules and the labeling information of the original dataset. Therefore, DBN-TSK-FC is essentially a deep TSK fuzzy classifier from the perspective of fuzzy rules, and it indeed avoids the very slow fine-tuning training required after the unsupervised pretraining of the existing DBN learning. The experimental results on the MovementAAL_RSS dataset indicate the effectiveness of the proposed classifier DBN-TSK-FC.

Clustering Model for Microblogging Sites using Dimension Reduction Techniques

In recent times, microblogging sites such as Twitter have become popular communication platforms for exchanging information. From the point of view of individual user, a reasonably active Twitter user can easily get hundreds of microblogs (tweets) in his/her timeline every day. In addition, a large number of the tweets contain fundamentally the same information, because of retweeting and re-posting. These huge amounts of repetitive data may cause data over-burden for the users, and no user can effectively process so much data. In this situation, methodologies to manage the data over-burden should be developed. One of the effective methods for managing the data over-burden on Twitter is to cluster semantically similar tweets into groups, with the goal that a user may see just a couple of tweets in each group. In this work, various graph clustering approaches based on dimension reduction are proposed for clustering microblogs. Through experiments on several microblog datasets, the authors demonstrate that the proposed techniques perform better than several classical text clustering algorithms.

Local community detection for multi-layer mobile network based on the trust relation

With the fast development of mobile Internet, people’s social exchange media has transformed from the traditional social network to mobile network. With the explosion of massive information, it has become an interesting topic to detect network user groups with close correlation in the mobile social network. These groups are hidden in the continuously changing relations of social network, and it is very difficult to obtain the information of entire social network. In addition, these social relations are intertwined and complicated under the influence of various networks, and as a result, researches on single-layer network are simple and incomplete. Therefore, this paper proposed a local community detection algorithm for multi-layer complicated network based on the trust relation (MTLCD) to constrain the node tensor. We compared the performance of our algorithm with other classic network clustering algorithms such as GL, LART and PMM in four actual multi-layer network datasets of Bio GRID, Remote sensing, Twitter and Mobile QQ Zone, and the multi-layer modularity was used as the measurement index to evaluate the algorithm performance. The experimental results and analysis prove that: in the MTLCD algorithm, the core node obtained based on the trust relation can better identify the local community in dataset with trust relation. In addition, we also found that this algorithm had higher accuracy and stability, and it can accurately reflect the local community structure which the core node belongs to.

Good Clusterings Have Large Volume

In “Good Clusterings Have Large Volume,” S. Borgwardt and F. Happach study the strong contrast between the favorable performance of clustering algorithms in practice and their bad theoretical worst cases. They explain this contrast using polyhedral theory. Classical clustering algorithms can be interpreted as the search for vertices in the so-called bounded-shape partition polytopes, following a "random" direction. The vertices correspond to clusterings with extraordinary separation properties, and the quality of this separation depends on the volume of the normal cone of the vertex—the larger the better. This gives rise to a new quality criterion for clusterings and explains why good clusterings are also the most likely to be found. The authors further characterize the edges of the polytopes, which allows for an explicit description of the normal cone, and in turn the computation of a best separation between the clusters of a given clustering.

Optimal Pragmatic Clustering for Wireless Networks

Nodes’ clustering in wireless networks is one of the solutions that used to improve network performance. This paper discusses the clustering in wireless networks. Then it presents a novel clustering algorithm named Pragmatic Genetic Algorithm (PGA). It combines two of the well known artificial intelligence techniques: K-means and Genetic algorithm. The proposed algorithm aims at minimizing the execution time of the clustering, especially in time-sensitive wireless networks applications. The performance of PGA has been compared with the classical clustering algorithms, namely, K-means and KGA. The experiments have been conducted using synthetic and real data from public repositories. PGA obtained excellent results in execution and stable accuracy even when the number of nodes was increased.

A Novel Solutions for Malicious Code Detection and Family Clustering Based on Machine Learning

Malware has become a major threat to cyberspace security, not only because of the increasing complexity of malware itself, but also because of the continuously created and produced malicious code. In this paper, we propose two novel methods to solve the malware identification problem. One is to solve to malware classification. Different from traditional machine learning, our method introduces the ensemble models to solve the malware classification problem. The other is to solve malware family clustering. Different from the classic malware family clustering algorithm, our method introduces the t-SNE algorithm to visualize the feature data and then determines the number of malware families. The two proposed novel methods have been extensively tested on a large number of real-world malware samples. The results show that the first one is far superior to the existed individual models and the second one has a good adaptation ability. Our methods can be used for malicious code classification and family clustering, also with higher accuracy.

Bayesian Nonparametric Clustering as a Community Detection Problem

It is well known that a wide class of bayesian nonparametric priors lead to the representation of the distribution of the observable variables as a mixture density with an infinite number of components, and that such a representation induces a clustering structure in the observations. However, cluster identification is not straightforward a posteriori and some post-processing is usually required. In order to circumvent label switching, pairwise posterior similarity has been introduced, and it has been used in order to either apply classical clustering algorithms or estimate the underlying partition by minimising a suitable loss function. This paper proposes to map observations on a weighted undirected graph, where each node represents a sample item and edge weights are given by the posterior pairwise similarities. It will be shown how, after building a particular random walk on such a graph, it is possible to apply a community detection algorithm, known as map equation method, by optimising the description length of the partition. A relevant feature of this method is that it allows for both the quantification of the posterior uncertainty of the classification and the selection of variables to be used for classification purposes.

A novel clustering algorithm based on the deviation factor model

For classical clustering algorithms, it is difficult to find clusters that have non-spherical shapes or varied size and density. In view of this, many methods have been proposed in recent years to overcome this problem, such as introducing more representative points per cluster, considering both interconnectivity and closeness, and adopting the density-based method. However, the density defined in DBSCAN is decided by minPts and Eps, and it is not the best solution to describe the data distribution of one cluster. In this paper, a deviation factor model is proposed to describe the data distribution and a novel clustering algorithm based on artificial immune system is presented. The experimental results show that the proposed algorithm is more effective than DBSCAN, k-means, etc.

The Information Bottleneck and Geometric Clustering.

The information bottleneck (IB) approach to clustering takes a joint distribution and maps the data to cluster labels , which retain maximal information about (Tishby, Pereira, & Bialek, 1999 ). This objective results in an algorithm that clusters data points based on the similarity of their conditional distributions . This is in contrast to classic geometric clustering algorithms such as -means and gaussian mixture models (GMMs), which take a set of observed data points and cluster them based on their geometric (typically Euclidean) distance from one another. Here, we show how to use the deterministic information bottleneck (DIB) (Strouse & Schwab, 2017 ), a variant of IB, to perform geometric clustering by choosing cluster labels that preserve information about data point location on a smoothed data set. We also introduce a novel intuitive method to choose the number of clusters via kinks in the information curve. We apply this approach to a variety of simple clustering problems, showing that DIB with our model selection procedure recovers the generative cluster labels. We also show that, in particular limits of our model parameters, clustering with DIB and IB is equivalent to -means and EM fitting of a GMM with hard and soft assignments, respectively. Thus, clustering with (D)IB generalizes and provides an information-theoretic perspective on these classic algorithms.

Spectral Clustering by Joint Spectral Embedding and Spectral Rotation.

Spectral clustering is an important clustering method widely used for pattern recognition and image segmentation. Classical spectral clustering algorithms consist of two separate stages: 1) solving a relaxed continuous optimization problem to obtain a real matrix followed by 2) applying K -means or spectral rotation to round the real matrix (i.e., continuous clustering result) into a binary matrix called the cluster indicator matrix. Such a separate scheme is not guaranteed to achieve jointly optimal result because of the loss of useful information. To obtain a better clustering result, in this paper, we propose a joint model to simultaneously compute the optimal real matrix and binary matrix. The existing joint model adopts an orthonormal real matrix to approximate the orthogonal but nonorthonormal cluster indicator matrix. It is noted that only in a very special case (i.e., all clusters have the same number of samples), the cluster indicator matrix is an orthonormal matrix multiplied by a real number. The error of approximating a nonorthonormal matrix is inevitably large. To overcome the drawback, we propose replacing the nonorthonormal cluster indicator matrix with a scaled cluster indicator matrix which is an orthonormal matrix. Our method is capable of obtaining better performance because it is easy to minimize the difference between two orthonormal matrices. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed method (called JSESR).