Cluster Analysis Problem Research Articles

A Comparison of Information Criteria in Clustering Based on Mixture of Multivariate Normal Distributions

Clustering analysis based on a mixture of multivariate normal distributions is commonly used in the clustering of multidimensional data sets. Model selection is one of the most important problems in mixture cluster analysis based on the mixture of multivariate normal distributions. Model selection involves the determination of the number of components (clusters) and the selection of an appropriate covariance structure in the mixture cluster analysis. In this study, the efficiency of information criteria that are commonly used in model selection is examined. The effectiveness of information criteria has been determined according to the success in the selection of the number of components and in the selection of an appropriate covariance matrix.

Viral Clustering: A Robust Method to Extract Structures in Heterogeneous Datasets

Cluster validation constitutes one of the most challenging problems in unsupervised cluster analysis. For example, identifying the true number of clusters present in a dataset has been investigated for decades, and is still puzzling researchers today. The difficulty stems from the high variety of the dataset characteristics. Some datasets exhibit a strong structure with a few well-separated and normally distributed clusters, but most often real-world datasets contain possibly many overlapping non-gaussian clusters with heterogeneous variances and shapes. This calls for the design of robust clustering algorithms that could adapt to the structure of the data and in particular accurately guess the true number of clusters. They have recently been interesting attempts to design such algorithms, e.g. based on involved non-parametric statistical inference techniques. In this paper, we develop Viral Clustering (VC), a simple algorithm that jointly estimates the number of clusters and outputs clusters. The VC algorithm relies on two antagonist and interacting components. The first component tends to regroup neighbouring samples together, while the second component tends to spread samples in various clusters. This spreading component is performed using an analogy with the way virus spread over networks. We present extensive numerical experiments illustrating the robustness of the VC algorithm, and its superiority compared to existing algorithms.

A New Definition of Intuitionistic Fuzzy Similarity Degree

As far as the problem of intuitionistic fuzzy cluster analysis is concerned, this paper proposes a new formula of similarity degree with attribute weight of each index. We conduct a fuzzy cluster analysis based on the new intuitionistic fuzzy similarity matrix, which is constructed via this new weighted similarity degree method and can be transformed into a fuzzy similarity matrix. Moreover, an example is given to demonstrate the feasibility and validity of this method.

On the translation-invariance of image distance metric

An appropriate choice of the distance metric is a fundamental problem in pattern recognition, machine learning and cluster analysis. Some methods that based on the distance of samples, e.g, the k-means clustering algorithm and the k-nearest neighbor classifier, are crucially relied on the performance of the distance metric. In this paper, the property of translation invariance for the distance metric of images is especially emphasized. The consideration is twofold. Firstly, some of the commonly used distance metrics, such as the Euclidean and Minkowski distance, are independent of the training set and/or the domain-specific knowledge. Secondly, the translation invariance is a necessary property for any intuitively reasonable image metric. The image Euclidean distance (IMED) and generalized Euclidean distance (GED) are image metrics that take the spatial relationship between pixels into consideration. Sun et al.(IEEE Conference on Computer Vision and Pattern Recognition, pp 1398–1405, 2009) showed that IMED is equivalent to a translation-invariant transform and proposed a metric learning algorithm based on the equivalency. In this paper, we provide a complete treatment on this topic and extend the equivalency to the discrete frequency domain. Based on the connection, we show that GED and IMED can be implemented as low-pass filters, which reduce the space and time complexities significantly. The transform domain metric learning proposed in (Sun et al. 2009) is also resembled as a translation-invariant counterpart of LDA. Experimental results demonstrate improvements in algorithm efficiency and performance boosts on the small sample size problems.

Flexible parametric bootstrap for testing homogeneity against clustering and assessing the number of clusters

There are two notoriously hard problems in cluster analysis, estimating the number of clusters, and checking whether the population to be clustered is not actually homogeneous. Given a dataset, a clustering method and a cluster validation index, this paper proposes to set up null models that capture structural features of the data that cannot be interpreted as indicating clustering. Artificial datasets are sampled from the null model with parameters estimated from the original dataset. This can be used for testing the null hypothesis of a homogeneous population against a clustering alternative. It can also be used to calibrate the validation index for estimating the number of clusters, by taking into account the expected distribution of the index under the null model for any given number of clusters. The approach is illustrated by three examples, involving various different clustering techniques (partitioning around medoids, hierarchical methods, a Gaussian mixture model), validation indexes (average silhouette width, prediction strength and BIC), and issues such as mixed type data, temporal and spatial autocorrelation.

Variable Selection and Outlier Detection for Automated K-means Clustering

An important problem in cluster analysis is the selection of variables that define cluster structure that also eliminate noisy variables that mask cluster structure; in addition, outlier detection is a fundamental task for cluster analysis. Here we provide an automated K-means clustering process combined with variable selection and outlier identification. The Automated K-means clustering procedure consists of three processes: (i) automatically calculating the cluster number and initial cluster center whenever a new variable is added, (ii) identifying outliers for each cluster depending on used variables, (iii) selecting variables defining cluster structure in a forward manner. To select variables, we applied VS-KM (variable-selection heuristic for K-means clustering) procedure (Brusco and Cradit, 2001). To identify outliers, we used a hybrid approach combining a clustering based approach and distance based approach. Simulation results indicate that the proposed automated K-means clustering procedure is effective to select variables and identify outliers. The implemented R program can be obtained at http://www.knou.ac.kr/∼sskim/SVOKmeans.r.

Применение задач оптимизации в кластерном анализе

Cluster analysis [3] is a relatively new branch of mathematics that studies the methods partitioning a set of objects, given a finite set of attributes into homogeneous groups (clusters). Cluster analysis is widely used in psychology, sociology, economics (market segmentation), and many other areas in which there is a problem of classification of objects according to their characteristics. Clustering methods implemented in a package STATISTICA [1] and SPSS [2], they return the partitioning into clusters, clustering and dispersion statistics dendrogram of hierarchical clustering algorithms. MS Excel Macros for main clustering methods and application examples are given in the monograph [5].
 One of the central problems of cluster analysis is to define some criteria for the number of clusters, we denote this number by K, into which separated are a given set of objects. There are several dozen approaches [4] to determine the number K. In particular, according to [6], the number of clusters K - minimum number which satisfies where - the minimum value of total dispersion for partitioning into K clusters, N - number of objects. Among the clusters automatically causes the consistent application of abnormal clusters [4]. In 2010, proposed and experimentally validated was a method for obtaining the number of K by applying the density function [4].
 The article offers two simple approaches to determining K, where each cluster has at least two objects. In the first number K is determined by the shortest Hamiltonian cycles in the second - through the minimum spanning tree. The examples of clustering with detailed step by step solutions and graphic illustrations are suggested. Shown is the use of macro VBA Excel, which returns the minimum spanning tree to the problems of clustering. The article contains a macro code, with commentaries to the main unit.

Missing Data Imputation by Nearest-neighbor Trained BP for Fuzzy Clustering

Incomplete data set is a common problem in clustering analysis. It is quite important to process the missing data which is the key factor impacting the clustering performance. In view of the problem that the Fuzzy C-means (FCM) algorithm is not directly applicable to the case of incomplete data, a FCM clustering algorithm for incomplete data sets based on improved BP to estimate the missing attributes is proposed in this paper. The improved BP can be trained by incomplete data sets, and we adopt the nearest-neighbor rule to select training samples for missing attributes. Training samples are composed by complete samples and incomplete samples of the incomplete data set, which make full use of data set information and can obtain more accuracy imputations of missing attributes. The experimental results for several UCI data sets demonstrate that the proposed method for clustering of incomplete data sets has a good effect.

The Application of Fuzzy Equivalence Relation Based on the Quotient Space in Cluster Analysis

A fuzzy clustering analysis model based on the quotient space is proposed. Firstly, the conversion from coarse to fine granularity and the hierarchical structure are used to reduce the multidimensional samples. Secondly, the fuzzy compatibility relation matrix of the model is converted into fuzzy equivalence relation matrix. Finally, the diagram of clustering genealogy is generated according to the fuzzy equivalence relation matrix, which enables the dynamic selection of different thresholds to effectively solve the problem of cluster analysis of the samples with multi-dimensional attributes.

Oscillatory neural networks based on the Kuramoto model for cluster analysis

This paper presents the results of a study of synchronization processes in oscillatory neural networks of various structures based on the Kuramoto model. The estimates of synchronization processes occurring in the oscillatory networks are examined. The results of studying the practical application of oscillatory networks for solving cluster analysis problems are presented.

A New Algorithm for Fuzzy Clustering Handling Incomplete Dataset

One of the most difficult problems in cluster analysis is the identification of the number of groups in a dataset especially in the presence of missing value. Since traditional clustering methods assumed the real number of clusters to be known. However, in real world applications the number of clusters is generally not known a priori. Also, most of clustering methods were developed to analyse complete datasets, they cannot be applied to many practical problems, e.g., on incomplete data. This paper focuses, first, on an algorithm of a fuzzy clustering approach, called OCS-FSOM. The proposed algorithm is based on neural network and uses Optimal Completion Strategy for missing value estimation in incomplete dataset. Then, we propose an extension of our algorithm, to tackle the problem of estimating the number of clusters, by using a multi level OCS-FSOM method. The new algorithm called Multi-OCSFSOM is able to find the optimal number of clusters by using a statistical criterion, that aims at measuring the quality of obtained partitions. Carried out experiments on real-life datasets highlights a very encouraging results in terms of exact determination of optimal number of clusters.

Predicting the Power Load of EV in Public Transportation Sector Based on Fuzzy Clustering Analysis

The electric demand of EV in the public transportation sector is increasingly important to the future city’s power distribution and even infrastructure construction. According to the characteristic of public transportation, this paper analyzed the influence factors of EV power load and they were divided into three parts. Then a predicting model of EV power load in public transportation based on fuzzy clustering analysis method was put forward. We used BP (Back Propagation) neural network algorithm to solve the fuzzy clustering analysis problem. Finally the predicting model was operated in a practical example. Results showed that this predicting model of EV power load in public transportation based on fuzzy clustering analysis could be appropriately applied in reality.

A hybrid clustering algorithm based on missing attribute interval estimation for incomplete data

Partially missing data sets are a prevailing problem in clustering analysis. We propose a hybrid algorithm combining fuzzy clustering with particle swarm optimization (PSO) for incomplete data clustering, and missing attributes are represented as intervals. Furthermore, we develop a neighbor interval reconstruction (NIR) method based on pre-classification results that estimates the nearest-neighbor interval of missing attribute using the nearest-neighbor rule, which avoids endpoints of intervals determined by different species information, thereby improving the accuracy of missing attribute intervals and enhancing the robustness of missing attribute imputation. Then, the PSO and fuzzy c-means hybrid algorithm are used for clustering the interval-valued data set, and the global optimization ability of the PSO can improve the accuracy of clustering results compared with gradient-based optimization methods. The experimental results for several UCI data sets show the superiority of the proposed NIR hybrid algorithm.

Estimating the Number of Clusters via the GUD Statistic

Estimating the number of clusters is one of the most difficult problems in cluster analysis. Most previous approaches require knowing the data matrix and may not work when only a Euclidean distance matrix is available. Other approaches also suffer from the curse of dimensionality and work poorly in high dimension. In this article, we develop a new statistic, called the GUD statistic, based on the idea of the Gap method, but use the determinant of the pooled within-group scatter matrix instead of the within-cluster sum of squared distances. Some theory is developed to show this statistic can work well when only the Euclidean distance matrix is known. More generally, this statistic can even work for any dissimilarity matrix that satisfies some properties. We also propose a modification for high-dimensional datasets, called the R-GUD statistic, which can give a robust estimation in high-dimensional settings. The simulation shows our method needs less information but is generally found to be more accurate and robust than other methods considered in the study, especially in many difficult settings.

PFClust: an optimised implementation of a parameter-free clustering algorithm

BackgroundA well-known problem in cluster analysis is finding an optimal number of clusters reflecting the inherent structure of the data. PFClust is a partitioning-based clustering algorithm capable, unlike many widely-used clustering algorithms, of automatically proposing an optimal number of clusters for the data.ResultsThe results of tests on various types of data showed that PFClust can discover clusters of arbitrary shapes, sizes and densities. The previous implementation of the algorithm had already been successfully used to cluster large macromolecular structures and small druglike compounds. We have greatly improved the algorithm by a more efficient implementation, which enables PFClust to process large data sets acceptably fast.ConclusionsIn this paper we present a new optimized implementation of the PFClust algorithm that runs considerably faster than the original.

A comparison of five recursive partitioning methods to find person subgroups involved in meaningful treatment–subgroup interactions

In case multiple treatment alternatives are available for some medical problem, the detection of treatment---subgroup interactions (i.e., relative treatment effectiveness varying over subgroups of persons) is of key importance for personalized medicine and the development of optimal treatment assignment strategies. Randomized Clinical Trials (RCT) often go without clear a priori hypotheses on the subgroups involved in treatment---subgroup interactions, and with a large number of pre-treatment characteristics in the data. In such situations, relevant subgroups (defined in terms of pre-treatment characteristics) are to be induced during the actual data analysis. This comes down to a problem of cluster analysis, with the goal of this analysis being to find clusters of persons that are involved in meaningful treatment---person cluster interactions. For such a cluster analysis, five recently proposed methods can be used, all being of a recursive partitioning type. However, these five methods have been developed almost independently, and the relations between them are not yet understood. The present paper closes this gap. It starts by outlining the basic principles behind each method, and by illustrating it with an application on an RCT data set on two treatment strategies for substance abuse problems. Next, it presents a comparison of the methods, hereby focusing on major similarities and differences. The discussion concludes with practical advice for end users with regard to the selection of a suitable method, and with an important challenge for future research in this area.

On complexity of some problems of cluster analysis of vector sequences

NP-completeness of two clustering (partition) problems is proved for a finite sequence of Euclidean vectors. In the optimization versions of both problems it is required to partition the elements of the sequence into a fixed number of clusters minimizing the sum of squares of the distances from the cluster elements to their centers. In the first problem the sizes of clusters are the part of input, while in the second they are unknown (they are the variables for optimization). Except for the center of one (special) cluster, the center of each cluster is the mean value of all vectors contained in it. The center of the special cluster is zero. Also, the partition must satisfy the following condition: The difference between the indices of two consecutive vectors in every nonspecial cluster is bounded below and above by two given constants.

An automatic method to determine the number of clusters using decision-theoretic rough set

Clustering provides a common means of identifying structure in complex data, and there is renewed interest in clustering as a tool for the analysis of large data sets in many fields. Determining the number of clusters in a data set is one of the most challenging and difficult problems in cluster analysis. To combat the problem, this paper proposes an efficient automatic method by extending the decision-theoretic rough set model to clustering. A new clustering validity evaluation function is designed based on the risk calculated by loss functions and possibilities. Then a hierarchical clustering algorithm, ACA-DTRS algorithm, is proposed, which is proved to stop automatically at the perfect number of clusters without manual interference. Furthermore, a novel fast algorithm, FACA-DTRS, is devised based on the conclusion obtained in the validation of the ACA-DTRS algorithm. The performance of algorithms has been studied on some synthetic and real world data sets. The algorithm analysis and the results of comparison experiments show that the new method, without manual parameter specified in advance, is more valid to determine the number of clusters and more efficient in terms of time cost.

A Framework for Multiple Imputation in Cluster Analysis

Multiple imputation is a common technique for dealing with missing values and is mostly applied in regression settings. Its application in cluster analysis problems, where the main objective is to classify individuals into homogenous groups, involves several difficulties which are not well characterized in the current literature. In this paper, we propose a framework for applying multiple imputation to cluster analysis when the original data contain missing values. The proposed framework incorporates the selection of the final number of clusters and a variable reduction procedure, which may be needed in data sets where the ratio of the number of persons to the number of variables is small. We suggest some ways to report how the uncertainty due to multiple imputation of missing data affects the cluster analysis outcomes-namely the final number of clusters, the results of a variable selection procedure (if applied), and the assignment of individuals to clusters. The proposed framework is illustrated with data from the Phenotype and Course of Chronic Obstructive Pulmonary Disease (PAC-COPD) Study (Spain, 2004-2008), which aimed to classify patients with chronic obstructive pulmonary disease into different disease subtypes.

Cluster Analysis by Variance Ratio Criterion and Firefly Algorithm

In order to solve the cluster analysis problem more efficiently, we presented a new approach based on firefly algorithm (FA). First, we created the optimization model using the variance ratio criterion (VRC) as fitness function. Second, FA was introduced to find the maximal point of the VRC. The experimental dataset contains 400 data of 4 groups with three different levels of overlapping degrees: non-overlapping, partial overlapping, and severely overlapping. We compared the FA with genetic algorithm (GA) and combinatorial particle swarm optimization (CPSO). Each algorithm was run 20 times. The results show that FA can found the largest VRC values among all three algorithms, while costs the least time. Therefore, FA is effective and rapid for the cluster analysis problem.