Cluster Analysis Problem Research Articles

A Hierarchical Method for Determining the Number of Clusters

A fundamental and difficult problem in cluster analysis is the determination of the "true" number of clusters in a dataset. The common trail-and-error method generally depends on certain clustering algorithms and is inefficient when processing large datasets. In this paper, a hierarchical method is proposed to get rid of repeatedly clustering on large datasets. The method firstly obtains the CF (clustering feature) via scanning the dataset and agglomerative generates the hierarchical partitions of dataset, then a curve of the clustering quality w.r.t the varying partitions is incrementally constructed. The partitions corresponding to the extremum of the curve is used to estimate the number of clusters finally. A new validity index is also presented to quantify the clustering quality, which is independent of clustering algorithm and emphasis on the geometric features of clusters, handling efficiently the noisy data and arbitrary shaped clusters. Experimental results on both real world and synthesis datasets demonstrate that the new method outperforms the recently published approaches, while the efficiency is significantly improved.

Weighted Clustering and Evolutionary Analysis of Hybrid Attributes Data Streams

It presents some definitions of projected cluster and projected cluster group on hybrid attributes after having given some definitions on ordered attributes and sorted attributes to solve clustering analysis problem of infinite hybrid attributes data streams in finite space. In order to improve the clustering quality of hybrid attributes data streams, it presents a two-step projected clustering method, which can often make better clustering effects in two simulated experiments although it is very simple. At last, it gives a dividing and merging framework of infinite hybrid attributes data streams. In order to implement this framework, it presents 8 properties in Section IV, some data structure definitions and 15 algorithms in appendix. The framework is verified and these algorithms are tested by German data set with a better clustering quality than WKMeans sometimes if having set right parameters.

Decision‐making method using a visual approach for cluster analysis problems; indicative classification algorithms and grouping scope

Abstract: Currently, classifying samples into a fixed number of clusters (i.e. supervised cluster analysis) as well as unsupervised cluster analysis are limited in their ability to support ‘cross‐algorithms’ analysis. It is well known that each cluster analysis algorithm yields different results (i.e. a different classification); even running the same algorithm with two different similarity measures commonly yields different results. Researchers usually choose the preferred algorithm and similarity measure according to analysis objectives and data set features, but they have neither a formal method nor tool that supports comparisons and evaluations of the different classifications that result from the diverse algorithms. Current research development and prototype decisions support a methodology based upon formal quantitative measures and a visual approach, enabling presentation, comparison and evaluation of multiple classification suggestions resulting from diverse algorithms. This methodology and tool were used in two basic scenarios: (I) a classification problem in which a ‘true result’ is known, using the Fisher iris data set; (II) a classification problem in which there is no ‘true result’ to compare with. In this case, we used a small data set from a user profile study (a study that tries to relate users to a set of stereotypes based on sociological aspects and interests). In each scenario, ten diverse algorithms were executed. The suggested methodology and decision support system produced a cross‐algorithms presentation; all ten resultant classifications are presented together in a ‘Tetris‐like’ format. Each column represents a specific classification algorithm, each line represents a specific sample, and formal quantitative measures analyse the ‘Tetris blocks’, arranging them according to their best structures, i.e. best classification.

Probabilistic algorithms for extreme point identification

For a finite set of distinct points S = {pi, i ε I} , in ℝ d there exists Î ⊆ I such that all points in Ŝ = {pi , i ε Î are extreme points and conv(Ŝ) = conv(S). Since a point pk is extreme if and only if the inequality is necessary with respect to the representation of the polar dual S Δ of S, Ŝ can be found by classifying the inequalities in the representation as necessary or redundant. Thus, the problem of finding Ŝ is polynomial. This paper shows the advantage of using probabilistic hit-and-run algorithms applied to the polar dual for the quick identification of points in Ŝ and shows how, in an application to a certain cluster analysis problem, it can be used to also identify points in S/Ŝ. Further, it shows that the hit-and-run variant known as Stand-and-Hit provides approximations for the exterior solid angles at the extreme points of S.

An algorithm for minimizing clustering functions †

The problem of cluster analysis is formulated as a problem of nonsmooth, nonconvex optimization. An algorithm for solving the latter optimization problem is developed which allows one to significantly reduce the computational efforts. This algorithm is based on the so-called discrete gradient method. Results of numerical experiments are presented which demonstrate the effectiveness of the proposed algorithm.

An optimization algorithm for clustering using weighted dissimilarity measures

One of the main problems in cluster analysis is the weighting of attributes so as to discover structures that may be present. By using weighted dissimilarity measures for objects, a new approach is developed, which allows the use of the k-means-type paradigm to efficiently cluster large data sets. The optimization algorithm is presented and the effectiveness of the algorithm is demonstrated with both synthetic and real data sets.

MODEL-BASED CLUSTERING IN GENE EXPRESSION MICROARRAYS: AN APPLICATION TO BREAST CANCER DATA

In microarray studies, the application of clustering techniques is often used to derive meaningful insights into the data. In the past, hierarchical methods have been the primary clustering tool employed to perform this task. The hierarchical algorithms have been mainly applied heuristically to these cluster analysis problems. Further, a major limitation of these methods is their inability to determine the number of clusters. Thus there is a need for a model-based approach to these clustering problems. To this end, McLachlan et al. [7] developed a mixture model-based algorithm (EMMIX-GENE) for the clustering of tissue samples. To further investigate the EMMIX-GENE procedure as a model-based approach, we present a case study involving the application of EMMIX-GENE to the breast cancer data as studied recently in van 't Veer et al. [10]. Our analysis considers the problem of clustering the tissue samples on the basis of the genes which is a non-standard problem because the number of genes greatly exceed the number of tissue samples. We demonstrate how EMMIX-GENE can be useful in reducing the initial set of genes down to a more computationally manageable size. The results from this analysis also emphasise the difficulty associated with the task of separating two tissue groups on the basis of a particular subset of genes. These results also shed light on why supervised methods have such a high misallocation error rate for the breast cancer data.

An algorithm for clustering based on non‐smooth optimization techniques

AbstractThe problem of cluster analysis is formulated as a problem of non‐smooth, non‐convex optimization, and an algorithm for solving the cluster analysis problem based on non‐smooth optimization techniques is developed. We discuss applications of this algorithm in large databases. Results of numerical experiments are presented to demonstrate the effectiveness of this algorithm.

Finding the Number of Clusters in a Dataset

One of the most difficult problems in cluster analysis is identifying the number of groups in a dataset. Most previously suggested approaches to this problem are either somewhat ad hoc or require parametric assumptions and complicated calculations. In this article we develop a simple, yet powerful nonparametric method for choosing the number of clusters based on distortion, a quantity that measures the average distance, per dimension, between each observation and its closest cluster center. Our technique is computationally efficient and straightforward to implement. We demonstrate empirically its effectiveness, not only for choosing the number of clusters, but also for identifying underlying structure, on a wide range of simulated and real world datasets. In addition, we give a rigorous theoretical justification for the method based on information-theoretic ideas. Specifically, results from the subfield of electrical engineering known as rate distortion theory allow us to describe the behavior of the distortion in both the presence and absence of clustering. Finally, we note that these ideas potentially can be extended to a wide range of other statistical model selection problems.

Concentration of measure and cluster analysis

An estimate on measure concentration for the problem of cluster analysis posed as a suitable integer linear program is stated and proved.

Cluster Analysis of the Realizations of Autoregression Time Series

Consideration was given to the problem of cluster analysis of observations which are realizations of the autoregression time series that are classified according to the unknown values of the autoregression parameters (coefficients). To solve the problem of classification, constructed were a procedure of cluster analysis in the space of realizations and a cluster procedure in the space of maximum likelihood estimates of the autoregression parameters. Efficiency of the proposed cluster procedures was studied both analytically and experimentally.

Disaggregative policy Delphi: Using cluster analysis as a tool for systematic scenario formation

A critical phase of scenario making is the choosing of scenarios. In the worst case, a futures researcher creates scenarios according to his/her subjective views and cannot see the real quality of the study material. Oversimplification is a typical example of this kind of bias. In this study, an attempt towards a more data sensitive method was made using Finnish transport policy as an example. A disaggregative Delphi method as opposed to traditional consensual Delphi was applied. The article summarises eight Delphi pitfalls and gives an example how to avoid them. A two-rounded disaggregative Delphi was conducted, the panelists being representatives of interest groups in the traffic sector. Panelists were shown the past development of three correlating key variables in Finland in 1970–1996: GDP, road traffic volume and the carbon dioxide emissions from road traffic. The panelists were invited to give estimates of their organisation to the probable and the preferable futures of the key variables for 1997–2025. They were also asked to give qualitative and quantitative arguments of why and the policy instruments of how their image of the future would occur. The first round data were collected by a fairly open questionnaire and the second round data by a fairly structured interview. The responses of the quantitative three key variables were grouped in a disaggregative way by cluster analysis. The clusters were complemented with respective qualitative arguments in order to form wider scenarios. This offers a relevance to decision-making not afforded by a nonsystematic approach. Of course, there are some problems of cluster analysis used in this way: The interviews revealed that quantitatively similar future images produced by the panelists occasionally had different kind of qualitative background theory. Also, cluster analysis cannot ultimately decide the number of scenarios, being a choice of the researcher. Cluster analysis makes the choice well argued, however.

Cluster number selection for a small set of samples using the Bayesian Ying-Yang model

One major problem in cluster analysis is the determination of the number of clusters. In this paper, we describe both theoretical and experimental results in determining the cluster number for a small set of samples using the Bayesian-Kullback Ying-Yang (BYY) model selection criterion. Under the second-order approximation, we derive a new equation for estimating the smoothing parameter in the cost function. Finally, we propose a gradient descent smoothing parameter estimation approach that avoids complicated integration procedure and gives the same optimal result.

A mixture model-based approach to the clustering of microarray expression data.

This paper introduces the software EMMIX-GENE that has been developed for the specific purpose of a model-based approach to the clustering of microarray expression data, in particular, of tissue samples on a very large number of genes. The latter is a nonstandard problem in parametric cluster analysis because the dimension of the feature space (the number of genes) is typically much greater than the number of tissues. A feasible approach is provided by first selecting a subset of the genes relevant for the clustering of the tissue samples by fitting mixtures of t distributions to rank the genes in order of increasing size of the likelihood ratio statistic for the test of one versus two components in the mixture model. The imposition of a threshold on the likelihood ratio statistic used in conjunction with a threshold on the size of a cluster allows the selection of a relevant set of genes. However, even this reduced set of genes will usually be too large for a normal mixture model to be fitted directly to the tissues, and so the use of mixtures of factor analyzers is exploited to reduce effectively the dimension of the feature space of genes. The usefulness of the EMMIX-GENE approach for the clustering of tissue samples is demonstrated on two well-known data sets on colon and leukaemia tissues. For both data sets, relevant subsets of the genes are able to be selected that reveal interesting clusterings of the tissues that are either consistent with the external classification of the tissues or with background and biological knowledge of these sets. EMMIX-GENE is available at http://www.maths.uq.edu.au/~gjm/emmix-gene/

A method of truncated codifferential with application to some problems of cluster analysis

A method of truncated codifferential descent for minimizing continuously codifferentiable functions is suggested. The convergence of the method is studied. Results of numerical experiments are presented. Application of the suggested method for the solution of some problems of cluster analysis are discussed. In numerical experiments Wisconsin Diagnostic Breast Cancer database was used.

A variable-selection heuristic for K-means clustering

One of the most vexing problems in cluster analysis is the selection and/or weighting of variables in order to include those that truly define cluster structure, while eliminating those that might mask such structure. This paper presents a variable-selection heuristic for nonhierarchical (K-means) cluster analysis based on the adjusted Rand index for measuring cluster recovery. The heuristic was subjected to Monte Carlo testing across more than 2200 datasets with known cluster structure. The results indicate the heuristic is extremely effective at eliminating masking variables. A cluster analysis of real-world financial services data revealed that using the variable-selection heuristic prior to the K-means algorithm resulted in greater cluster stability.

A New Approach to Solving Data Clusterization Problems

A new approach to solving some problems of cluster analysis is proposed, which reduces a multi-dimensional problem to a one-dimensional one.

ALL/AML Cancer Classification by Gene Expression Data Using SVM and CSVM Approach

Cancer classification plays an important role in cancer treatment. There has been no general approach for this problem now. The tasks for cancer classification are of two aspects: identifying new cancer classes and assigning tumors to known classes, which are called class discovery and class prediction by Golub et al. [1]. From mathematical point of view, class discovery is a cluster analysis problem, while class prediction is usually called classification problem (we’ll use the later name to keep consist with pattern recognition literatures). Until now, cancer classification has been based primarily on morphological appearance of tumor [1]. This has serious limitations because of ambiguity. Golub et al. presented a new approach to cancer classification based on gene expression monitoring by DNA microarrays in [1]. They chose acute leukemia as a test case, and the target is to distinguish between ALL (acute lymphoblastic leukemia) and AML (acute myeloid leukemia), which is a typical cancer classification problem not well solved despite many years of efforts. This paper is a report of our work on the classification (prediction) part of this problem following their original work. Golub et al. adopted a feature selection (gene selection) procedure before classification. A metric was defined to evaluate the correlation of each gene to the classification. After some “good” genes were selected from all the 6817 genes, the classification is done by a weighted voting scheme. The classifier was trained on a 38-sample training set, and another 34-sample set was used for testing. With leave-one-out cross-validation on the training set with 50 selected genes, 36 out of 38 samples were correctly classified and 2 were rejected (no-call). The performance on the test set was that 29 samples out of 34 were correctly classified and the other 5 were rejected. If the classifier were compelled to give these 5 no-calls a prediction, the prediction would be wrong. Since the feature selection procedure is of single selection type, and the classification method is also an intuitive one, we believe that there is still much space for the performance to be improved. In our approach to the problem, we took all the genes for the classification (the selection problem will be discussed in another paper), and applied the support vector machine(SVM) method and one of its improved version CSVM as the classifier. Thanks to the better generalization ability of SVM and CSVM, much better performance was obtained.

Robust Classification of Multinomial Observations with Possible Outliers

Cluster analysis problem of a mixture of multinomial random observations is considered with the presence of outliers in the sample. Robust decision rule based on the truncation principle is proposed and investigated.

Nuevas técnicas de particionamiento en clasificación automática

En este artículo se exponen algunas técnicas novedosas para la búsqueda de óptimos globales en el problema de la Clasificación Automática por medio de particiones con las cuales se mejoran sensiblemente los resultados obtenidos con los métodos tradicionales. Los métodos aquí desarrollados son bien conocidos en el campo de la Optimización Combinatoria: i) el sobrecalentamiento simulado, ii) la búsqueda tabú; iii) los algoritmos genéticos, Se utilizan estos tres enfoques aplkicacos al problema del particionamiento de objetos en Clasificación Automática, siguiendo un esquema de búsqueda análogo al planteado en el tradicional algoritmo de transferencias de Régnier.