Clustering Of Multivariate Data Research Articles

Directed Clustering of Multivariate Data Based on Linear or Quadratic Latent Variable Models

Directed Clustering of Multivariate Data Based on Linear or Quadratic Latent Variable Models

Local Versions of Sum-of-Norms Clustering

Sum-of-norms clustering is a convex optimization problem whose solution can be used for the clustering of multivariate data. We propose and study a localized version of this method, and show in particular that it can separate arbitrarily close balls in the stochastic ball model. More precisely, we prove a quantitative bound on the error incurred in the clustering of disjoint connected sets. Our bound is expressed in terms of the number of datapoints and the localization length of the functional.

Obtaining synthetic indications and sorting relevant structures from complex hierarchical clusters of multivariate data

Hierarchical clustering of multivariate data usually provide useful information on the similarity among elements. Unfortunately, the clustering does not immediately suggest the data-governing structure. Moreover, the number of information retrieved by the data clustering can be sometimes so large to make the results little interpretable. This work presents two tools to derive relevant information from a large number of quantitative multivariate data, simply by post-processing the dendrograms resulting from hierarchical clustering. The first tool helps gaining a good insight in the physical relevance of the obtained clusters, i.e. whether the detected families of elements result from true or spurious similarities due to, e.g., experimental uncertainty. The second tool provides a deeper knowledge of the factors governing the distribution of the elements in the multivariate space, that is the determination of the most relevant parameters which affect the similarities among the configurations. These tools are, in particular, suitable to process experimental results to cope with related uncertainties, or to analyse multivariate data resulting from the study of complex or chaotic systems.

Visual Clustering Analysis of some traditional Mango (Mangifera indica L.) varieties of Murshidabad District, West Bengal using Clust Vis web tool

A clustergram or a heatmap is one of several techniques that directly visualize data without the need for dimensionality reduction. Heatmap is a representation of data in the form of a map or diagram in which data values are represented as colours. Cluster heatmaps have high data density, allowing them to compact large amounts of information into a small space. “ClustVis”, is a web tool for visualizing clustering of multivariate data using Principal Component Analysis and Heatmap. Using this web tool, genetic relationships among the traditional mango (Mangifera indica L.) varieties can be visualized. In this investigation ten (10) indigenous mango varieties were selected. These were elite varieties of Murshidabad viz. Anaras, Bhabani, Champa, Dilpasand, Kalabati, Kohinoor, Kohitoor, Molamjam. The morphological and biological characters were analyzed using this tool. Analysis and assessment of the current status of mango genetic resources will be important for ascertaining the relationship among traditional varieties. This data may be used for appropriate conservation and sustainable utilization measures. This information may also be needed to carry out breeding programs to develop improved cultivars for sustainable livelihoods of local communities.

Exhausted phenotype of follicular CD8 T cells in CVID

Exhausted phenotype of follicular CD8 T cells in CVID

Clustering Approach Using Artificial Bee Colony Algorithm for Healthcare Waste Disposal Facility Location Problem

In this study, an Artificial Bee Colony (ABC) based clustering algorithm is proposed for solving continuous multiple facility location problems. Unlike the original version applied to multivariate data clustering, the ABC based clustering here solves the two-dimensional clustering. On the other hand, the multiple facility location problem the proposed clustering algorithm deals with is aimed to find site locations for healthcare wastes. After applying ABC based clustering algorithm on test data, a real-world facility location problem is solved for identifying healthcare waste disposal facility locations for Istanbul Municipality. Geographical coordinates and healthcare waste amounts of Istanbul hospitals are used to decide the locations of sterilization facilities to be established for reducing the medical waste generated. ABC based clustering is performed for different number of clusters predefined by Istanbul Metropolitan Municipality, and the total cost—the amount of healthcare waste produced by a hospital, multiplied by its distance to the sterilization facility—is calculated to decide the number of facilities to be opened. Benchmark results with four algorithms for test data and with two algorithms for real world problem reveal the superior performance of the proposed methodology.

Combined use of infrared and hard X-ray microprobes for spectroscopy-based neuroanatomy

Understanding the pathological triggers that affect the structural and physiological integrity of biochemical milieu of neurons is crucial to extend our knowledge on brain disorders, that are in many circumstances hardly treatable. Over recently, by using sophisticated hyperspectral micro-imaging modalities, it has been placed within our reach to get an insight into high fidelity histological details along with corresponding biochemical information in a label-free fashion, without using any additional chemical fixatives. However, in order to push forwards extensive application of these methods in the clinical arena, it is viable to make further iterations in novel data analysis protocols in order to boost their sensitivity. Therefore, in our study we proposed a new combined approach utilizing both benchtop Fourier transform infrared (FTIR) and synchrotron X-ray fluorescence (SR-XRF) micro-spectroscopies coupled with multivariate data clustering using the K-means algorithm for combined molecular and elemental micro-imaging, so that these complimentary analytical tools could be used for delineating between various brain structures based on their biochemical composition. By utilizing mid-IR transmission FTIR experiments, the biochemical composition in terms of lipids, proteins and phosphodiesters became accessible. In turn, the SR-XRF experiment was carried out at the advanced IAEA X-ray spectrometry station at Elettra Sincrotrone Trieste. By measuring in vacuum and by using the primary exciting X-ray beam, monochromatized to 10.5 keV, we took advantage of accessing the characteristic X-ray lines of a variety of elements ranging from carbon to zinc. Herein, we can report that the developed methodology has high specificity for label-free discriminating between lipid- and protein-rich brain tissue areas.

Exponential family mixed membership models for soft clustering of multivariate data

For several years, model-based clustering methods have successfully tackled many of the challenges presented by data-analysts. However, as the scope of data analysis has evolved, some problems may be beyond the standard mixture model framework. One such problem is when observations in a dataset come from overlapping clusters, whereby different clusters will possess similar parameters for multiple variables. In this setting, mixed membership models, a soft clustering approach whereby observations are not restricted to single cluster membership, have proved to be an effective tool. In this paper, a method for fitting mixed membership models to data generated by a member of an exponential family is outlined. The method is applied to count data obtained from an ultra running competition, and compared with a standard mixture model approach.

Improved Ant Colony Clustering Algorithm and Its Performance Study.

Clustering analysis is used in many disciplines and applications; it is an important tool that descriptively identifies homogeneous groups of objects based on attribute values. The ant colony clustering algorithm is a swarm-intelligent method used for clustering problems that is inspired by the behavior of ant colonies that cluster their corpses and sort their larvae. A new abstraction ant colony clustering algorithm using a data combination mechanism is proposed to improve the computational efficiency and accuracy of the ant colony clustering algorithm. The abstraction ant colony clustering algorithm is used to cluster benchmark problems, and its performance is compared with the ant colony clustering algorithm and other methods used in existing literature. Based on similar computational difficulties and complexities, the results show that the abstraction ant colony clustering algorithm produces results that are not only more accurate but also more efficiently determined than the ant colony clustering algorithm and the other methods. Thus, the abstraction ant colony clustering algorithm can be used for efficient multivariate data clustering.

Study of Multivariate Data Clustering Based on K-Means and Independent Component Analysis

For last two decades, clustering is well-recognized area in the research field of data mining. Data clustering plays the major research at pattern recognition, Signal processing, bioinformatics and Artificial Intelligence. Clustering process is an unsupervised learning techniques where it generates a group of object based on their similarity in such a way that the objects belonging to other groups are similar and those belonging to other are dissimilar. This paper analysis the three different data types clustering techniques like K-Means, Principal components analysis (PCA) and Independent component analysis (ICA) in real and simulated data. The recent developments by considering a rather unexpected application of the theory of Independent component analysis (ICA) found in data clustering, outlier detection and multivariate data visualization. Accurate identification of data clustering plays an important role in statistical analysis. In this paper we explore the connection among these three techniques to identify multivariate data clustering and develop a new method k-means on PCA or ICA then the result shows that ICA based clustering performs well than others.

A Hybrid Clustering Approach using Artificial Bee Colony (ABC) and Particle Swarm Optimization

In this paper, Cluster analysis is a group objects like observations, events etc based on the information that are found in the data describing the objects or their relations. The main goal of the clustering is that the objects in a group will be similar or related to one other and different from (or unrelated to) the objects in other groups. In this paper, proposed a hybrid model of PSABC algorithm. The PSABC algorithm is a combination of Particle Swarm Algorithm (PSO) and Artificial Bee Colony (ABC) Algorithm used for data clustering on benchmark problems. The PSABC algorithm is compared with other existing classification techniques to evaluate the performance of the proposed approach. Thirteen of typical test data sets from the UCI Machine Learning Repository are used to demonstrate the results of the techniques. The simulation results indicate that PSABC algorithm can efficiently be used for multivariate data clustering.

GGSA: A Grouping Gravitational Search Algorithm for data clustering

Gravitational Search Algorithm (GSA) is a stochastic population-based metaheuristic designed for solving continuous optimization problems. It has a flexible and well-balanced mechanism for enhancing exploration and exploitation abilities. In this paper, we adapt the structure of GSA for solving the data clustering problem, the problem of grouping data into clusters such that the data in each cluster share a high degree of similarity while being very dissimilar to data from other clusters. The proposed algorithm, which is called Grouping GSA (GGSA), differs from the standard GSA in two important aspects. First, a special encoding scheme, called grouping encoding, is used in order to make the relevant structures of clustering problems become parts of solutions. Second, given the encoding, special GSA updating equations suitable for the solutions with grouping encoding are used. The performance of the proposed algorithm is evaluated through several benchmark datasets from the well-known UCI Machine Learning Repository. Its performance is compared with the standard GSA, the Artificial Bee Colony (ABC), the Particle Swarm Optimization (PSO), the Firefly Algorithm (FA), and nine other well-known classical classification techniques from the literature. The simulation results indicate that GGSA can effectively be used for multivariate data clustering.

Maximum likelihood estimation in constrained parameter spaces for mixtures of factor analyzers

Mixtures of factor analyzers are becoming more and more popular in the area of model based clustering of multivariate data. According to the likelihood approach in data modeling, it is well known that the unconstrained likelihood function may present spurious maxima and singularities. To reduce such drawbacks, in this paper we introduce a procedure for parameter estimation of mixtures of factor analyzers, which maximizes the likelihood function under the mild requirement that the eigenvalues of the covariance matrices lie into some interval [a,b]. Moreover, we give a recipe on how to select appropriate bounds for the constrained EM algorithm, directly from the handled data. We then analyze and measure its performance, compared with the usual non-constrained approach, and also with other constrained models in the literature. Results show that the data-driven constraints improve the estimation and the subsequent classification, at the same time.

Pair-copula based mixture models and their application in clustering

Finite mixtures are often used to perform model based clustering of multivariate data sets. In real life applications, such data may exhibit complex nonlinear form of dependence among the variables. Also, the individual variables (margins) may follow different families of distributions. Most of the existing mixture models are unable to accommodate these two aspects of the data. This paper presents a finite mixture model that involves a pair-copula based construction of a multivariate distribution. Such a model de-couples the margins and the dependence structures. Hence, the margins can be modeled using different families. Again, many possible dependence structures can also be studied using different copulas. The resulting mixture model (called DVMM) is then capable of capturing a broad family of distributions including non-Gaussian models. Here we study DVMM in the context of clustering of multivariate data. We design an expectation maximization procedure for estimating the mixture parameters. We perform extensive experiments on the basis of a number of well-known data sets. A detailed evaluation of the clustering quality obtained by DVMM in comparison to other mixture models is presented. The experimental results show that the performance of DVMM is quite satisfactory.

Subspace K-means clustering

To achieve an insightful clustering of multivariate data, we propose subspace K-means. Its central idea is to model the centroids and cluster residuals in reduced spaces, which allows for dealing with a wide range of cluster types and yields rich interpretations of the clusters. We review the existing related clustering methods, including deterministic, stochastic, and unsupervised learning approaches. To evaluate subspace K-means, we performed a comparative simulation study, in which we manipulated the overlap of subspaces, the between-cluster variance, and the error variance. The study shows that the subspace K-means algorithm is sensitive to local minima but that the problem can be reasonably dealt with by using partitions of various cluster procedures as a starting point for the algorithm. Subspace K-means performs very well in recovering the true clustering across all conditions considered and appears to be superior to its competitor methods: K-means, reduced K-means, factorial K-means, mixtures of factor analyzers (MFA), and MCLUST. The best competitor method, MFA, showed a performance similar to that of subspace K-means in easy conditions but deteriorated in more difficult ones. Using data from a study on parental behavior, we show that subspace K-means analysis provides a rich insight into the cluster characteristics, in terms of both the relative positions of the clusters (via the centroids) and the shape of the clusters (via the within-cluster residuals).

Fast Bayesian scan statistics for multivariate event detection and visualization

The multivariate Bayesian scan statistic (MBSS) is a recently proposed, general framework for event detection and characterization in multivariate space-time data. MBSS integrates prior information and observations from multiple data streams in a Bayesian framework, computing the posterior probability of each type of event in each space-time region. MBSS has been shown to have many advantages over previous event detection approaches, including improved timeliness and accuracy of detection, easy interpretation and visualization of results, and the ability to model and accurately differentiate between multiple event types. This work extends the MBSS framework to enable detection and visualization of irregularly shaped clusters in multivariate data, by defining a hierarchical prior over all subsets of locations. While a naive search over the exponentially many subsets would be computationally infeasible, we demonstrate that the total posterior probability that each location has been affected can be efficiently computed, enabling rapid detection and visualization of irregular clusters. We compare the run time and detection power of this 'Fast Subset Sums' method to our original MBSS approach (assuming a uniform prior over circular regions) on semi-synthetic outbreaks injected into real-world Emergency Department data from Allegheny County, Pennsylvania. We demonstrate substantial improvements in spatial accuracy and timeliness of detection, while maintaining the scalability and fast run time of the original MBSS method.

The snake for visualizing and for counting clusters in multivariate data

AbstractWe introduce the ‘snake’, a new tool for the visualization and exploration of a multivariate dataset. The snake connects each data point along a single short path. Using techniques from the Traveling Salesman Problem (TSP), it is possible to find such a path in polynomial (nearly quadratic) computational time. A plot of the individual segment lengths versus their position along the path transforms the original multidimensional dataset into a one‐dimensional ‘time‐series’ of interpoint distances. The snake traces the local structure of a datacloud, so this visualization is most useful for detecting density fluctuations: regions of high density appear as many consecutive short segments, while regions of low density appear as many consecutive long segments. Dips in the time series reveal the presence of clustering and can be used to count the number of modes in the datacloud. We illustrate the technique on a variety of artificial and real‐world datasets. Copyright © 2010 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 3: 236‐252, 2010

A novel clustering approach: Artificial Bee Colony (ABC) algorithm

Artificial Bee Colony (ABC) algorithm which is one of the most recently introduced optimization algorithms, simulates the intelligent foraging behavior of a honey bee swarm. Clustering analysis, used in many disciplines and applications, is an important tool and a descriptive task seeking to identify homogeneous groups of objects based on the values of their attributes. In this work, ABC is used for data clustering on benchmark problems and the performance of ABC algorithm is compared with Particle Swarm Optimization (PSO) algorithm and other nine classification techniques from the literature. Thirteen of typical test data sets from the UCI Machine Learning Repository are used to demonstrate the results of the techniques. The simulation results indicate that ABC algorithm can efficiently be used for multivariate data clustering.

Variational Bayesian Generative Topographic Mapping

General finite mixture models are powerful tools for the density-based grouping of multivariate i.i.d. data, but they lack data visualization capabilities, which reduces their practical applicability to real-world problems. Generative topographic mapping (GTM) was originally formulated as a constrained mixture of distributions in order to provide simultaneous visualization and clustering of multivariate data. In its inception, the adaptive parameters were determined by maximum likelihood (ML), using the expectation-maximization (EM) algorithm. The original GTM is, therefore, prone to data overfitting unless a regularization mechanism is included. In this paper, we define an alternative variational formulation of GTM that provides a full Bayesian treatment to a Gaussian process (GP)-based variation of the model. The generalization capabilities of the proposed Variational Bayesian GTM are assessed in some detail and compared with those of alternative GTM regularization approaches in terms of test log-likelihood, using several artificial and real datasets.

Bayesian clustering with AutoClass explicitly recognises uncertainties in landscape classification

Clustering of multivariate data is a commonly used technique in ecology, and many approaches to clustering are available. The results from a clustering algorithm are uncertain, but few clustering approaches explicitly acknowledge this uncertainty. One exception is Bayesian mixture modelling, which treats all results probabilistically, and allows comparison of multiple plausible classifications of the same data set. We used this method, implemented in the AutoClass program, to classify catchments (watersheds) in the Murray Darling Basin (MDB), Australia, based on their physiographic characteristics (e.g. slope, rainfall, lithology). The most likely classification found nine classes of catchments. Members of each class were aggregated geographically within the MDB. Rainfall and slope were the two most important variables that defined classes. The second‐most likely classification was very similar to the first, but had one fewer class. Increasing the nominal uncertainty of continuous data resulted in a most likely classification with five classes, which were again aggregated geographically. Membership probabilities suggested that a small number of cases could be members of either of two classes. Such cases were located on the edges of groups of catchments that belonged to one class, with a group belonging to the second‐most likely class adjacent. A comparison of the Bayesian approach to a distance‐based deterministic method showed that the Bayesian mixture model produced solutions that were more spatially cohesive and intuitively appealing. The probabilistic presentation of results from the Bayesian classification allows richer interpretation, including decisions on how to treat cases that are intermediate between two or more classes, and whether to consider more than one classification. The explicit consideration and presentation of uncertainty makes this approach useful for ecological investigations, where both data and expectations are often highly uncertain.