Data Clustering Algorithm Research Articles

Sparse Count Data Clustering Using an Exponential Approximation to Generalized Dirichlet Multinomial Distributions.

Clustering frequency vectors is a challenging task on large data sets considering its high dimensionality and sparsity nature. Generalized Dirichlet multinomial (GDM) distribution is a competitive generative model for count data in terms of accuracy, yet its parameters estimation process is slow. The exponential-family approximation of the multivariate Polya distribution has shown to be efficient to train and cluster data directly, without dimensionality reduction. In this article, we derive an exponential-family approximation to the GDM distributions, and we call it (EGDM). A mixture model is developed based on the new member of the exponential-family of distributions, and its parameters are learned through the deterministic annealing expectation-maximization (DAEM) approach as a new clustering algorithm for count data. Moreover, we propose to estimate the optimal number of EGDM mixture components based on the minimum message length (MML) criterion. We have conducted a set of empirical experiments, concerning text, image, and video clustering, to evaluate the proposed approach performance. Results show that the new model attains a superior performance, and it is considerably faster than the corresponding method for GDM distributions.

A ranking paired based artificial bee colony algorithm for data clustering

A ranking paired based artificial bee colony algorithm for data clustering

Rat Swarm Optimizer for Data Clustering

Rat Swarm Optimization (RSO) is one of the newest swarm intelligence optimization algorithms that is inspired from the behaviors of chasing and fighting of rats in nature. In this paper we will apply the RSO to one of the most challenging problems, which is data clustering. The search capability of RSO is used here to find the best clusters centers. The proposed algorithm RSO for clustering (RSOC) is tested on several benchmarks and compared to some other optimization algorithms for data clustering including some wellknown and powerful algorithms such as Particle Swarm Optimization (PSO), and Genetic Algorithm (GA) and other recent such as the Hybridization of Krill Herd Algorithm and harmony search (H-KHA), hybrid Harris Hawks Optimization with differential evolution (H-HHO), and Multi-Verse Optimizer (MVO). Results are validated through a bunch of measures: homogeneity, completeness, v-measure, purity, and error rate. The computational results are encouraging, Where they demonstrate the effectiveness of RSOC over other techniques.

A ranking paired based artificial bee colony algorithm for data clustering

Data clustering aims to partition a dataset into k subsets according to a prespecified similarity measure. It is NP-hard, and has lots of real applications. This paper presents a ranking paired based artificial bee colony algorithm (RPABC) to solve data clustering. First, a chaotic map is employed to generate initial food sources. Second, in order to speed up the search, RPABC uses a ranking paired learning strategy to produce new positions. Finally, the best food source is utilised to guide the search in the onlooker bees' phase. Several datasets from the literature are used to test the RPABC. The computational results show that the proposed method is able to provide high quality clusters, and is more stable than the compared algorithms.

Формирование базы правил нечёткого классификатора с помощью метаэвристического алгоритма «саранчи»

The article presents a description of a hybrid algorithm for generating fuzzy rules for a fuzzy classifier using grasshopper optimization algorithm and the K-means data clustering algorithm. The performance of clustering was evaluated by three fitness functions: total variance, Davis–Bouldin index, and Calinski–Harabasz index. Triangular and Gaussian membership functions have been investigated. The efficiency of the generated fuzzy rule bases has been tested on real datasets. The best combination is to use the total variance as the fitness function and the Gaussian function as the membership function.

Towards automation of transcranial Doppler ultrasound data in pediatric cerebral malaria

ObjectiveTo work towards automating Transcranial Doppler (TCD) imaging data in cerebral malaria (CM) patients. MethodsFunctional principal component analysis (FPCA) was used to identify important features of TCD data and quantify its variation explained by each feature. Cox regression was used to test the association between TCD measurements and mortality hazards rate. A functional clustering algorithm was also employed to cluster TCD waveform data into distinct phenotypic groups. Fisher's exact test and one-way analysis of variance (ANOVA) were used to test the association between the TCD-defined phenotypic groups and 14 variables including demographic data, laboratory measurements, and clinical features. ResultsThe systolic, diastolic, and mean velocities as well as the Windkessel notch together explained more than 98% of total TCD data variation in the middle cerebral arteries (MCAs) and the basilar artery (BA). TCD measurements were associated with the survival hazards rates after adjusting for age and gender (chi-squared test statistic = 31.56, p-value = 0.025). A functional data clustering algorithm identified 7 clusters for the MCA TCD data. No associations between phenotypic groups and demographics, clinical features, or laboratory measurements were detected. ConclusionTCD may be useful as a diagnostic and monitoring tool in CM. Developing an algorithm-driven approach to interpreting the waveforms will enhance the scalability of this tool.

Analysis of Artistic Modeling of Opera Stage Clothing Based on Big Data Clustering Algorithm

In order to deal with the problem that the traditional stage costume artistry analysis method cannot correct the results of big data clustering, which leads to deviations in the extraction of costume artistry features, this paper proposes a clothing artistic modeling method based on big data clustering algorithm. The proposed method provides a database for big data clustering by constructing the attribute set of the big data feature sequence training set and, at the same time, constructing a second-order cone programming model to correct the big data. Aiming at the problem that traditional stage costume art analysis methods cannot correct the clustering results of big data. On this basis, the costume elements of the opera stage are segmented, initialized, and transformed into a binary function. Finally, using the convolutional neural network, combining the element segmentation results and the large data clustering space state vector, a feature extraction model of stage costume art is constructed. Experimental results show that the model has good convergence, short time-consuming, high accuracy, and ideal feature recognition capabilities.

Multiple Collaborative Supervision Pattern Recognition Method within Social Organizations Based on Data Clustering Algorithm

This paper proposes a multiple collaborative supervision pattern recognition method within social organizations based on data clustering algorithm to realize diversified supervision within social organizations and improve the effect of the said pattern recognition. Firstly, the characteristics and functions of social organizations are analyzed, and the definition of social organizations is given. Further, this paper studies the meaning and characteristics of social organization supervision, analyzes the failure of internal supervision of social organizations, and then determines the internal governance elements of social organizations. In addition, the basic steps of pattern recognition are given. Finally, multiple collaborative supervision patterns recognition within social organizations is realized based on data clustering algorithm. Experiments show that this method can improve the recognition accuracy of multiple collaborative supervision patterns and reduce the recognition time.

A Modified Cuckoo Search Algorithm for Data Clustering

Clustering of data is one of the necessary data mining techniques, where similar objects are grouped in the same cluster. In recent years, many nature-inspired based clustering techniques have been proposed, which have led to some encouraging results. This paper proposes a Modified Cuckoo Search (MoCS) algorithm. In this proposed work, an attempt has been made to balance the exploration of the Cuckoo Search (CS) algorithm and to increase the potential of the exploration to avoid premature convergence. This algorithm is tested using fifteen benchmark test functions and is proved as an efficient algorithm in comparison to the CS algorithm. Further, this method is compared with well-known nature-inspired algorithms such as Ant Colony Optimization (ACO), Artificial Bee Colony (ABC), Particle Swarm Optimization (PSO), Particle Swarm Optimization with Age Group topology (PSOAG) and CS algorithm for clustering of data using six real datasets. The experimental results indicate that the MoCS algorithm achieves better results as compared to other algorithms in finding optimal cluster centers.

Smart Meter Evaluation System Based on Big Data

Aiming at the problem of unstable operation of smart meters, this paper proposes a smart meter evaluation system based on big data. First introduce big data related technologies, such as data storage, analysis, mining, etc.. Secondly, design the big data smart meter evaluation system model. Finally, use big data technology and clustering algorithm to realize the design of the big data smart meter operation evaluation system. The system can convert massive data from multiple systems into operational evaluation reports, which helps to reduce the waste of human and material resources.

Minimum spanning tree‐based cluster analysis: A new algorithm for determining inconsistent edges

AbstractIn recent years, graph‐based data clustering algorithms have become popular as they perform connectivity‐based rather than centroid‐based partitioning. Methods related to minimum spanning tree (MST)‐based data clustering are types of graph‐based algorithms that can recognize arbitrary shapes of clusters by eliminating inconsistent edges from MST graphs. In all MST‐based data clustering algorithms, definition of inconsistent edges is the main problem that needs to be addressed. The longest edges in MST graphs are considered as inconsistent edges under ideal conditions. Nevertheless, outliers often exist in real‐world tasks, which makes the longest edges inaccurate cluster separation indicators. In this paper, we propose a new data clustering algorithm using MST and a critical distance method. The proposed algorithm solves the main issue of MST‐based data clustering, namely identifying and removing inconsistent edges to obtain clusters even in the event that the dataset contains some outliers. It begins by constructing the MST over a given weighted graph based on Euclidean distance and then splits up the graph into clusters by eliminating inconsistent edges using critical distance as a threshold. Integration of the advantages of both MST and critical distance methodology to obtain optimal clusters is the main contribution of this work. The conducted experimental analysis and results using different datasets prove that our proposed clustering algorithm yields better overall performance compared with the most common data clustering algorithms. Taking the Liver and Tumor datasets as an example, the proposed algorithm outperforms all other clustering algorithms with clustering accuracy equal to 0.579 and 0.660, respectively.

Machine learning algorithm for feature space clustering of mixed data with missing information based on molecule similarity

Clustering Algorithms have just fascinated significant devotion in machine learning applications owing to their great competence. Nevertheless, the existing algorithms quite have approximately disputes that need to be further deciphered. For example, most existing algorithms transform one type of feature into another type, which disregards the explicit possessions of information. In addition, most of them deliberate whole features, which may lead to difficulty in calculation and effect in sub-optimal presentation. To address the above difficulties, this paper proposes a novel technique for clustering categorical and numerical features based on feature space clustering of mixed data with missing information (FSCMMI). The procedure involves three stages. Initially, FSCMMI divides the given dataset depending on missing information in instances and features types. The second stage uses the decision-tree procedure to identify the association between instances. Finally, the third stage is used for computing the closeness measure for numerical features and categorical features. Meanwhile, we propose a new training algorithm to cluster mixed datasets. Extensive experimental results on benchmark datasets show that the proposed FSCMMI outperforms several state-of-art clustering methods in terms of accuracy and efficiency.

A fault-tolerant clustering algorithm for processing data from multiple streams

Nowadays often multiple sensors provide real-time data about the same process. However, most data stream analysis algorithms work over a single feature vector. A common approach to overcome this issue is to combine the features extracted from each sensor into a single vector. The more sensors monitor a process, the more likely there is a malfunction or delay in data transmission. Hence the interest in developing algorithms capable of handling missing data or incorporating delayed data when it becomes available. This work presents a dynamic ensemble clustering algorithm based on evidence accumulation that can process multiple data streams. Each algorithm of the ensemble only processes one data stream, and obtaining the final partition does not require that data from all the streams is available. Therefore, final results can be provided even when there are sensor malfunctions or network delays. Furthermore, if delayed data arrives it can be used to extract evidence from the moment of arrival onwards. The algorithm was applied to identify arrhythmias over 71 standard 12-lead electrocardiogram recordings, achieving a 0.314% ± 0.010 error rate. The strategies of combining all the leads in a single feature vector or working with each lead independently were compared, obtaining better results in the second case.

Statement of retraction: collaborative correlation space big data clustering algorithm for abnormal flow monitoring

Statement of retraction: collaborative correlation space big data clustering algorithm for abnormal flow monitoring

Differential expression analysis using a model-based gene clustering algorithm for RNA-seq data

BackgroundRNA-seq is a tool for measuring gene expression and is commonly used to identify differentially expressed genes (DEGs). Gene clustering is used to classify DEGs with similar expression patterns for the subsequent analyses of data from experiments such as time-courses or multi-group comparisons. However, gene clustering has rarely been used for analyzing simple two-group data or differential expression (DE). In this study, we report that a model-based clustering algorithm implemented in an R package, MBCluster.Seq, can also be used for DE analysis.ResultsThe input data originally used by MBCluster.Seq is DEGs, and the proposed method (called MBCdeg) uses all genes for the analysis. The method uses posterior probabilities of genes assigned to a cluster displaying non-DEG pattern for overall gene ranking. We compared the performance of MBCdeg with conventional R packages such as edgeR, DESeq2, and TCC that are specialized for DE analysis using simulated and real data. Our results showed that MBCdeg outperformed other methods when the proportion of DEG (PDEG) was less than 50%. However, the DEG identification using MBCdeg was less consistent than with conventional methods. We compared the effects of different normalization algorithms using MBCdeg, and performed an analysis using MBCdeg in combination with a robust normalization algorithm (called DEGES) that was not implemented in MBCluster.Seq. The new analysis method showed greater stability than using the original MBCdeg with the default normalization algorithm.ConclusionsMBCdeg with DEGES normalization can be used in the identification of DEGs when the PDEG is relatively low. As the method is based on gene clustering, the DE result includes information on which expression pattern the gene belongs to. The new method may be useful for the analysis of time-course and multi-group data, where the classification of expression patterns is often required.

Evolutionary state‐based novel multi‐objective periodic bacterial foraging optimization algorithm for data clustering

AbstractClustering divides objects into groups based on similarity. However, traditional clustering approaches are plagued by their difficulty in dealing with data with complex structure and high dimensionality, as well as their inability in solving multi‐objective data clustering problems. To address these issues, an evolutionary state‐based novel multi‐objective periodic bacterial foraging optimization algorithm (ES‐NMPBFO) is proposed in this article. The algorithm is designed to alleviate the high‐computing complexity of the standard bacterial foraging optimization (BFO) algorithm by introducing periodic BFO. Moreover, two learning strategies, global best individual (gbest) and personal historical best individual (pbest), are used in the chemotaxis operation to enhance the convergence speed and guide the bacteria to the optimum position. Two elimination‐dispersal operations are also proposed to prevent falling into local optima and improve the diversity of solutions. The proposed algorithm is compared with five other algorithms on six validity indexes in two data clustering cases comprising nine general benchmark datasets and four credit risk assessment datasets. The experimental results suggest that the proposed algorithm significantly outperforms the competing approaches. To further examine the effectiveness of the proposed strategies, two variants of ES‐NMPBFO were designed, and all three forms of ES‐NMPBFO were tested. The experimental results show that all of the proposed strategies are conducive to the improvement of solution quality, diversity and convergence.

GBIF Integration of Open Data

The Global Biodiversity Information Facility (GBIF) runs a global data infrastructure that integrates data from more than 1700 institutions. Combining data at this scale has been achieved by deploying open Application Programming Interfaces (API) that adhere to the open data standards provided by Biodiversity Information Standards (TDWG). In this presentation, we will provide an overview of the GBIF infrastructure and APIs and provide insight into lessons learned while operating and evolving the systems, such as long-term API stability, ease of use, and efficiency. This will include the following topics: The registry component provides RESTful APIs for managing the organizations, repositories and datasets that comprise the network and control access permissions. Stability and ease of use have been critical to this being embedded in many systems. Changes within the registry trigger data crawling processes, which connect to external systems through their APIs and deposit datasets into GBIF's central data warehouse. One challenge here relates to the consistency of data across a distributed network. Once a dataset is crawled, the data processing infrastructure organizes and enriches data using reference catalogues accessed through open APIs, such as the vocabulary server and the taxonomic backbone. Being able to process data quickly as source data and reference catalogues change is a challenge for this component. The data access APIs provide search and download services. Asynchronous APIs are required for some of these aspects, and long-term stability is a requirement for widespread adoption. Here we will talk about policies for schema evolution to avoid incompatible changes, which would cause failures in client systems. The APIs that drive the user interface have specific needs such as efficient use of the network bandwidth. We will present how we approached this, and how we are currently adopting GraphQL as the next generation of these APIs. There are several APIs that we believe are of use for the data publishing community. These include APIs that will help in data quality aspects, and new data of interest thanks to the data clustering algorithms GBIF deploys. The registry component provides RESTful APIs for managing the organizations, repositories and datasets that comprise the network and control access permissions. Stability and ease of use have been critical to this being embedded in many systems. Changes within the registry trigger data crawling processes, which connect to external systems through their APIs and deposit datasets into GBIF's central data warehouse. One challenge here relates to the consistency of data across a distributed network. Once a dataset is crawled, the data processing infrastructure organizes and enriches data using reference catalogues accessed through open APIs, such as the vocabulary server and the taxonomic backbone. Being able to process data quickly as source data and reference catalogues change is a challenge for this component. The data access APIs provide search and download services. Asynchronous APIs are required for some of these aspects, and long-term stability is a requirement for widespread adoption. Here we will talk about policies for schema evolution to avoid incompatible changes, which would cause failures in client systems. The APIs that drive the user interface have specific needs such as efficient use of the network bandwidth. We will present how we approached this, and how we are currently adopting GraphQL as the next generation of these APIs. There are several APIs that we believe are of use for the data publishing community. These include APIs that will help in data quality aspects, and new data of interest thanks to the data clustering algorithms GBIF deploys.

Fuzzy clustering algorithm for outlier-interval data based on the robust exponent distance

Fuzzy clustering algorithm for outlier-interval data based on the robust exponent distance

Effective density-based clustering algorithms for incomplete data

Density-based clustering is an important category among clustering algorithms. In real applications, many datasets suffer from incompleteness. Traditional imputation technologies or other techniques for handling missing values are not suitable for density-based clustering and decrease clustering result quality. To avoid these problems, we develop a novel density-based clustering approach for incomplete data based on Bayesian theory, which conducts imputation and clustering concurrently and makes use of intermediate clustering results. To avoid the impact of low-density areas inside non-convex clusters, we introduce a local imputation clustering algorithm, which aims to impute points to high-density local areas. The performances of the proposed algorithms are evaluated using ten synthetic datasets and five real-world datasets with induced missing values. The experimental results show the effectiveness of the proposed algorithms.

A Distributed Neighbourhood DBSCAN Algorithm for Effective Data Clustering in Wireless Sensor Networks

A Distributed Neighbourhood DBSCAN Algorithm for Effective Data Clustering in Wireless Sensor Networks