Automatic Clustering Problem Research Articles

K-Means Hybridization with Enhanced Firefly Algorithm for High-Dimension Automatic Clustering

K-means clustering is not able to select the right number of clusters of data items having high-dimension. So, for determining the ideal number of clusters, we have combined PCA with the Silhouette and Elbow approaches. Additionally, we have a large number of meta-heuristic swarm intelligence algorithms which is influenced by nature and were previously used to solve the automatic data clustering problem. Firefly offers reliable and effective automatic data clustering. The Firefly algorithm automatically divides the entire population into subpopulations, which slows down the convergence and reduces the of capturing local minima in high-dimensional optimization problems. Thus, for automatic clustering, we demonstrated an improved firefly, i.e., we offered a hybridized K-means with an ODFA model. The experimental section displays the results and graphs for the Silhouette, Elbow, and Firefly algorithms.

Improved SOSK-Means Automatic Clustering Algorithm with a Three-Part Mutualism Phase and Random Weighted Reflection Coefficient for High-Dimensional Datasets

Automatic clustering problems require clustering algorithms to automatically estimate the number of clusters in a dataset. However, the classical K-means requires the specification of the required number of clusters a priori. To address this problem, metaheuristic algorithms are hybridized with K-means to extend the capacity of K-means in handling automatic clustering problems. In this study, we proposed an improved version of an existing hybridization of the classical symbiotic organisms search algorithm with the classical K-means algorithm to provide robust and optimum data clustering performance in automatic clustering problems. Moreover, the classical K-means algorithm is sensitive to noisy data and outliers; therefore, we proposed the exclusion of outliers from the centroid update’s procedure, using a global threshold of point-to-centroid distance distribution for automatic outlier detection, and subsequent exclusion, in the calculation of new centroids in the K-means phase. Furthermore, a self-adaptive benefit factor with a three-part mutualism phase is incorporated into the symbiotic organism search phase to enhance the performance of the hybrid algorithm. A population size of 40+2g was used for the symbiotic organism search (SOS) algorithm for a well distributed initial solution sample, based on the central limit theorem that the selection of the right sample size produces a sample mean that approximates the true centroid on Gaussian distribution. The effectiveness and robustness of the improved hybrid algorithm were evaluated on 42 datasets. The results were compared with the existing hybrid algorithm, the standard SOS and K-means algorithms, and other hybrid and non-hybrid metaheuristic algorithms. Finally, statistical and convergence analysis tests were conducted to measure the effectiveness of the improved algorithm. The results of the extensive computational experiments showed that the proposed improved hybrid algorithm outperformed the existing SOSK-means algorithm and demonstrated superior performance compared to some of the competing hybrid and non-hybrid metaheuristic algorithms.

Enhanced Firefly-K-Means Clustering with Adaptive Mutation and Central Limit Theorem for Automatic Clustering of High-Dimensional Datasets

Metaheuristic algorithms have been hybridized with the standard K-means to address the latter’s challenges in finding a solution to automatic clustering problems. However, the distance calculations required in the standard K-means phase of the hybrid clustering algorithms increase as the number of clusters increases, and the associated computational cost rises in proportion to the dataset dimensionality. The use of the standard K-means algorithm in the metaheuristic-based K-means hybrid algorithm for the automatic clustering of high-dimensional real-world datasets poses a great challenge to the clustering performance of the resultant hybrid algorithms in terms of computational cost. Reducing the computation time required in the K-means phase of the hybrid algorithm for the automatic clustering of high-dimensional datasets will inevitably reduce the algorithm’s complexity. In this paper, a preprocessing phase is introduced into the K-means phase of an improved firefly-based K-means hybrid algorithm using the concept of the central limit theorem to partition the high-dimensional dataset into subgroups of randomly formed subsets on which the K-means algorithm is applied to obtain representative cluster centers for the final clustering procedure. The enhanced firefly algorithm (FA) is hybridized with the CLT-based K-means algorithm to automatically determine the optimum number of cluster centroids and generate corresponding optimum initial cluster centroids for the K-means algorithm to achieve optimal global convergence. Twenty high-dimensional datasets from the UCI machine learning repository are used to investigate the performance of the proposed algorithm. The empirical results indicate that the hybrid FA-K-means clustering method demonstrates statistically significant superiority in the employed performance measures and reducing computation time cost for clustering high-dimensional dataset problems, compared to other advanced hybrid search variants.

Boosting k-means clustering with symbiotic organisms search for automatic clustering problems.

Kmeans clustering algorithm is an iterative unsupervised learning algorithm that tries to partition the given dataset into k pre-defined distinct non-overlapping clusters where each data point belongs to only one group. However, its performance is affected by its sensitivity to the initial cluster centroids with the possibility of convergence into local optimum and specification of cluster number as the input parameter. Recently, the hybridization of metaheuristics algorithms with the K-Means algorithm has been explored to address these problems and effectively improve the algorithm’s performance. Nonetheless, most metaheuristics algorithms require rigorous parameter tunning to achieve an optimum result. This paper proposes a hybrid clustering method that combines the well-known symbiotic organisms search algorithm with K-Means using the SOS as a global search metaheuristic for generating the optimum initial cluster centroids for the K-Means. The SOS algorithm is more of a parameter-free metaheuristic with excellent search quality that only requires initialising a single control parameter. The performance of the proposed algorithm is investigated by comparing it with the classical SOS, classical K-means and other existing hybrids clustering algorithms on eleven (11) UCI Machine Learning Repository datasets and one artificial dataset. The results from the extensive computational experimentation show improved performance of the hybrid SOSK-Means for solving automatic clustering compared to the standard K-Means, symbiotic organisms search clustering methods and other hybrid clustering approaches.

On exploring data lakes by finding compact, isolated clusters

• Data lakes store unprocessed business data at large scale. • Clustering helps data engineers understand the structure of their data lakes. • RóMULO is a meta-heuristic multi-way clustering proposal to cluster data lakes. • The results confirm that RóMULO is a promising contribution to assist data engineers. Data engineers are very interested in data lake technologies due to the incredible abundance of datasets. They typically use clustering to understand the structure of the datasets before applying other methods to infer knowledge from them. This article presents the first proposal that explores how to use a meta-heuristic to address the problem of multi-way single-subspace automatic clustering, which is very appropriate in the context of data lakes. It was confronted with five strong competitors that combine the state-of-the-art attribute selection proposal with three classical single-way clustering proposals, a recent quantum-inspired one, and a recent deep-learning one. The evaluation focused on exploring their ability to find compact and isolated clusterings as well as the extent to which such clusterings can be considered good classifications . The statistical analyses conducted on the experimental results prove that it ranks the first regarding effectiveness using six standard coefficients and it is very efficient in terms of CPU time, not to mention that it did not result in any degraded clusterings or timeouts . Summing up: this proposal contributes to the array of techniques that data engineers can use to explore their data lakes.

Automatic Data Clustering Using Hybrid Chaos Game Optimization with Particle Swarm Optimization Algorithm.

In cluster analysis, classical approaches suffer from the problem of identifying the number of clusters, known as the automatic clustering problem. Therefore, automatic clustering has become a popular research area and offers opportunities in various data analysis applications such as bioinformatics, medicine, image processing and consumer segmentation. It is considered as NP- complete problem where it is preferable to use approximate approaches. In this study, we propose an hybrid approach between chaos game optimization and particle swarm optimization (CGOPSO). The Davies-Bouldin index (DBI) is used as a main objective of the proposed approach with the purpose to find the most accurate number of cluster centroids and their positions. To assess its performance, we compared CGOPSO with different other existing algorithms in the literature over 12 classical datasets using two different validity indexes: Davies Bouldin index (DBI) and Compact-Seperated index (CSI). The experimental results have demonstrated that CGOPSO shows better performance than other algorithms.

K-Means-Based Nature-Inspired Metaheuristic Algorithms for Automatic Data Clustering Problems: Recent Advances and Future Directions

K-means clustering algorithm is a partitional clustering algorithm that has been used widely in many applications for traditional clustering due to its simplicity and low computational complexity. This clustering technique depends on the user specification of the number of clusters generated from the dataset, which affects the clustering results. Moreover, random initialization of cluster centers results in its local minimal convergence. Automatic clustering is a recent approach to clustering where the specification of cluster number is not required. In automatic clustering, natural clusters existing in datasets are identified without any background information of the data objects. Nature-inspired metaheuristic optimization algorithms have been deployed in recent times to overcome the challenges of the traditional clustering algorithm in handling automatic data clustering. Some nature-inspired metaheuristics algorithms have been hybridized with the traditional K-means algorithm to boost its performance and capability to handle automatic data clustering problems. This study aims to identify, retrieve, summarize, and analyze recently proposed studies related to the improvements of the K-means clustering algorithm with nature-inspired optimization techniques. A quest approach for article selection was adopted, which led to the identification and selection of 147 related studies from different reputable academic avenues and databases. More so, the analysis revealed that although the K-means algorithm has been well researched in the literature, its superiority over several well-established state-of-the-art clustering algorithms in terms of speed, accessibility, simplicity of use, and applicability to solve clustering problems with unlabeled and nonlinearly separable datasets has been clearly observed in the study. The current study also evaluated and discussed some of the well-known weaknesses of the K-means clustering algorithm, for which the existing improvement methods were conceptualized. It is noteworthy to mention that the current systematic review and analysis of existing literature on K-means enhancement approaches presents possible perspectives in the clustering analysis research domain and serves as a comprehensive source of information regarding the K-means algorithm and its variants for the research community.

Automatic Clustering of DNA Sequences With Intelligent Techniques

With the discovery of new DNAs, a fundamental problem arising is how to categorize those DNA sequences into correct species. Unfortunately, identifying all data groups correctly and assigning a set of DNAs into k clusters where k must be predefined are one of the major drawbacks in clustering analysis, especially when the data have many dimensions and the number of clusters is too large and hard to guess. Furthermore, finding a similarity measure that preserves the functionality and represents both the composition and distribution of the bases in a DNA sequence is one of the main challenges in computational biology. In this paper, a new soft computing metaheuristic framework is introduced for automatic clustering to generate the optimal cluster formation and to determine the best estimate for the number of clusters. Pulse coupled neural network (PCNN) is utilized for the calculation of DNA sequence similarity or dissimilarity. Bat algorithm is hybridized with the well-known genetic algorithm to solve the automatic data clustering problem. Extensive computational experiments are conducted on the expanded human oral microbiome database (eHOMD). A comparative study between the experimental results shows that the proposed hybrid algorithm achieved superior performance over the standard genetic algorithm and bat algorithm. Moreover, the hybrid performance was compared with competing algorithms from the literature review to ascertain its superiority. Mann-Whitney-Wilcoxon rank-sum test is conducted to statistically validate the obtained clusters.

Modified ISR hyper-heuristic for tuning automatic genetic clustering chromosome size

Recent works using hyper-heuristics for solving clustering problems have been focusing on Genetic Algorithm. However, to the best of this research knowledge, no work is using hyper-heuristics dedicated for tuning the Genetic algorithm`s chromosome size for automatic clustering problem. The ability to tune the chromosome size is important because it allows the automatic clustering algorithm to be adaptive and dynamic. This paper proposes and evaluates a modified Improvement Selection Rules hyper-heuristic algorithm for tuning automatic genetic clustering chromosome size. The paper reviews related works of Genetic algorithm`s parameters tuning and selective hyper-heuristic algorithms and proposes a modified algorithm. The Iris, Breast Cancer, Wine and E-coli datasets are used for evaluation of the algorithm, based on the fitness, accuracy and robustness. The results indicate that the hyper-heuristic algorithm has produced good performance (fitness) and accuracy but consume considerably higher execution times.

Nature-inspired metaheuristic techniques for automatic clustering: a survey and performance study

The application of several swarm intelligence and evolutionary metaheuristic algorithms in data clustering problems has in the past few decades gained wide popularity and acceptance due to their success in solving and finding good quality solutions to a variety of complex real-world optimization problems. Clustering is considered one of the most important data analysis techniques in the domain of data mining. A clustering problem refers to the partitioning of unlabeled data objects into a certain number of clusters based on their attribute values or features, with the objective of maximizing intra-clusters homogeneity and inter-cluster heterogeneity. This paper presents an up-to-date survey of major nature-inspired metaheuristic algorithms that have been employed to solve automatic clustering problems. Further, a comparative study of several modified well-known global metaheuristic algorithms is carried out to solve automatic clustering problems. Also, three hybrid swarm intelligence and evolutionary algorithms, namely, particle swarm differential evolution algorithm, firefly differential evolution algorithm and invasive weed optimization differential evolution algorithm, are proposed to deal with the task of automatic data clustering. In contrast to many of the existing traditional and evolutionary computational clustering techniques, the clustering algorithms presented in this paper do not require any predetermined information or prior-knowledge of the dataset that is to be classified, but rather they are capable of spontaneously identifying the optimal number of partitions of the data points during the course of program execution. Forty-one benchmarked datasets that comprise eleven artificial and thirty real world datasets are collated and utilized to evaluate the performances of the representative nature-inspired clustering algorithms. According to the extensive experimental results, comparisons and statistical significance, the firefly algorithm appeared to be more appropriate for better clustering of both low and high dimensional data objects than were other state-of-the-art algorithms. Further, an experimental study demonstrates the superiority of the three proposed hybrid algorithms over the standard state-of-the-art methods in finding meaningful clustering solutions to the problem at hand.

Автоматическая кластеризация документов СМИ на основе анализа их смыслового содержания

The article describes the solution to the problem of automatic clustering of media documents based on the analysis of their semantic analysis. The proposed solution is based on the methods of machine grammar, semantic-syntactic and conceptual analysis of texts, as well as methods for identifying the conceptual composition of a collection of documents and formalizing the semantic content of texts. The developed algorithm of the document clustering process provides for the possibility of its implementation in a fully automatic mode without prior machine learning.

Novel quantum inspired approaches for automatic clustering of gray level images using Particle Swarm Optimization, Spider Monkey Optimization and Ageist Spider Monkey Optimization algorithms

Abstract This paper is intended to identify the optimal number of clusters automatically from an image dataset using some quantum behaved nature inspired meta-heuristic algorithms. Due to the lack of sufficient information, it is difficult to identify the appropriate number of clusters from a dataset, which has enthused the researchers to solve the problem of automatic clustering and to open up a new era of cluster analysis with the help of several natures inspired meta-heuristic algorithms. In this paper, three quantum inspired meta-heuristic techniques, viz., Quantum Inspired Particle Swarm Optimization (QIPSO), Quantum Inspired Spider Monkey Optimization (QISMO) and Quantum Inspired Ageist Spider Monkey Optimization (QIASMO), have been proposed. A comparison has been outlined between the quantum inspired algorithms with their corresponding classical counterparts. The efficiency of the quantum inspired algorithms has been established over their corresponding classical counterparts with regards to fitness, mean, standard deviation, standard errors of fitness, convergence curves (for benchmarked mathematical functions) and computational time. Finally, the results of two statistical superiority tests, viz., t- test and Friedman test have been provided to prove the superiority of the proposed methods. The superiority of the proposed methods has been established on five publicly available real life image datasets, five Berkeley image datasets of different dimensions and four benchmark mathematical functions both visually and quantitatively.

Evolutionary multi-objective automatic clustering enhanced with quality metrics and ensemble strategy

Abstract Automatic clustering problem, which needs to detect the appropriate clustering without a pre-defined number of clusters (k), is difficult and challenging in unsupervised learning owing to the lack of prior domain knowledge. Despite a rising tendency with the application of evolutionary multi-objective optimization (EMO) techniques for automatic clustering, there still exist some obvious under-explored issues. In this paper, we resort to quality metrics and ensemble strategy for the sake of explicit/implicit knowledge discovery to guide the optimization process. The quality and diversity of solutions defined in terms of cluster validities, as similar to performance indicator for multi-objective optimization, are applied to assist in addressing automatic clustering problems and decreasing unnecessary computational overhead. To be specific, the main components like initialization, reproduction operations, and environmental selection which involved during EMO based automatic clustering are discussed and refined. For the determination of the final partitioning, quality metrics and cluster ensemble strategy are both considered to improve the retrieve system in the unsupervised way. Experiments are conducted from several different aspects and the corresponding analyses are provided, which confirm that the proposals are more efficient and effective for automatic clustering.

Automatic Data Clustering Using Hybrid Firefly Particle Swarm Optimization Algorithm

The firefly algorithm is a nature-inspired metaheuristic optimization algorithm that has become an important tool for solving most of the toughest optimization problems in almost all areas of global optimization and engineering practices. However, as with other metaheuristic algorithms, the performance of the firefly algorithm depends on adequate parameter tuning. In addition, its diversification as a global metaheuristic can lead to reduced speed, as well as an associated decrease in the rate of convergence when applied to solve problems with large number of variables such as data clustering problems. Clustering is an unsupervised data analysis technique used for identifying homogeneous groups of objects based on the values of their attributes. To mitigate the aforementioned drawbacks, an improved firefly algorithm is hybridized with the well-known particle swarm optimization algorithm to solve automatic data clustering problems. To investigate the performance of the proposed hybrid algorithm, it is compared with four popular metaheuristic methods from literature using twelve standard datasets from the UCI Machine Learning Repository and the two moons dataset. The extensive computational experiments and results analysis carried out shows that the proposed algorithm not only achieves superior performance over the standard firefly and particle swarm optimization algorithms, but also exhibits high level of stability and can be efficiently utilized to solve other clustering problems with high dimensionality.

A Hybrid Heuristic with Hopkins Statistic for the Automatic Clustering Problem

Cluster Analysis is a multivariate method to handle real problems associated with several fields. This area combines several methods of unsupervised classification, which can be applied in order to identify groups in a data set. The Clustering Problems are classified as NP-Hard and, in order to obtain such classification, the number of groups k may be fixed or, in the Automatic approach, the ideal k must be identified upon evaluation of some validation index. In this paper the Silhouette Index was considered and a new proposed Hybrid Heuristic Algorithm (HHA) operates to identify the ideal number of groups. The HHA consider two heuristic algorithms based on metaheuristics: an algorithm based on Iterated Local Search (ILS) that considers a density-based approach and a literature Evolutionary Algorithm (EA). Besides, the HHA have a heuristic algorithm that verify clustering tendency, considering the Hopkins Statistic. Basically, according with the clustering tendency level, the HHA use a specific heuristic (ILS or EA). The computational experiments used three literature data sets with eighty-two instances, and all of them were considered and reported by different researchers. The effectiveness and the efficiency of the proposed heuristic are reflected in substantially lower computational time and in the solutions quality, that are competitive when compared with the best results reported in the literature.

A novel combinatorial merge-split approach for automatic clustering using imperialist competitive algorithm

Abstract Cluster analysis has a wide application in many areas, including pattern recognition, information retrieval, and image processing. In most real-world clustering problems, the number of clusters must be predetermined. Automatic clustering is a promising solution for this challenge which automatically determines the number and structure of clusters in data. In recent years, the evolutionary algorithm due to their search mechanisms has been popular in solving automatic clustering problems. Imperialist Competitive Algorithm (ICA) is a successful evolutionary algorithm. In this paper, for the first time, Imperialist Competitive Algorithm (ICA) is used for solving automatic clustering problems, called “the automatic clustering using ICA (AC-ICA)”. In the proposed algorithm, in order to increase the exploration ability, the movement of colonies toward the imperialist was changed at the assimilation step. A new method has been provided for changing the number of clusters by combining random and homogeneity based merge-split approach. Furthermore, an efficient method based on density has been proposed for reinitializing empty cluster centers. To use AC-ICA in automatic clustering, the initialization and imperialist competition steps were changed. Based on changes in these two steps, a framework was provided for changing different types of ICA to solve automatic clustering problems. Then, the basic ICA and its three recently developed types, were changed by this framework and their performances in automatic clustering were compared to AC-ICA. The examinations were done on six synthetic and ten real word data sets. The comparison of the proposed algorithm's results with basic ICA, its three recently developed types and several state-of-art automatic clustering methods, shows AC-ICA's superiority in terms of the speed of convergence to the optimal solution and quality of the obtained solution. We also applied our algorithm to a real world application (i.e., face recognition) and the achieved results were acceptable.

An improved differential evolution with cluster decomposition algorithm for automatic clustering

This study proposes an improved differential evolution (DE) algorithm to solve an automatic clustering problem. Automatic clustering is a clustering technique in data mining which can automatically define the best number of clusters, as well as construct those clusters. In order to overcome the weakness of the DE algorithm for automatic clustering, some improvement scenarios are proposed. They include the best solution effect, saturated solution, acceleration and handling downhill scenarios. The best solution effect and acceleration factor are proposed to boost the convergence of the DE algorithm. The saturated solution helps the algorithm maintain the diversity of the population. The handling downhill scenario allows chromosomes to move to a worse solution, since it might later lead to a better solution. The proposed algorithm is evaluated using four well-known datasets. The computational results indicate that the proposed improvements can enhance the DE algorithm’s performance. In addition, the proposed method is also applied to cluster customers for a ladies’ office wear clothing franchise. The results can be used to plan a marketing strategy for the case company.

Automatic clustering using an improved artificial bee colony optimization for customer segmentation

In cluster analysis, determining number of clusters is an important issue because information about the most appropriate number of clusters do not exist in the real-world problems. Automatic clustering is a clustering approach which is able to automatically find the most suitable number of clusters as well as divide the instances into the corresponding clusters. This study proposes a novel automatic clustering algorithm using a hybrid of improved artificial bee colony optimization algorithm and K-means algorithm (iABC). The proposed iABC algorithm improves the onlooker bee exploration scheme by directing their movements to a better location. Instead of using a random neighborhood location, the improved onlooker bee considers the data centroid to find a better initial centroid for the K-means algorithm. To increase efficiency of the improvement, the updating process is only applied on the worst cluster centroid. The proposed iABC algorithm is verified using some benchmark datasets. The computational result indicates that the proposed iABC algorithm outperforms the original ABC algorithm for automatic clustering problem. Furthermore, the proposed iABC algorithm is utilized to solve the customer segmentation problem. The result reveals that the iABC algorithm has better and more stable result than original ABC algorithm.

A Knowledge Context Fuzzy Clustering Method Based on Genetic Algorithm

A fuzzy clustering method based on genetic algorithm is proposed aiming at the problem of automatic clustering of knowledge context. Firstly, the knowledge context model is constructed to determine the similarity measure of knowledge context. Then the initial clustering centers are obtained based on the density peak method. Then the fuzzy C mean clustering result is solved by genetic algorithm, and the clustering of knowledge context is realized. Finally, the knowledge context clustering of an aircraft part design process is taken as an example to illustrate the effectiveness of the algorithm.

Automatic Data Clustering Using Parameter Adaptive Harmony Search Algorithm and Its Application to Image Segmentation

AbstractIn this paper, the problem of automatic data clustering is treated as the searching of optimal number of clusters so that the obtained partitions should be optimized. The automatic data clustering technique utilizes a recently developed parameter adaptive harmony search (PAHS) as an underlying optimization strategy. It uses real-coded variable length harmony vector, which is able to detect the number of clusters automatically. The newly developed concepts regarding “threshold setting” and “cutoff” are used to refine the optimization strategy. The assignment of data points to different cluster centers is done based on the newly developed weighted Euclidean distance instead of Euclidean distance. The developed approach is able to detect any type of cluster irrespective of their geometric shape. It is compared with four well-established clustering techniques. It is further applied for automatic segmentation of grayscale and color images, and its performance is compared with other existing techniques. For real-life datasets, statistical analysis is done. The technique shows its effectiveness and the usefulness.