Numerous Clustering Algorithms Research Articles

Significance-based decision tree for interpretable categorical data clustering

Numerous clustering algorithms prioritize accuracy, but in high-risk domains, the interpretability of clustering methods is crucial as well. The inherent heterogeneity of categorical data makes it particularly challenging for users to comprehend clustering outcomes. Currently, the majority of interpretable clustering methods are tailored for numerical data and utilize decision tree models, leaving interpretable clustering for categorical data as a less explored domain. Additionally, existing interpretable clustering algorithms often depend on external, potentially non-interpretable algorithms and lack transparency in the decision-making process during tree construction. In this paper, we tackle the problem of interpretable categorical data clustering by growing a decision tree in a statistically meaningful manner. We formulate the evaluation of candidate splits as a multivariate two-sample testing problem, where a single p-value is derived by combining significance evidence from all individual categories. This approach provides a reliable and controllable method for selecting the optimal split while determining its statistical significance. Extensive experimental results on real-world data sets demonstrate that our algorithm achieves comparable performance in terms of cluster quality, running efficiency, and explainability relative to its counterparts.

Robust deep image clustering using convolutional autoencoder with separable discrete Krawtchouk and Hahn orthogonal moments

By cooperatively learning features and assigning clusters, deep clustering is superior to conventional clustering algorithms. Numerous deep clustering algorithms have been developed for a variety of application levels; however, the majority are still incapable of learning robust noise-resistant latent features, which limits the clustering performance. To address this open research challenge, we introduce, for the first time, a new approach called: Robust Deep Embedded Image Clustering algorithm with Separable Krawtchouk and Hahn Moments (RDEICSKHM). Our approach leverages the advantages of Krawtchouk and Hahn moments, such as local feature extraction, discrete orthogonality, and noise tolerance, to obtain a meaningful and robust image representation. Moreover, we employ LayerNormalization to further improve the latent space quality and facilitate the clustering process. We evaluate our approach on four image datasets: MNIST, MNIST-test, USPS, and Fashion-MNIST. We compare our method with several deep clustering methods based on two metrics: clustering accuracy (ACC) and normalized mutual information (NMI). The experimental results show that our method achieves superior or competitive performance on all datasets, demonstrating its effectiveness and robustness for deep image clustering.

Adaptive multi-view subspace learning based on distributed optimization

As the rapid development of Internet of Things (IoT), information is collected from different sensors and stored in distributed devices which can be regarded as the multi-view data. There are currently numerous clustering algorithms designed to handle multi-view data. However, most of these algorithms still suffer from the following problems: They are designed to operate directly on raw data, which preserves excessive redundant information. They primarily focus on pairwise relationships between views, neglecting the intricate high-order connections among multiple views. The prior information of singular values is not taken into account in multiple views and different views are considered to have equal contributions for clustering. To efficiently address above problems, adaptive multi-view subspace learning based on distributed optimization (AMSLDO) is proposed in this paper. Specifically, the original multi-view data is projected to a low-dimensional space for subspace representation, and multiple representation matrices are stacked in a tensor with weighted tensor nuclear norm to obtain high-order correlations and discover the prior information of singular values. Furthermore, the consensus matrix is learned from different representation matrices through adaptive weights. Meanwhile, samples are partitioned into the ideal number of clusters through Laplacian rank constraint. An efficient distributed optimization algorithm based on the Alternating Direction Method of Multipliers (ADMM) framework is designed to solve the proposed model. Extensive experiments are conducted on six datasets, demonstrating the superiority of the proposed model compared with eleven state-of-the-art methods.

Melanoma Skin Cancer Segmentation using Robust Multi-View Fuzzy C- Means Algorithm

Skin cancer has become a significant issue and the primary cause of death for people all over the world. Melanoma is one of the most serious types of skin cancer since it can spread to any area of the body. So, the prognosis of melanoma in its early stages is very important for figuring out how likely it is that the patient will get better. Because of this, clinical imaging science is a very important part of finding certain types of skin lesions quickly and correctly. In this study, we showed a new way to use digital image processing to separate skin lesions in dermoscopic images. First, the image is preprocessed with a median filter to eliminate extra hair, noise, and artefacts in order to create a better image for segmentation. To improve the accuracy of dermoscopic images, the preprocessed images are segmented based on an innovative method built on an improved robust Multiview Fuzzy C-Means clustering algorithm (RMV-FCM). Following segmentation, features are extracted using the ABCD rule to produce optimal features that may be used as input for classification. Lastly, the Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Random Forest (RF), and Nave Bayes classifiers use machine learning techniques to characterize skin lesions as malignant and benign (NB). The RMV-FCM algorithm exhibits more adaptability and higher clustering performance when compared to a variety of related clustering techniques. In terms of detection accuracy, as noted in the conclusion of this research, the novelty of the work shows that RMV-FCM is exceptional in comparison to numerous related clustering algorithms, and SVM is exceptional in comparison to other standard classifiers. The simulated outcome indicates that the suggested strategy accurately diagnoses skin tumors with a 97.5% accuracy rate. The results of the suggested segmentation are extremely accurate when compared to other algorithms in the same domain

Let the Data Choose: Flexible and Diverse Anchor Graph Fusion for Scalable Multi-View Clustering

In the past few years, numerous multi-view graph clustering algorithms have been proposed to enhance the clustering performance by exploring information from multiple views. Despite the superior performance, the high time and space expenditures limit their scalability. Accordingly, anchor graph learning has been introduced to alleviate the computational complexity. However, existing approaches can be further improved by the following considerations: (i) Existing anchor-based methods share the same number of anchors across views. This strategy violates the diversity and flexibility of multi-view data distribution. (ii) Searching for the optimal anchor number within hyper-parameters takes much extra tuning time, which makes existing methods impractical. (iii) How to flexibly fuse multi-view anchor graphs of diverse sizes has not been well explored in existing literature. To address the above issues, we propose a novel anchor-based method termed Flexible and Diverse Anchor Graph Fusion for Scalable Multi-view Clustering (FDAGF) in this paper. Instead of manually tuning optimal anchor with massive hyper-parameters, we propose to optimize the contribution weights of a group of pre-defined anchor numbers to avoid extra time expenditure among views. Most importantly, we propose a novel hybrid fusion strategy for multi-size anchor graphs with theoretical proof, which allows flexible and diverse anchor graph fusion. Then, an efficient linear optimization algorithm is proposed to solve the resultant problem. Comprehensive experimental results demonstrate the effectiveness and efficiency of our proposed framework. The source code is available at https://github.com/Jeaninezpp/FDAGF.

Consistency-Induced Multiview Subspace Clustering.

Multiview clustering has received great attention and numerous subspace clustering algorithms for multiview data have been presented. However, most of these algorithms do not effectively handle high-dimensional data and fail to exploit consistency for the number of the connected components in similarity matrices for different views. In this article, we propose a novel consistency-induced multiview subspace clustering (CiMSC) to tackle these issues, which is mainly composed of structural consistency (SC) and sample assignment consistency (SAC). To be specific, SC aims to learn a similarity matrix for each single view wherein the number of connected components equals to the cluster number of the dataset. SAC aims to minimize the discrepancy for the number of connected components in similarity matrices from different views based on the SAC assumption, that is, different views should produce the same number of connected components in similarity matrices. CiMSC also formulates cluster indicator matrices for different views, and shared similarity matrices simultaneously in an optimization framework. Since each column of similarity matrix can be used as a new representation of the data point, CiMSC can learn an effective subspace representation for the high-dimensional data, which is encoded into the latent representation by reconstruction in a nonlinear manner. We employ an alternating optimization scheme to solve the optimization problem. Experiments validate the advantage of CiMSC over 12 state-of-the-art multiview clustering approaches, for example, the accuracy of CiMSC is 98.06% on the BBCSport dataset.

Learning a bi-directional discriminative representation for deep clustering

Nowadays, deep clustering achieves superior performance by jointly performing representation learning and cluster assignment. Although numerous deep clustering algorithms have emerged, most of them have difficulty learning representations that fit the clustering distribution. To address this issue, we propose a bi-directional discriminative representation learning clustering (BDRC) framework in this paper. In our framework, a dual autoencoder network, a bi-directional mutual information maximization module and a self-supervised cluster prediction module are combined into a joint optimization framework. To learn more cluster-friendly representations, the bi-directional mutual information maximization module is executed on both samples and their nearest neighbors to explore the cluster relationships between samples. In order to improve the stability of the model, a self-supervised cluster prediction module is devised to predict clustering assignments to supervise the autoencoder using the KL-divergence. Moreover, the UMAP is used to find the manifold of the latent representations which can better preserve the global structure. Experiments on some benchmark datasets demonstrate the superiority of the proposed BDRC algorithm.

How does a combination of numerical modeling, clustering, artificial intelligence, and evolutionary algorithms perform to predict regional groundwater levels?

The prediction of groundwater levels in arid and semi-arid regions is of great importance to tailor the best water management strategies. In this study, we propose a new approach that combines simulation, clustering, and optimization tools for groundwater level prediction. This approach simulates groundwater levels (GWL) using the MODFLOW method, clusters the study aquifer into different clusters using the k-mean method, and predicts regional GWL using the artificial neural network (ANN) and adaptive neuro-fuzzy inference system (ANFIS) methods that were optimized by the Harris Hawks Optimization (HHO), Whale Optimization Algorithm (WOA), and Particle Swarm Optimization (PSO). The efficacy of our approach was evaluated via a case study in northwest Iran. The MODFLOW method simulated the distribution of GWL across the study area with R2 = 0.99, root mean square error (RMSE) = 0.97 m (m) and mean absolute error (MAE) = 0.82 m. The k-means method clustered the aquifer into seven clusters based on the hydraulic conductivity, storage coefficient, groundwater level, groundwater depth, groundwater withdrawal, and aquifer saturation thickness parameters. The prediction of groundwater level for each cluster demonstrated the accurate performance of all optimized models with mean RMSE = 0.6 m and mean absolute percentage error (MAPE) = 0.23 m. The prediction phase identified groundwater level in the previous month (obtained from the MODFLOW method), withdrawal of aquifer, precipitation, temperature, and evaporation as the most influential variables for groundwater levels in different clusters. We recommend the methodology proposed here for the prediction of groundwater levels in different aquifers with heterogeneous characteristics that pose computational burdens and uncertainties.

A split–merge clustering algorithm based on the k-nearest neighbor graph

Numerous graph-based clustering algorithms relying on k-nearest neighbor (KNN) have been proposed. However, the performance of these algorithms tends to be affected by many factors such as cluster shape, cluster density and outliers. To address these issues, we present a split–merge clustering algorithm based on the KNN graph (SMKNN), which is based on the idea that two adjacent clusters can be merged if the data points located in the connection layers of the two clusters tend to be consistent in distribution. In Stage 1, a KNN graph is constructed. In Stage 2, the subgraphs are obtained by removing the pivot points from the KNN graph, in which the pivot points are determined by the size of local distance ratio of data points. In Stage 3, the adjacent cluster pairs satisfying the maximum similarity are merged, in which the similarity measure of two clusters is designed with two concepts including external connection edges and internal connection edges. By the experiments on ten synthetic data sets and eight real data sets, we compared SMKNN with two traditional algorithms, two density-based algorithms, nine graph-based algorithms and four neural network based algorithms in accuracy. The experimental results demonstrate a good performance of the proposed clustering method.

EEIT2-F: energy-efficient aware IT2-fuzzy based clustering protocol in wireless sensor networks

<p><span>Improving the network lifetime is still a vital challenge because most wireless sensor networks (WSNs) run in an unreached environment, and offer almost impossible human access and tracking. Clustering is one of the most effective methods for ensuring that the relevant device process takes place to improve network scalability, decrease energy consumption and maintain an extended network lifetime. Many research have been developed on the numerous effective clustering algorithms to address this problem. Such algorithms almost dominate on the cluster head (CH) selection and cluster formation; using the intelligent type1 fuzzy-logic (T1-FL) scheme. In this paper, we suggest an interval type2 FL (IT2-FL) methodology that assumes uncertain levels of a decision to be more efficient than the T1-FL model. It is the so-called energy-efficient interval type2 fuzzy (EEIT2-F) low energy adaptive clustering hierarchical (LEACH) protocol. The IT2-FL system depends on three inputs of the residual energy of each node, the node distance from the base station (sink node), and the centrality of each node. Accordingly, the simulation results show that the suggested clustering protocol outperforms the other existing proposals in terms of energy consumption and network lifetime.</span></p>

Learning missing instances in latent space for incomplete multi-view clustering

Real world objects can usually be described from multiple views. How to combine multiple views to make better use of these data has attracted a lot of attentions, and numerous multi-view clustering algorithms have been proposed. However, the data in real scenes are often incomplete. We propose an incomplete multi-view clustering method (MISS), which simultaneously learns Missing Instances and Self-representations in latent Space. Instead of filling missing instances in the original data space, we learn missing instances and self-representations in the latent space so that we can grasp the features of missing instances, while preserving the original latent spatial structure of the data. In addition, a novel fusion matrix is introduced. The missing instances are represented by linear combinations of the known instances in the latent space. This fusion matrix will be learned simultaneously according to the learning of self-representation in the latent space for multi-view clustering. Furthermore, a completeness constraint is proposed to guarantee that learning direction of missing instances is close to the original data as possible. Because of the focus on the features of missing instances and the keeping of the original data structure, MISS can not only handle randomly data missing, where some instances are randomly removed from each view, but also continuous data missing, i.e., all instances of some clusters are missing completely. Experimental results on seven datasets show that MISS achieves improvements over the state-of-the-art incomplete multi-view clustering methods.

An analysis framework for clustering algorithm selection with applications to spectroscopy.

Cluster analysis is a valuable unsupervised machine learning technique that is applied in a multitude of domains to identify similarities or clusters in unlabelled data. However, its performance is dependent of the characteristics of the data it is being applied to. There is no universally best clustering algorithm, and hence, there are numerous clustering algorithms available with different performance characteristics. This raises the problem of how to select an appropriate clustering algorithm for the given analytical purposes. We present and validate an analysis framework to address this problem. Unlike most current literature which focuses on characterizing the clustering algorithm itself, we present a wider holistic approach, with a focus on the user’s needs, the data’s characteristics and the characteristics of the clusters it may contain. In our analysis framework, we utilize a softer qualitative approach to identify appropriate characteristics for consideration when matching clustering algorithms to the intended application. These are used to generate a small subset of suitable clustering algorithms whose performance are then evaluated utilizing quantitative cluster validity indices. To validate our analysis framework for selecting clustering algorithms, we applied it to four different types of datasets: three datasets of homemade explosives spectroscopy, eight datasets of publicly available spectroscopy data covering food and biomedical applications, a gene expression cancer dataset, and three classic machine learning datasets. Each data type has discernible differences in the composition of the data and the context within which they are used. Our analysis framework, when applied to each of these challenges, recommended differing subsets of clustering algorithms for final quantitative performance evaluation. For each application, the recommended clustering algorithms were confirmed to contain the top performing algorithms through quantitative performance indices.

A Systematic Review of Density Grid-Based Clustering for Data Streams

Various applications, such as electronic business, satellite remote sensing, intrusion discovery, and network traffic monitoring, generate large unbounded data stream sequences at a rapid pace. The clustering of data streams has attracted considerable interest due to the increasing usage of evolving data streams. In particular, evolving data streams affect clustering because they introduce numerous challenges, such as time and memory limits and one-pass clustering. Furthermore, researchers need to be able to determine arbitrarily shaped clusters present in evolving data streams from applications. Due to these characteristics, conventional density grid-based clustering techniques cannot be used. Moreover, the existing density grid-based clustering algorithms have low cluster quality for clustering evolving data streams. This study conducted a systematic literature review (SLR) and noted numerous research-related issues encountered in solving the aforementioned problems. We summarized numerous grid-based clustering algorithms that have been used and determined their distinctive and limited features. We also observed how these algorithms address the challenges affecting the clustering of evolving data streams and studied their advantages and disadvantages. SLR was based on 104 articles published between 2010 and 2021. Numerous challenges remain for grid-based clustering algorithms, particularly in terms of time-limited and high-dimensional data handling. Last, our findings indicated a variety of active studies on density grid-based clustering.

Integration of axiomatic design and design structure matrix for the modular design of automobile parts

As the demands of customers in the modern industry increased, the number of products, and the variety of components has increased. These issues have led to difficulties in product development and production. Modularization of products has advantages such as cost reduction, product development time reduction, and production time reduction. Modular design of products has been studied in the design activities of the modern industry. In this study, a modular design method is proposed to design a modular product based on axiomatic design (AD) and design structure matrix (DSM). AD and DSM are efficiently integrated into the proposed method. Functional requirements and design parameters are defined based on the Independence Axiom of AD, and the zigzagging process of AD is employed for the decomposition of the functional requirements (FRs) and design parameters (DPs). The design sequence is established based on the design matrix. Coupled or functionally close DPs are grouped into a module (Module 1). These modules are efficiently used in the design sequence. DSM is used to modularize the design parameters of the lowest level of axiomatic design. DSM is constructed based on physical interfaces and numerical clustering algorithms are used to identify strongly related components. They are grouped into a module (Module 2). Module 2 is exploited for production and management. Therefore, these two modules for different purposes can be used to increase efficiency in the design and production process. The proposed method is applied to two automobile parts such as the suspension system and cooling system. The results are discussed from the viewpoint of usefulness.

Multi-view document clustering based on geometrical similarity measurement

Numerous works implemented multi-view clustering algorithms in document clustering. A challenging problem in document clustering is the similarity metric. Existing multi-view document clustering methods broadly utilized two measurements: the Cosine similarity (CS) and the Euclidean distance (ED). The first did not consider the magnitude difference (MD) between the two vectors. The second can’t register the divergence of two vectors that offer a similar ED. In this paper, we originally created five models of similarity metric. This methodology foils the downside of the CS and ED similarity metrics by figuring the divergence between documents with the same ED while thinking about their sizes. Furthermore, we proposed our multi-view document clustering plan which dependent on the proposed similarity metric. Firstly, CS, ED, triangle’s area similarity and sector’s area similarity metric, and our five similarity metrics have been applied to every view of a dataset to generate a corresponding similarity matrix. Afterward, we ran clustering algorithms on these similarity matrices to evaluate the performance of single view. Later, we aggregated these similarity matrices to obtain a unified similarity matrix and apply spectral clustering algorithm on it to generate the final clusters. The experimental results show that the proposed similarity functions can gauge the similitude between documents more accurately than the existing metrics, and the proposed clustering scheme surpasses considerably up-to-date algorithms.

Density Peak Clustering Based on Relative Density Optimization

Among numerous clustering algorithms, clustering by fast search and find of density peaks (DPC) is favoured because it is less affected by shapes and density structures of the data set. However, DPC still shows some limitations in clustering of data set with heterogeneity clusters and easily makes mistakes in assignment of remaining points. The new algorithm, density peak clustering based on relative density optimization (RDO-DPC), is proposed to settle these problems and try obtaining better results. With the help of neighborhood information of sample points, the proposed algorithm defines relative density of the sample data and searches and recognizes density peaks of the nonhomogeneous distribution as cluster centers. A new assignment strategy is proposed to solve the abundance classification problem. The experiments on synthetic and real data sets show good performance of the proposed algorithm.

Exploration of Morphological Procedure on the Recognition of Fundus Image

A gainful fuzzy k-means clustering algorithm under Morphological Image Processing (MIP) is performed. Image processing is one of quickly developing examination territory nowadays and now it is particularly coordinated with all identified with science field. Image Processing can be utilized for breaking down various restorative and MRI Image to get the uncommon and anomaly in the image. Image segmentation manages segmentation of vein segmentation algorithm utilizing fundus Image. In this task, this segmentation is done utilizing k-means clustering and c-means clustering algorithm and Morphological operator for better execution. This upgrades the vein variations from the norm progressively and in a moderately brief time when contrasted with numerous other clustering algorithms.

A Novel Hierarchical Clustering Approach Based on Universal Gravitation

The target of the clustering analysis is to group a set of data points into several clusters based on the similarity or distance. The similarity or distance is usually a scalar used in numerous traditional clustering algorithms. Nevertheless, a vector, such as data gravitational force, contains more information than a scalar and can be applied in clustering analysis to promote clustering performance. Therefore, this paper proposes a three-stage hierarchical clustering approach called GHC, which takes advantage of the vector characteristic of data gravitational force inspired by the law of universal gravitation. In the first stage, a sparse gravitational graph is constructed based on the top k data gravitations between each data point and its neighbors in the local region. Then the sparse graph is partitioned into many subgraphs by the gravitational influence coefficient. In the last stage, the satisfactory clustering result is obtained by merging these subgraphs iteratively by using a new linkage criterion. To demonstrate the performance of GHC algorithm, the experiments on synthetic and real-world data sets are conducted, and the results show that the GHC algorithm achieves better performance than the other existing clustering algorithms.

Multi-view spectral clustering via sparse graph learning

Although numerous multi-view spectral clustering algorithms have been developed, most of them generally encounter the following two deficiencies. First, high time cost. Second, inferior operability. To this end, in this work we provide a simple yet effective method for multi-view spectral clustering. The main idea is to learn a consistent similarity matrix with sparse structure from multiple views. We show that proposed method is fast, straightforward to implement, and can achieve comparable or better clustering results compared to several state-of-the-art algorithms. Furthermore, the computation complexity of proposed method is approximately equivalent to the single-view spectral clustering. For these advantages, it can be considered as a baseline for multi-view spectral clustering.

Multi-kernel sparse subspace clustering on the Riemannian manifold of symmetric positive definite matrices

Recently, clustering in the Riemannian manifolds has received a great interest. The main specificity of this kind of spaces is their ability to detect nonlinear forms in the real world data groups. Accordingly, numerous Euclidean clustering algorithms have been emerged to the Riemannian framework. One of them is the Sparse Subspace Clustering (SSC) method. Within this context, an interesting method named Kernel Sparse Subspace Clustering in the Riemannian manifold (KSSCR) has been proposed. In order to improve the clustering performance of the SSC method, KSSCR is based on the idea of data projection into a Reproducing Kernel Hilbert Space (RKHS) through a Riemannian kernel. This modification in terms of projection space has yielded to significant clustering results compared to the original SSC. This paper proposed a Multi-Kernel SSCR (MKSSCR) method. The idea is to create a linear combination of a set of Riemannian kernels. In fact, by using a single kernel, the faraway points will be neglected. Thus, the key motivation of this work is to use a mixture of kernels. The purpose here is to emphasis the closest data points without ignoring the faraway ones in the process of selecting neighbors for each data point. Several experiments carried out on varied face clustering data sets demonstrate the clustering accuracy improvement by the proposed method compared to other state-of-the-art methods.