Data Clustering Algorithm Research Articles

Intuitive-K-prototypes: A mixed data clustering algorithm with intuitionistic distribution centroid

Data sets are usually mixed with numerical and categorical attributes in the real world. Data mining of mixed data makes a lot of sense. This paper proposes an Intuitive-K-prototypes clustering algorithm with improved prototype representation and attribute weights. The proposed algorithm defines intuitionistic distribution centroid for categorical attributes. In our approach, a heuristic search for initial prototypes is performed. Then, we combine the mean of numerical attributes and intuitionistic distribution centroid to represent the cluster prototype. In addition, intra-cluster complexity and inter-cluster similarity are used to adjust attribute weights, with higher priority given to those with lower complexity and similarity. The membership and non-membership distance are calculated using the intuitionistic distribution centroid. These distances are then combined parametrically to obtain the composite distance. The algorithm is judged for its clustering effectiveness on the real UCI data set, and the results show that the proposed algorithm outperforms the traditional clustering algorithm in most cases.

NS-IDBSCAN: An efficient incremental clustering method for geospatial data in network space

The exponential growth of big data presents a significant task for the incremental clustering problem. This paper proposes a density-based total clustering method in network space (NS-IDBSCAN) to cluster newly generated data. Instead of re-clustering the entire massive database, which would be much larger than the new datasets, we perform clustering with only the added data. Based on the current clustering results of the old data, the proposed algorithm checks the role of each newly added point and its neighbors in performing clustering. Depending on the density, an added point can be classified as noise, border, or core. This approach dramatically reduces the response time. Moreover, using a hash table with the key as the element's index corresponding to the point Id to store the cluster Id eliminates the need for searching by direct access, further increasing processing speed. Additionally, the proposed algorithm improves accuracy by reducing the intra-cluster distance when eliminating the phenomenon that the border points may belong to more than one cluster, as presented in the discussion. The proposed algorithm is compared with the spatial data clustering algorithm in network space. Test results on three data sources downloaded from OpenStreetMap, ESRI Open Data, and Inside Airbnb demonstrate that the proposed method significantly speeds up the processing time. The accuracy of the proposed algorithm is measured using eight indicators: Silhouette, BSS, WSS, WB, Davis-Bouldin, Dunn, Calinski-Harabasz, and NMI.

Quantitative evaluation of internal cluster validation indices using binary data sets

AbstractAimsDifferent clustering methods often classify the same data set differently. Selecting the “best” clustering solution from alternatives is possible with cluster validation indices. Because of the large variety of cluster validation indices (CVIs), choosing the most suitable index concerning the data set and clustering algorithms is challenging. We aim to assess different internal clustering validation indices.MethodsArtificial binary data sets with equal‐ and unequal‐sized well‐separated a priori clusters were simulated and three levels of noise were then added. Twenty replications of each of the six types of data sets (two group sizes × three levels of noise) were created and analyzed by three clustering algorithms with Jaccard dissimilarity. Twenty‐seven clustering validation indices are evaluated including both geometric and non‐geometric indices.ResultsAlthough, in theory, all CVIs could differentiate between good and wrong classifications, only a few perform as expected with noisy data. Tau and silhouette widths proved to be the best geometric CVIs both for equal and unequal cluster sizes. Among non‐geometric indices, crispness and OptimClass performed best.ConclusionWe recommend using these best‐performing CVIs. We suggest plotting the CVI value against the number of clusters because the lack of a sharp peak means that the position of the maximum is uncertain.

Automatic removal of large blood vasculature for objective assessment of brain tumors using quantitative dynamic contrast-enhanced magnetic resonance imaging.

The presence of a normal large blood vessel (LBV) in a tumor region can impact the evaluation of quantitative dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) parameters and tumor classification. Hence, there is a need for automatic removal of LBVs from brain tissues including intratumoral regions for achieving an objective assessment of tumors. This retrospective study included 103 histopathologically confirmed brain tumor patients who underwent MRI, including DCE-MRI data acquisition. Quantitative DCE-MRI analysis was performed for computing various parameters such as wash-out slope (Slope-2), relative cerebral blood volume (rCBV), relative cerebral blood flow (rCBF), blood plasma volume fraction (Vp), and volume transfer constant (Ktrans). An approach based on data-clustering algorithm, morphological operations, and quantitative DCE-MRI maps was proposed for the segmentation of normal LBVs in brain tissues, including the tumor region. Here, three widely used data-clustering algorithms were evaluated on two types of quantitative maps: (a) Slope-2, and (b) a new proposed combination of rCBV and Slope-2 maps. Fluid-attenuated inversion recovery-MRI hyperintense lesionswere also automatically segmented using deep learning-based architecture. The accuracy of LBV segmentation was qualitatively assessed blindly by two experienced observers, and Likert scoring was also obtained from each individual and compared using Cohen's Kappa test, and multiple statistical features from quantitative DCE-MRI parameters were obtained in the segmented tumor. t-test and receiver operating characteristic (ROC) curve analysis were performed for comparing the effect of removal of LBVs on parameters as well as on tumor grading. k-means clustering exhibited better accuracy and computational efficiency. Tumors, in particular high-grade gliomas (HGGs), showed a high contrast compared with normal tissues (relative % difference = 18.5%) on quantitative maps after the removal of LBVs. Statistical features (95th percentile values) of all parameters in the tumor region showed a statistically significant difference (p< 0.05) between with and without LBV maps. Similar results were obtained for the ROC curve analysis for differentiation between low-grade gliomas and HGGs. Moreover, after the removal of LBVs, the rCBV, rCBF, and Vp maps show better visualization of tumor regions.

Parameter extension of q-divergence-based fuzzy c-means clustering

AbstractIn this study, we propose a fuzzy clustering algorithm for vectorial data, which is constructed by extending the fuzzification parameters in the q-divergence-based fuzzy c-means algorithm (QFCM). The proposed algorithm, referred to as extended QFCM (eQFCM), is an extension of both QFCM and the penalized fuzzy c-means algorithm proposed by Yang, referred to as Y-type FCM (YFCM). eQFCM extends both the two-parameter QFCM and YFCM algorithms to a four-parameter model. Through numerical experiments using an artificial dataset, we substantiate the theoretical discussion, and the effects of fuzzification parameter to clustering results are observed. Furthermore, the results of some numerical experiments using real datasets are presented to demonstrate that the proposed algorithm outperformed both QFCM and YFCM algorithms in terms of clustering accuracy.

Optimizing Patient Stratification in Healthcare: A Comparative Analysis of Clustering Algorithms for EHR Data

Advanced data analytics are increasingly being employed in healthcare research to improve patient classification and personalize medicinal therapies. In this paper, we focus on the critical problem of clustering electronic health record (EHR) data to enable appropriate patient categorization. In the era of personalized medicine, optimizing patient classification is critical to healthcare analytics. This research presents a comparative assessment of different clustering algorithms for Electronic Health Record (EHR) data, with the goal of improving the efficacy and productivity of patient clustering methods. Our study focuses on Fuzzy Technique for Order of Preference by Similarity to Ideal Solution (Fuzzy TOPSIS) as a Multi-Criteria Decision-Making (MCDM) strategy, includes an in-depth assessment of eight clustering algorithms: K-Means, DBSCAN, Hierarchical Clustering, Mean Shift, Affinity Propagation, Spectral Clustering, Gaussian Mixture Models (GMM), as well as Self-Organizing Maps. The evaluation factors used for evaluation in this research are Cluster Quality Metrics, Scalability, Robustness to Noise, Cluster Shape and Density, Interpretability, Cluster Number, Dimensionality, and Consistency and Stability. These criteria and alternatives were chosen after conducting a thorough assessment of the literature and consulting with domain experts. All participated specialists actively engaged in the decision-making process, bringing unique insights into the best clustering algorithms for healthcare data. The results of this study illustrate each algorithm’s strengths and weaknesses in the setting of patient stratification, providing insight into their performance across multiple dimensions. The fuzzy TOPSIS MCDM strategy is a reliable instrument for synthesizing expert opinions and methodically evaluating the found clustering alternatives. This study advances healthcare analytics by giving practitioners and researchers with informative perspectives on the selection of clustering algorithms designed to address the unique problems of patient stratification utilizing EHR data.

Research and Application of Big Data Clustering Algorithm based on AI Technology in Cloud Environment

Traditional big data filling algorithms often give inaccurate results due to vulnerabilities to different data types.To solve this problem, this study presents a new big data clustering algorithm powered by AI technology in a cloud environment. The study proposes an advanced Big Data clustering algorithm that leverages AI technology in a cloud environment. It optimizes clustering based on predicted strength using parallel processing. The research focuses on optimizing the clustering algorithm based on the predicted intensity through parallel processing. Experimental results demonstrate that image clustering stability is achieved when the number of clusters exceeds 4, indicating reduced sensitivity to random factors. Although it was not possible to precisely determine the optimal number of clusters, the use of an optimization algorithm showed that at four clusters the prediction intensity reached its peak, ensuring more accurate cluster identification. Through rigorous testing, the optimal number of clusters was determined to be 4. Clustering results show that visitors characterized by certain attributes show higher interest in most columns. This algorithm makes it easier to cluster incomplete large data, improves clustering speed, and improves the accuracy of filling in missing data. Compared to existing methods, this algorithm leverages AI technology in the cloud environment to optimize clustering based on prediction intensity, providing improved accuracy and efficiency during processing in big data management.

Hybrid data clustering algorithm and interactive experience in E-learning electronic course simulation of legal education

With the development of multimedia technology, interactive entertainment experience is an important feature of online teaching mode. Students can engage in interactive learning through various mobile apps and online tools. E-learning provides an innovative way of learning that enables distance learning through electronic course simulation. The objective of this research is to design and develop an improved E-learning course simulation for law teaching by applying hybrid data clustering algorithms and enhancing interactive experiences to improve student engagement and learning outcomes. This article analyzes the simulation of legal education E-learning electronic courses based on mixed data clustering algorithms and interactive experiences. On the basis of the mixed data clustering model of the k-means algorithm, we propose a probability based mixed clustering model that can effectively handle the clustering problem of mixed data. Based on this algorithm, we have established a legal teaching network system platform based on artificial intelligence. The hybrid data clustering algorithm solves the instability problem of the k-means algorithm in the construction of the legal teaching network course platform, providing a reliable solution for the development of artificial intelligence network courses. Based on teaching concepts and combined with various technological characteristics such as network technology and multimedia technology in the information age, users who pass through this teaching system can fully utilize the teaching resources and various functions in the system. The hybrid data clustering algorithm proposed in this article has good clustering performance.

PPHOPCM Privacy-Preserving High-order Possibilistic C-Means Algorithm for Big Data Clustering with Cloud Computing

PPHOPCM Privacy-Preserving High-order Possibilistic C-Means Algorithm for Big Data Clustering with Cloud Computing

Design of health information management model for elderly care using an advanced higher-order hybrid clustering algorithm from the perspective of sports and medicine integration.

In the context of integrating sports and medicine domains, the urgent resolution of elderly health supervision requires effective data clustering algorithms. This paper introduces a novel higher-order hybrid clustering algorithm that combines density values and the particle swarm optimization (PSO) algorithm. Initially, the traditional PSO algorithm is enhanced by integrating the Global Evolution Dynamic Model (GEDM) into the Distribution Estimation Algorithm (EDA), constructing a weighted covariance matrix-based GEDM. This adapted PSO algorithm dynamically selects between the Global Evolution Dynamic Model and the standard PSO algorithm to update population information, significantly enhancing convergence speed while mitigating the risk of local optima entrapment. Subsequently, the higher-order hybrid clustering algorithm is formulated based on the density value and the refined PSO algorithm. The PSO clustering algorithm is adopted in the initial clustering phase, culminating in class clusters after a finite number of iterations. These clusters then undergo the application of the density peak search algorithm to identify candidate centroids. The final centroids are determined through a fusion of the initial class clusters and the identified candidate centroids. Results showcase remarkable improvements: achieving 99.13%, 82.22%, and 99.22% for F-measure, recall, and precision on dataset S1, and 75.22%, 64.0%, and 64.4% on dataset CMC. Notably, the proposed algorithm yields a 75.22%, 64.4%, and 64.6% rate on dataset S, significantly surpassing the comparative schemes' performance. Moreover, employing the text vector representation of the LDA topic vector model underscores the efficacy of the higher-order hybrid clustering algorithm in efficiently clustering text information. This innovative approach facilitates swift and accurate clustering of elderly health data from the perspective of sports and medicine integration. It enables the identification of patterns and regularities within the data, facilitating the formulation of personalized health management strategies and addressing latent health concerns among the elderly population.

Optical Flow-Based Obstacle Detection for Mid-Air Collision Avoidance.

The sky may seem big enough for two flying vehicles to collide, but the facts show that mid-air collisions still occur occasionally and are a significant concern. Pilots learn manual tactics to avoid collisions, such as see-and-avoid, but these rules have limitations. Automated solutions have reduced collisions, but these technologies are not mandatory in all countries or airspaces, and they are expensive. These problems have prompted researchers to continue the search for low-cost solutions. One attractive solution is to use computer vision to detect obstacles in the air due to its reduced cost and weight. A well-trained deep learning solution is appealing because object detection is fast in most cases, but it relies entirely on the training data set. The algorithm chosen for this study is optical flow. The optical flow vectors can help us to separate the motion caused by camera motion from the motion caused by incoming objects without relying on training data. This paper describes the development of an optical flow-based airborne obstacle detection algorithm to avoid mid-air collisions. The approach uses the visual information from a monocular camera and detects the obstacles using morphological filters, optical flow, focus of expansion, and a data clustering algorithm. The proposal was evaluated using realistic vision data obtained with a self-developed simulator. The simulator provides different environments, trajectories, and altitudes of flying objects. The results showed that the optical flow-based algorithm detected all incoming obstacles along their trajectories in the experiments. The results showed an F-score greater than 75% and a good balance between precision and recall.

Categorical Data Clustering: A Bibliometric Analysis and Taxonomy

Numerous real-world applications apply categorical data clustering to find hidden patterns in the data. The K-modes-based algorithm is a popular algorithm for solving common issues in categorical data, from outlier and noise sensitivity to local optima, utilizing metaheuristic methods. Many studies have focused on increasing clustering performance, with new methods now outperforming the traditional K-modes algorithm. It is important to investigate this evolution to help scholars understand how the existing algorithms overcome the common issues of categorical data. Using a research-area-based bibliometric analysis, this study retrieved articles from the Web of Science (WoS) Core Collection published between 2014 and 2023. This study presents a deep analysis of 64 articles to develop a new taxonomy of categorical data clustering algorithms. This study also discusses the potential challenges and opportunities in possible alternative solutions to categorical data clustering.

Analysis of the effectiveness of clustering algorithms for multimodal samples using computer simulation of an educational experiment

Introduction. The article is devoted to the problem of primary data processing of pedagogical experiments having a multimodal character. The purpose of the study is to identify the most effective and universal clustering algorithms for pedagogical experiments. Materials and Methods. The study used the method of modeling a pedagogical experiment. The analysis of 5 clustering algorithms is conducted. The effectiveness of clustering algorithms was evaluated based on the proportion of observations with clustering errors at various tolerance levels and the Jacquard similarity coefficient. Regression analysis was used to assess the influence of modeling parameters of a pedagogical experiment and indicators of descriptive statistics on the effectiveness of clustering algorithms. Results. The assessment of the effectiveness of various data clustering algorithms is provided, as well as a correlation and regression analysis of factors affecting clustering efficiency indicators was carried out. Conclusions. The most effective clustering algorithms for multimodal samples include the K-means algorithm and the agglomerative hierarchical algorithm. The results obtained in this research can be used for statistical analysis of pedagogical, psychological, sociological, biological and medical research data.

Benchmarking Unsupervised Clustering Algorithms for Atomic Force Microscopy Data on Polyhydroxyalkanoate Films.

Surface of polyhydroxyalkanoate (PHA) films of varying monomer compositions are analyzed using atomic force microscopy (AFM) and unsupervised machine learning (ML) algorithms to investigate and classify films based on global attributes such as the scan size, film thickness, and monomer type. The experiment provides benchmarked results for 12 of the most widely used clustering algorithms via a hybrid investigation approach while highlighting the impact of using the Fourier transform (FT) on high-dimensional vectorized data for classification on various pools of data. Our findings indicate that the use of a one-dimensional (1D) FT of vectorized data produces the most accurate outcome. The experiment also provides insights into case-by-case investigations of algorithm performances and the impact of various data pools. Lastly, we show an early version of our tool aimed at investigating surfaces using ML approaches and discuss the results of our current experiment to configure future improvements.

Big data clustering algorithm of power system user load characteristics based on K-means and SOM neural network

Big data clustering algorithm of power system user load characteristics based on K-means and SOM neural network

A novel framework for under-determined blind source separation based on adaptive source counting using mixed linear and circular data clustering algorithm for low latency applications

A novel framework for under-determined blind source separation based on adaptive source counting using mixed linear and circular data clustering algorithm for low latency applications

An Improved Boosting Bald Eagle Search Algorithm with Improved African Vultures Optimization Algorithm for Data Clustering

An Improved Boosting Bald Eagle Search Algorithm with Improved African Vultures Optimization Algorithm for Data Clustering

Federated Fuzzy Clustering for Decentralized Incomplete Longitudinal Behavioral Data.

The use of medical data for machine learning, including unsupervised methods such as clustering, is often restricted by privacy regulations such as the Health Insurance Portability and Accountability Act (HIPAA). Medical data is sensitive and highly regulated and anonymization is often insufficient to protect a patient's identity. Traditional clustering algorithms are also unsuitable for longitudinal behavioral health trials, which often have missing data and observe individual behaviors over varying time periods. In this work, we develop a new decentralized federated multiple imputation-based fuzzy clustering algorithm for complex longitudinal behavioral trial data collected from multisite randomized controlled trials over different time periods. Federated learning (FL) preserves privacy by aggregating model parameters instead of data. Unlike previous FL methods, this proposed algorithm requires only two rounds of communication and handles clients with varying numbers of time points for incomplete longitudinal data. The model is evaluated on both empirical longitudinal dietary health data and simulated clusters with different numbers of clients, effect sizes, correlations, and sample sizes. The proposed algorithm converges rapidly and achieves desirable performance on multiple clustering metrics. This new method allows for targeted treatments for various patient groups while preserving their data privacy and enables the potential for broader applications in the Internet of Medical Things.

REALIZATION OF THE DECISION-MAKING SUPPORT SYSTEM FOR TWITTER USERS’ PUBLICATIONS ANALYSIS

Context. The paper emphasizes the need for a decision-making system that can analyze users’ messages and determine the sentiment to understand how news and events impact people’s emotions. Such a system would employ advanced techniques to analyze users’ messages, delving into the sentiment expressed within the text. The primary goal is to gain insights into how news and various events reverberate through people’s emotions.&#x0D; Objective. The objective is to create a decision-making system that can analyze and determine the sentiment of user messages, understand the emotional response to news and events, and distribute the data into clusters to gain a broader understanding of users’ opinions. This multifaceted objective involves the integration of advanced techniques in natural language processing and machine learning to build a robust decision-making system. The primary goals are sentiment analysis, comprehension of emotional responses to news and events, and data clustering for a holistic view of user opinions.&#x0D; Method. The use of long-short-term memory neural networks for sentiment analysis and the k-means algorithm for data clustering is proposed for processing large volumes of user data. This strategic combination aims to tackle the challenges posed by processing large volumes of user-generated data in a more nuanced and insightful manner.&#x0D; Results. The study and conceptual design of the decision-making system have been completed and the decision-making system was created. The system incorporates sentiment analysis and data clustering to understand users’ opinions and the sentiment value of such opinions dividing them into clusters and visualizing the findings.&#x0D; Conclusions. The conclusion is that the development of a decision-making system capable of analyzing user sentiment and clustering data can provide valuable insights into users’ reactions to news and events in social networks. The proposed use of longshort-term memory neural networks and the k-means algorithm is considered suitable for sentiment analysis and data clustering tasks. The importance of studying existing works and systems to understand available algorithms and their applications is emphasized. The article also describes created and implemented a decision-making system and demonstrated the functionality of the system using a sample dataset.

Big data clustering based on spark chaotic improved particle swarm optimization

In recent years, the surge in continuously accelerating data generation has given rise to the prominence of big data technology. The MapReduce architecture, situated at the core of this technology, provides a robust parallel environment. Spark, a leading framework in the big data landscape, extends the capabilities of the traditional MapReduce model. Coping with big data, especially in the realm of clustering, requires more efficient techniques. Meta-heuristic-based clustering, known for offering global solutions within reasonable time frames, emerges as a promising approach. This paper introduces a parallel-distributed clustering algorithm for big data within the Spark Framework, named Spark, chaotic improved PSO (S-CIPSO). Centered on particle swarm optimization (PSO), the proposed algorithm is enhanced with a chaotic map and an efficient procedure. Test results, conducted on both real and artificial datasets, establish the superior performance and quality of clustering results achieved by the proposed approach. Additionally, the scalability and robustness of S-CIPSO are validated, demonstrating its effectiveness in handling large-scale datasets.