Clustering Algorithm Research Articles

Enhancing operational research in mechatronic systems via modularization: comparative analysis of four clustering algorithms using validation indices

Enhancing operational research in mechatronic systems via modularization: comparative analysis of four clustering algorithms using validation indices

DGSIST: Clustering spatial transcriptome data based on deep graph structure Infomax

Although spatial transcriptomics data provide valuable insights into gene expression profiles and the spatial structure of tissues, most studies rely solely on gene expression information, underutilizing the spatial data. To fully leverage the potential of spatial transcriptomics and graph neural networks, the DGSI (Deep Graph Structure Infomax) model is proposed. This innovative graph data processing model uses graph convolutional neural networks and employs an unsupervised learning approach. It maximizes the mutual information between graph-level and node-level representations, emphasizing flexible sampling and aggregation of nodes and their neighbors. This effectively captures and incorporates local information from nodes into the overall graph structure. Additionally, this paper developed the DGSIST framework, an unsupervised cell clustering method that integrates the DGSI model, SVD dimensionality reduction algorithm, and k-means++ clustering algorithm. This aims to identify cell types accurately. DGSIST fully uses spatial transcriptomics data and outperforms existing methods in accuracy. Demonstrations of DGSIST’s capability across various tissue types and technological platforms have shown its effectiveness in accurately identifying spatial domains in multiple tissue sections. Compared to other spatial clustering methods, DGSIST excels in cell clustering and effectively eliminates batch effects without needing batch correction. DGSIST excels in spatial clustering analysis, spatial variation identification, and differential gene expression detection and directly applies to graph analysis tasks, such as node classification, link prediction, or graph clustering. Anticipation lies in the contribution of the DGSIST framework to a deeper understanding of the spatial organizational structures of diseases such as cancer.

Clustering Functional Data With Measurement Errors: A Simulation-Based Approach.

Clustering analysis of functional data, which comprises observations that evolve continuously over time or space, has gained increasing attention across various scientific disciplines. Practical applications often involve functional data that are contaminated with measurement errors arising from imprecise instruments, sampling errors, or other sources. These errors can significantly distort the inherent data structure, resulting in erroneous clustering outcomes. In this article, we propose a simulation-based approach designed to mitigate the impact of measurement errors. Our proposed method estimates the distribution of functional measurement errors through repeated measurements. Subsequently, the clustering algorithm is applied to simulated data generated from the conditional distribution of the unobserved true functional data given the observed contaminated functional data, accounting for the adjustments made to rectify measurement errors. We illustrate through simulations show that the proposed method has improved numerical performance than the naive methods that neglect such errors. Our proposed method was applied to a childhood obesity study, giving more reliable clustering results.

Improved identification of network anomalies through optimal CURE clustering

Abstract In this paper, we propose an advanced network anomaly behavior identification framework to overcome the constraints inherent in conventional rule- or signature-based approaches, which often struggle with emerging and previously unknown threats. Central to our framework is an Enhanced CURE (Cluster Updating and REfining) clustering algorithm, meticulously tailored and refined to incorporate a density-based methodology. This enhancement enables the algorithm to discern subtle shifts in network anomaly patterns with heightened precision. The implementation workflow commences with the application of the optimized CURE algorithm to analyze network data, followed by the deployment of a sophisticated anomaly degree ranking mechanism. This mechanism, through meticulous calculation of individual data points' anomaly degrees and subsequent ranking, effectively isolates those deviating significantly from standard behavioral norms, incorporating a strategic threshold to filter out false positives. To validate our methodology's efficacy and its superiority over existing techniques, experiments were conducted utilizing a substantial real-world network dataset. These tests affirm not only a marked increase in the accuracy of abnormal behavior recognition and a reduction in computational intricacy but also demonstrate the adaptability across diverse network ecosystems. Our approach has proven successful in pinpointing a wide array of network anomalies, encompassing malicious cyberattacks, fraudulent activities, unauthorized intrusions, and breaches of security protocols, thereby highlighting its comprehensive capability in bolstering network defense strategies. Despite the notable advancements and successful identification of various network anomalies, our framework currently lacks integration with real-time learning capabilities, limiting its immediate responsiveness to rapidly evolving attack patterns and necessitating ongoing research for dynamic updates and adaptive learning mechanisms.

Analysis and Comparison of the Performance of K-Means Algorithm and X-Means Algorithm in Disease Type Clustering in Mitra Medika Hospital

Analysis and Comparison of the Performance of K-Means Algorithm and X-Means Algorithm in Disease Type Clustering in Mitra Medika Hospital

Joint Beamforming Design and User Clustering Algorithm in NOMA-Assisted ISAC Systems.

To enhance the performance of non-orthogonal multiple access (NOMA)-assisted integrated sensing and communication (ISAC) systems in multi-user distributed scenarios, an improved Gaussian Mixture Model (GMM)-based user clustering algorithm is proposed. This algorithm is tailored for ISAC systems, significantly improving bandwidth reuse gains and reducing serial interference. First, using the Sum of Squared Errors (SSE), the algorithm reduces sensitivity to the initial cluster center locations, improving clustering accuracy. Then, direction weight factors are introduced based on the base station position and a penalty function involving users' Euclidean distances and sensing power. Modifications to the EM algorithm in calculating posterior probabilities and updating the covariance matrix help align user clusters with the characteristics of NOMAISAC systems. This improves users' interference resistance, lowers decoding difficulty, and optimizes the system's sensing capabilities. Finally, a fractional programming (FP) approach addresses the non-convex joint beamforming design problem, enhancing power and channel gains and achieving co-optimizing sensing and communication signals. The simulation results show that, under the improved GMM user clustering algorithm and FP optimization, the NOMA-ISAC system improves user spectral efficiency by 4.3% and base station beam intensity by 5.4% compared to traditional ISAC systems.

DPC clustering algorithm based on K-nearest neighbors and kernel density estimation

Clustering by fast search and find of density peaks (DPC) is a density-based clustering algorithm. This algorithm does not require iteration and too many parameter settings, but it cannot identify cluster centers with small cluster density because the local structure of the data is not considered when calculating the local density. To address the above problems, a DPC clustering algorithm (KKDPC) based on K-reciprocal Neighbors (KN) and Kernel Density Estimation (KDE) is proposed. First, the number of reciprocal neighbors and local kernel density of data points are obtained by nearest neighbor and kernel density estimation methods ; second, the number of K reciprocal neighbors is added to the local kernel density to obtain a new local density; finally, the relative distance is obtained according to the local density of the data points, and the cluster center is selected and the non-center point is allocated by constructing a decision graph. Experiments are conducted using artificial and real data sets, and compared with seven clustering algorithms: DPC, DBSCAN, K-means, FCM, DPC-KNN, DPC-NN, and DPC-FWSN. The performance of the KKDPC clustering algorithm is verified by calculating the adjusted mutual information (AMI), adjusted Rand index (ARI), and normalized mutual information (NMI). The experimental results show that this method can more accurately identify the cluster center and effectively improve the clustering accuracy.

An Algorithm for Clustering with Confidence-Based Must-Link and Cannot-Link Constraints

We study here the semisupervised k-clustering problem where information is available on whether pairs of objects are in the same or different clusters. This information is available either with certainty or with a limited level of confidence. We introduce the pair-wise confidence constraints clustering (PCCC) algorithm, which iteratively assigns objects to clusters while accounting for the information provided on the pairs of objects. Our algorithm uses integer programming for the assignment of objects, which allows us to include relationships as hard constraints that are guaranteed to be satisfied or as soft constraints that can be violated subject to a penalty. This flexibility distinguishes our algorithm from the state of the art, in which all pair-wise constraints are considered hard or all are considered soft. We developed an enhanced multistart approach and a model-size reduction technique for the integer program that contribute to the effectiveness and efficiency of the algorithm. Unlike existing algorithms, our algorithm scales to large-scale instances with up to 60,000 objects, 100 clusters, and millions of cannot-link constraints (which are the most challenging constraints to incorporate). We compare the PCCC algorithm with state-of-the-art approaches in an extensive computational study. Even though the PCCC algorithm is more general than the state-of-the-art approaches in its applicability, it outperforms the state-of-the-art approaches on instances with all hard or all soft constraints in terms of both run time and various metrics of solution quality. The code of the PCCC algorithm is publicly available on GitHub. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: The research of D. S. Hochbaum was supported by the AI Institute NSF Award [Grant 2112533]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0419 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0419 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Identifying the Vertical Stratification of Sediment Samples by Visible and Near-Infrared Spectroscopy

Vertical stratification in marine sediment profiles indicates physical and chemical sedimentary processes and, thus, is the first step in sedimentary research and in studying their relationship with global climate change. Traditional technologies for studying vertical stratification have low efficiency; thus, new technologies are highly needed. Recently, visible and near-infrared spectroscopy (VNIR) has been explored to rapidly determine sediment parameters, such as clay content, particle size, total carbon (TC), total nitrogen (TN), and so on. Here, we explored vertical stratification in a sediment column in the South China Sea using VNIR. The sediment column was 160 cm and divided into 160 samples by 1 cm intervals. All samples were classified into three layers by depth, that is, 0–50 cm (the upper layer), 50–100 cm (the middle layer), and 100–160 cm (the bottom layer). Concentrations of TC and TN in each sample were measured by Elementa Vario EL III. Visible and near-infrared reflectance spectra of each sample were collected by Agilent Cary 5000. A global model and several classification models for vertical stratification in sediments were established by a Support Vector Machine (SVM) after the characteristic spectra were identified using Competitive Adaptive Reweighted Sampling. In the classification models, K-means clustering and Density Peak Clustering (DPC) were employed as the unsupervised clustering algorithms. The results showed that the stratification was successful by VNIR, especially when using the combination of unsupervised clustering and machine learning algorithms. The correct classification rate (CCR) was much higher in the classification models than in the global model. And the classification models had a higher CCR using K-means combined with SVM (94.8%) and using DPC combined with SVM (96.0%). The higher CCR might be derived from the chemical classification. Indeed, similar results were also found in the chemical stratification. This study provided a theoretical basis for the rapid and synchronous measurement of chemical and physical parameters in sediment profiles by VNIR.

Uncertain data density peak clustering algorithm based on JS divergence

Aiming at the defects of traditional density-based uncertainty clustering algorithms, such as parameter sensitivity and poor clustering results for complex manifold uncertain data sets, a new uncertainty data density peak clustering algorithm based on JS divergence (UDPC-JS) is proposed. The algorithm first removes noise points by using the uncertain natural neighbor density factor defined by the uncertain natural neighbor. Secondly, the local density of uncertain data objects is calculated by combining the uncertain natural neighbor and JS divergence. The initial clustering center of the uncertain data set is found by combining the idea of representative points, and the distance between the initial clustering centers is defined based on JS divergence and graph. Then, the decision graph is constructed on the initial clustering center using the local density calculated based on the uncertain natural neighbor and JS divergence and the newly defined distance between the initial clustering centers based on JS divergence and graph, and the final clustering center is selected according to the decision graph. Finally, the unassigned uncertain data objects are assigned to the cluster where their initial clustering center is located. Experimental results show that the algorithm has better clustering effect and accuracy than the comparison algorithm, and has a greater advantage in processing complex manifold uncertain data sets.

Code and Data Repository for An Algorithm for Clustering with Confidence-Based Must-Link and Cannot-Link Constraints

The PCCC algorithm is a clustering method that incorporates both hard and soft must-link and cannot-link constraints.

K-means-ten clustering algorithm based on min-max criterion and region division

Aiming at the problem of unstable clustering results and low solution accuracy of -means- + algorithm, an I--means-+ clustering algorithm based on min-max criterion and region division is proposed. Firstly, the min-max criterion is proposed to calculate the distance from each data point to the nearest center, and the data point with the largest distance is preferentially selected as the new cluster center to avoid the situation where multiple initial centers are clustered in the same cluster; secondly, the data points in the split cluster are divided into different regions, and a data point is selected in each region as the candidate center to increase the diversity of the candidate centers; finally, for the clusters that fail to pair, the new split cluster is reselected by gain to pair with the original deleted cluster again to improve the pairing success rate and further reduce the objective function value. Experimental results show that compared with the I means-+ algorithm, the proposed algorithm has an average improvement in solution accuracy −k −and a more stable clustering result kunder the premise of basically equivalent operation efficiency compared with 6.47%-means and kmeans+ algorithms, the proposed algorithm has higher solution accuracy.

A Novel Crowding Clustering Algorithm for Unsupervised and Supervised Filter Feature Selection Problem

A Novel Crowding Clustering Algorithm for Unsupervised and Supervised Filter Feature Selection Problem

Scalable Clustering: Large Scale Unsupervised Learning of Gaussian Mixture Models with Outliers

Clustering is a widely used technique with a long and rich history in a variety of areas. However, most existing algorithms do not scale well to large datasets, or are missing theoretical guarantees of convergence. This paper introduces a provably robust clustering algorithm based on loss minimization that performs well on Gaussian mixture models with outliers. It provides theoretical guarantees that the algorithm obtains high accuracy with high probability under certain assumptions. Moreover, it can also be used as an initialization strategy for k-means clustering. Experiments on real-world large-scale datasets demonstrate the effectiveness of the algorithm when clustering a large number of clusters, and a k-means algorithm initialized by the algorithm outperforms many of the classic clustering methods in both speed and accuracy, while scaling well to large datasets such as ImageNet.

Travel pattern recognition of urban rail passengers based on spatiotemporal sequence similarity

Understanding passenger travel patterns is helpful for the allocation of passenger resources in urban rail transit. Based on rail transit smart card data, this paper proposes a method to identify travel patterns by modeling individual spatiotemporal sequences. First, all the stations visited by individual passengers are extracted, and the similarity of the stations is calculated by the frequency of inter-station travel, the distance between stations and the activity duration of the stations. The main spatial activity area of the individual is divided using a hierarchical clustering algorithm. Then, the spatiotemporal sequence is inferred based on the individual's travel order. The sequence is a set of discrete values that characterize the spatiotemporal state. PCA-KL and K-Means++ are used to extract the similarity sequence structure to identify the passenger travel pattern. Finally, the rail transit smart card data of Xi'an in a certain month is taken as an example to identify its passenger travel pattern. The results show that complex passenger flow has 5 travel modes, of which 3 typical modes are commuting travel in a macro sense, accounting for 79% of the passenger flow. It can be seen that the pattern recognition based on the similarity of individual spatiotemporal sequences in this paper fully reflects the particularity and versatility of the research method, and is highly operational for different cities.

Identification of PANoptosis-related genes for idiopathic pulmonary fibrosis by machine learning and molecular subtype analysis

Idiopathic pulmonary fibrosis (IPF) is a severe interstitial lung disease characterized by a grim prognosis, in which various forms of cell death are significant contributors to its development. The objective of this study is to explore diagnostic biomarkers and molecular subtypes associated with PANoptosis in IPF, and to develop reliable diagnostic models based on PANoptosis-related mechanisms. The peripheral blood transcriptomic data of IPF from the Gene Expression Omnibus (GEO) database and PANoptosis-related genes from the GeneCards database were utilized to conduct differential gene expression analysis and weighted gene co-expression network analysis (WGCNA), identifying PANoptosis-related differentially expressed genes (PDEGs). We yielded 9 PDEGs and employed machine learning algorithms to identify 3 key diagnostic biomarkers for PANoptosis in IPF: MMP9, FCMR, NIBAN3. Consensus clustering algorithm was applied to recognize two PANoptosis-related subtypes. Cluster 1 exhibited higher abundance of adaptive immune response cells and significant enrichment in DNA damage and repair-related pathways. Cluster 2 exhibited greater prevalence of innate immune response cells and predominant enhancement in pathways related to lipid cholesterol metabolism and vascular remodeling. Diagnostic models were developed with the aid of clinical decision-making and a novel approach to the diagnosis and treatment for IPF.

A hybrid prediction model of blast furnace permeability index combining least square support vector machine and artificial neural network

Blast furnace (BF) permeability index is a crucial parameter that can quickly, intuitively, and comprehensively reflect the furnace condition. Accurate prediction of this index is crucial for optimising production efficiency and ensuring the stable operation of the BF. In this study, a hybrid permeability index prediction model is constructed by combining a least squares support vector machine and an artificial neural network, using the mean shift clustering algorithm (MSCA) to classify the BF conditions is applied. The results show that the MSCA algorithm shows remarkable precision in classifying the stable and unstable operating states of BF, achieving an impressive accuracy rate of 93.98%. The hybrid prediction model could accurately predict the permeability index and has a mean absolute error of 0.6877, a mean square error of 0.4721 and an R2 of 0.9215, highlighting its robust predictive performance. These findings underscore the practical significance of our model in enhancing BF operational efficiency and reliability.

Theia 456: Tidally Shredding an Open Cluster

The application of clustering algorithms to the Gaia astrometric catalog has revolutionized our census of stellar populations in the milky Way, including the discovery of many new dispersed structures. We focus on one such structure, Theia 456 (COIN-Gaia-13), a loosely bound collection of ∼320 stars spanning ∼120 pc that has previously been shown to exhibit kinematic, chemical, and gyrochronal coherency, indicating a common origin. We obtain follow-up radial velocities and supplement these with Gaia astrometry to perform an in-depth dynamical analysis of Theia 456. By integrating stellar orbits through a Milky Way potential, we find the currently dispersed structure coalesced into a small cluster in the past. Via Bayesian modeling, we derive a kinematic age of 245 ± 3 Myr (statistical), a half-mass–radius of 9 ± 2 pc, and an initial one-dimensional velocity dispersion of 0.14 ± 0.02 km s−1. Our results are entirely independent of model isochrones, details of stellar evolution, and internal cluster dynamics, and the statistical precision in our age derivation rivals that of the most precise age-dating techniques known today, though our imperfect knowledge of the Milky Way potential and simple spherical model for Theia 456 at birth add additional uncertainties. Using posterior predictive checking, we confirm these results are robust under reasonable variations to the Milky Way potential. Such low-density structures that are disrupted by the Galactic tides before virializing may be ubiquitous, signifying that Theia 456 is a valuable benchmark for studying the dynamical history of stellar populations in the Milky Way.

A clustering ensemble algorithm for handling deep embeddings using cluster confidence

Abstract Clustering ensemble, which aims to learn a robust consensus clustering from multiple weak base clusterings, has achieved promising performance on various applications. With the development of big data, the scale and complexity of data is constantly increasing. However, most existing clustering ensemble methods typically employ shallow clustering algorithms to generate base clusterings. When confronted with high-dimensional complex data, these shallow algorithms fail to fully utilize the intricate features present in the latent data space. As a result, the quality and diversity of the generated base clusterings are insufficient, thus affecting the subsequent ensemble performance. To address this issue, we propose a novel clustering ensemble algorithm for handling deep embeddings using cluster confidence (CEDECC) to improve the robustness and performance. Instead of simply combining deep clustering with clustering ensembles, we take into consideration that the performance of existing deep clustering methods heavily relies on the quality of low-dimensional embeddings generated during the pre-training stage. The quality of embeddings is unstable due to the influence of different initialization parameters. In CEDECC, specifically, we first construct a cluster confidence measure to evaluate the quality of low-dimensional embeddings. Typically, high-quality low-dimensional embeddings yield accurate clustering results with the same model parameters. Then, we utilize multiple high-quality embeddings to generate the base partitions. In the ensemble strategy phase, we consider the cluster-wise diversity and propose a novel ensemble cluster estimation to improve the overall consensus performance of the model. Extensive experiments on three benchmark datasets and four real-world biological datasets have demonstrated that the proposed CEDECC consistently outperforms the state-of-the-art clustering ensemble methods.

Language Preservation through Popularization of Regional Indian Music: A Data-Driven Approach

The increased global connectivity due to internet penetration has resulted in the dominance of international languages and the decline of regional languages. To help preserve these languages, this study identifies the non-native regions in India that offer the most promising opportunities for expanding the popularity of music for each of the 14 regional languages. It analyses the audio features of 41,000 songs from 14 Indian languages on Spotify, employing two approaches – clustering algorithms and Random Forest Regressors. An overlap between the regions identified by both methods for any given language is considered a robust result, providing greater confidence in the results. The results show that while geographical proximity is a significant factor in determining non-native market fits for the music of a given language, alignment in preferences of audio features driven by cultural similarities plays an integral role as well. The findings have important implications for music industry stakeholders, including artists, management, digital platforms, and governments, who can leverage this data to devise strategies to expand the reach of regional music. This includes designing targeted marketing strategies, fostering cross-regional artist collaborations, and optimizing content recommendation algorithms, among others.