Multiple Clustering Algorithms Research Articles

On the Consistency and Large-Scale Extension of Multiple Kernel Clustering.

Existing multiple kernel clustering (MKC) algorithms have two ubiquitous problems. From the theoretical perspective, most MKC algorithms lack sufficient theoretical analysis, especially the consistency of learned parameters, such as the kernel weights. From the practical perspective, the high complexity makes MKC unable to handle large-scale datasets. This paper tries to address the above two issues. We first make a consistency analysis of an influential MKC method named Simple Multiple Kernel k-Means (SimpleMKKM). Specifically, suppose that ∧γn are the kernel weights learned by SimpleMKKM from the training samples. We also define the expected version of SimpleMKKM and denote its solution as γ*. We establish an upper bound of ||∧γn-γ*||∞ in the order of ~O(1/√n), where n is the sample number. Based on this result, we also derive its excess clustering risk calculated by a standard clustering loss function. For the large-scale extension, we replace the eigen decomposition of SimpleMKKM with singular value decomposition (SVD). Consequently, the complexity can be decreased to O(n) such that SimpleMKKM can be implemented on large-scale datasets. We then deduce several theoretical results to verify the approximation ability of the proposed SVD-based method. The results of comprehensive experiments demonstrate the superiority of the proposed method.

3D indoor area recognition for personnel security using integrated UWB and barometer approach

Real-time location tracking is essential for personal security in industrial scenes, e.g. monitoring worker safety in power substations. Ultra wideband (UWB) technology is suitable for indoor positioning thanks to its high penetration capability, high ranging accuracy and low power consumption. However, UWB based trilateration positioning requires high workload for field deployment of base stations. Indoor complex topology results in multipath and non-line of sight (NLOS) conditions of UWB signals, and degrades the positioning performance in terms of accuracy and reliability. This paper proposes a three-dimensional (3D) area recognition solution by integrating UWB time of flight (TOF) ranging and barometer measurements. The proposed solution utilizes a multi-tier distributed joint probabilistic inference model, which accomplishes the indoor 3D area recognition exploiting multiple clustering and prediction algorithms of machine learning. The field experiments showed that the proposed method can achieve an accuracy of 3D area recognition of more than 99.2%. The proposed method improves the computing efficiency by 93%. The errors of improved differential barometric height estimation method are less than 1 m, which means a success rate of 100% for floor identification, given a floor separation of 3–4 m. The proposed solution is suitable for personnel security applications of industrial scenes, which requires reliable real-time area information rather than just coordinates.

Machine learning identifies clusters of the normal adolescent spine based on sagittal balance.

This study applied a machine learning semi-supervised clustering approach to radiographs of adolescent sagittal spines from a single pediatric institution to identify patterns of sagittal alignment in the normal adolescent spine. We sought to explore the inherent variability found in adolescent sagittal alignment using machine learning to remove bias and determine whether clusters of sagittal alignment exist. Multiple semi-supervised machine learning clustering algorithms were applied to 111 normal adolescent sagittal spines. Sagittal parameters for resultant clusters were determined. Machine learning analysis found that the spines did cluster into distinct groups with an optimal number of clusters ranging from 3 to 5. We performed an analysis on both 3 and 5-cluster groups. The 3-cluster groups analysis found good consistency between methods with 96 of 111, while the analysis of 5-cluster groups found consistency with 105 of 111 spines. When assessing for differences in sagittal parameters between the groups for both analyses, there were differences in T4-12 TK, L1-S1 LL, SS, SVA, PI-LL mismatch, and TPA. However, the only parameter that was statistically different for all groups was SVA. Based on machine learning, the adolescent sagittal spine alignments do cluster into distinct groups. While there were distinguishing features with TK and LL, the most important parameter distinguishing these groups was SVA. Further studies may help to understand these findings in relation to spinal deformities.

Research on the Construction of Portrait of Higher Vocational College Students' Groups Based on Clustering Ensemble

The rapid development of higher vocational education necessitates personalized teaching tailored to individual students. This paper presents an ensemble optimized clustering framework to classify higher vocational college students accurately. By integrating multiple clustering algorithms and employing optimization mechanisms, the framework enhances clustering quality. Experiments on real student data show that this method excels in clustering performance and adapts to various feature data types, yielding results aligned with actual situations. This research provides a data foundation for personalized education in higher vocational colleges, promoting educational equity and cultivating skilled talents.

A Survey of Multiple Kernel Graph Clustering Algorithms

In recent years, algorithms based on multi-kernel learning and graph clustering have garnered significant attention in the field of data mining. This article reviews four advanced multi-kernel graph clustering algorithms: nearest neighbor linear kernel weighted multi-kernel graph clustering, multi-kernel graph clustering based on simultaneous global and local structure preservation, multi-kernel graph clustering based on direct consensus relationship graph learning, and custom non-negative matrix factorization multi-kernel graph tensor clustering. These algorithms have their own characteristics and, by combining multi-kernel learning, graph clustering, and other advanced technologies, they improve the accuracy and efficiency of clustering. However, they also face challenges such as high computational complexity and parameter settings, necessitating further research and optimization. These algorithms have wide application prospects in fields such as image recognition, text classification, and bioinformatics, providing new solutions for handling complex data structures.

Federated Fuzzy Clustering for Decentralized Incomplete Longitudinal Behavioral Data.

The use of medical data for machine learning, including unsupervised methods such as clustering, is often restricted by privacy regulations such as the Health Insurance Portability and Accountability Act (HIPAA). Medical data is sensitive and highly regulated and anonymization is often insufficient to protect a patient's identity. Traditional clustering algorithms are also unsuitable for longitudinal behavioral health trials, which often have missing data and observe individual behaviors over varying time periods. In this work, we develop a new decentralized federated multiple imputation-based fuzzy clustering algorithm for complex longitudinal behavioral trial data collected from multisite randomized controlled trials over different time periods. Federated learning (FL) preserves privacy by aggregating model parameters instead of data. Unlike previous FL methods, this proposed algorithm requires only two rounds of communication and handles clients with varying numbers of time points for incomplete longitudinal data. The model is evaluated on both empirical longitudinal dietary health data and simulated clusters with different numbers of clients, effect sizes, correlations, and sample sizes. The proposed algorithm converges rapidly and achieves desirable performance on multiple clustering metrics. This new method allows for targeted treatments for various patient groups while preserving their data privacy and enables the potential for broader applications in the Internet of Medical Things.

Characterizing urban spatial structure through built form typologies: A new framework using clustering ensembles

Prior research on urban form typologies has largely relied on qualitative classification methods, resulting in subjective and limited analyses. Recently, the emerging data-intensive studies often use a single clustering algorithm and parameter setting, raising concerns about the reliability of the findings. This paper introduces a novel clustering analytical framework for conducting typological studies on urban form that yield stable and reliable results. We employ clustering ensembles, which can combine multiple clustering algorithms to further provide a comprehensible output that facilitates interpretation and knowledge generation. By applying the new framework using 3D building data in Guangzhou, we identify eight typologies of urban built forms and reveal a consistent polycentric pattern across different clustering algorithms and parameter settings. The findings have implications for urban land use planning and regulations by integrating 3D representations of urban form.

Improved Fingerprint-Based Localization Based on Sequential Hybridization of Clustering Algorithms

The localization accuracy of a fingerprint-based localization system is dependent on several factors, one of which is the accuracy and efficiency at which the fingerprint database is clustered. Most highly efficient and accurate clustering algorithms have high time-dependent computational complexity (CC), which tends to limit their practical applicability. A technique that has yet to be explored is the sequential hybridization of multiple low-time CC clustering algorithms to produce a single moderate-time CC clustering algorithm with high localization accuracy. As a result, this paper proposes a clustering algorithm with a moderate time CC that is based on the sequential hybridization of the closest access point (CAP) and improved k-means clustering algorithms. The performance of the proposed sequential hybrid clustering algorithm is determined and compared to the modified affinity propagation clustering (m-APC), fuzzy c-mean (FCM), and 2-CAP algorithms presented in earlier research works using four experimentally generated and publicly available fingerprint databases. The performance metrics considered for the comparisons are the position root mean square error (RMSE) and clustering time based on big O notation. The simulation results show that the proposed sequential hybrid clustering algorithm has improved localization accuracy with position RMSEs of about 54%, 77%, and 52%, respectively, higher than those of the m-APC, FCM, and 2-CAP algorithms. In terms of clustering time, it is 99% and 79% faster than the m-APC and FCM algorithms, respectively, but 90% slower than the 2-CAP algorithm. The results have shown that it is possible to develop a clustering algorithm that has a moderate clustering time with very high localization accuracy through sequential hybridization of multiple clustering algorithms that have a low clustering time with poor localization accuracy. Doi: 10.28991/ESJ-2024-08-02-02 Full Text: PDF

Low-Rank Kernel Tensor Learning for Incomplete Multi-View Clustering

Incomplete Multiple Kernel Clustering algorithms, which aim to learn a common latent representation from pre-constructed incomplete multiple kernels from the original data, followed by k-means for clustering. They have attracted intensive attention due to their high computational efficiency. However, our observation reveals that the imputation of these approaches for each kernel ignores the influence of other incomplete kernels. In light of this, we present a novel method called Low-Rank Kernel Tensor Learning for Incomplete Multiple Views Clustering (LRKT-IMVC) to address the above issue. Specifically, LRKT-IMVC first introduces the concept of kernel tensor to explore the inter-view correlations, and then the low-rank kernel tensor constraint is used to further capture the consistency information to impute missing kernel elements, thereby improving the quality of clustering. Moreover, we carefully design an alternative optimization method with promising convergence to solve the resulting optimization problem. The proposed method is compared with recent advances in experiments with different missing ratios on seven well-known datasets, demonstrating its effectiveness and the advantages of the proposed interpolation method.

A metaheuristic algorithm for model predictive control of the oil-cooled motor in hybrid electric vehicles

The current energy management methods for hybrid vehicles are primarily focused on matching the power flow between the engine and the motor. In order to further reduce overall energy consumption and extend the vehicle's lifespan, this paper utilizes model predictive control (MPC) in the energy management of hybrid vehicles to control the energy consumption and temperature of the oil-cooled motor, with a focus on studying the algorithm for solving the optimization problem in MPC. The main work includes: Establishing a simplified model for the oil-cooled motor and training the working condition prediction model based on the worldwide harmonized light vehicles test cycle (WLTC) conditions utilizing long short-term memory (LSTM) recurrent neural network technology. Furthermore, proposing a novel metaheuristic algorithm based on the temporal nature of the problem, multiple hierarchical clustering algorithm (MHCA). In comparison with typical metaheuristic algorithms, the MPC optimized using MHCA exhibits significantly reduced computation time, with an average time of 0.1967s, indicating that using MHCA can theoretically achieve real-time control and eliminate lag in actual control processes. Furthermore, compared to other optimization algorithms, the MHCA algorithm demonstrates the lowest correlation between computation time and operational conditions, highlighting its high robustness in ensuring the safety of practical control.

Multi-scale feature adaptive fusion model for real-time detection in complex citrus orchard environments

A Multi-scale Feature Adaptive Fusion model (MFAF-YOLO) for real-time detection of citrus harvesting robots in complex field environments is proposed in this study. This proposed model improves detection accuracy while meeting the lightweight requirements of consumer-level cameras. In this study, different citrus fruit varieties, sourced from two distinct devices, are classified into ‘First priority’, ‘Second priority’, and unannotated citrus. This classification strategy guides robots in sequential picking in real-field scenarios, reducing detection redundancy and diminishing the damage rate at the robot end-effector. Additionally, multiple clustering algorithms were employed to adjust the anchor box sizes of the model. The impact of dual and triple detection heads on model accuracy across diverse clustering algorithms was also explored. An innovative multi-scale feature adaptive fusion module was embedded in the model's neck section, aiming to optimize model accuracy and reduce model size. In the dataset processed with multiple augmentation techniques, the novel MFAF-YOLO model achieved the mean Average Precision (mAP) of 90.2 %, reflecting an improvement of 3.8 % compared to the original YOLOv5s model, demonstrating superior generalization capabilities. Compared with seven other mainstream models, including YOLOv4 and MobileNet-YOLOv5s, the Average Precision (AP) value of ‘First priority’ and ‘Second priority’ is 93.2 % and 87.3 %, respectively, achieved the highest AP while maintaining a better detection speed. The model size of MFAF-YOLO is reduced to 10.4 MB, marking a 26.2 % decrease relative to the lightweight YOLOv5s model. Experimental results highlight the model's strong robustness and successful attainment of an effective balance between model accuracy and lightweight. This proposed model provides theoretical support for real-time picking and harvesting decision-making of citrus.

Comparative analysis of clustering algorithm and improved algorithm application in shipping congestion

In recent years, due to the lack of reasonable planning of ship operation routes, shipping congestion accidents have been increasing, seriously restricting port development and channel operation, posing a great threat to ship navigation safety and greatly restricting the development prospects of the shipping industry. This paper aims to compare and analyze the application of the clustering algorithm and its improved algorithm in solving port and waterway congestion problems. By analyzing the literature on the use of clustering algorithms to solve shipping congestion problems, and comparing the advantages and disadvantages of multiple clustering algorithms in solving problems, this paper concludes that Partition-based Methods are more suitable for identifying port congestion and the improved Fuzzy DBSCAN algorithm for identifying channel congestion. The research in this paper will help to select clustering algorithms when solving shipping congestion problems in the future.

A novel member enhancement‐based clustering ensemble algorithm

SummaryClustering ensemble is a popular approach for identifying data clusters that combines the clustering results from multiple base clustering algorithms to produce more accurate and robust data clusters. However, the performance of clustering ensemble algorithms is highly dependent on the quality of clustering members. To address this problem, this paper proposes a member enhancement‐based clustering ensemble (MECE) algorithm that selects the ensemble members by considering their distribution consistency. MECE has two main components, called heterocluster splitting and homocluster merging. The first component estimates two probability density functions (p.d.f.s) estimated on the sample points of an heterocluster and represents them using a Gaussian distribution and a Gaussian mixture model. If the random numbers generated by these two p.d.f.s have different probability distributions, the heterocluster is then split into smaller clusters. The second component merges the clusters that have high neighborhood densities into a homocluster, where the neighborhood density is measured using a novel evaluation criterion. In addition, a co‐association matrix is presented, which serves as a summary for the ensemble of diverse clusters. A series of experiments were conducted to evaluate the feasibility and effectiveness of the proposed ensemble member generation algorithm. Results show that the proposed MECE algorithm can select high quality ensemble members and as a result yield the better clusterings than six state‐of‐the‐art ensemble clustering algorithms, that is, cluster‐based similarity partitioning algorithm (CSPA), meta‐clustering algorithm (MCLA), hybrid bipartite graph formulation (HBGF), evidence accumulation clustering (EAC), locally weighted evidence accumulation (LWEA), and locally weighted graph partition (LWGP). Specifically, MECE algorithm has the nearly 23% higher average NMI, 27% higher average ARI, 15% higher average FMI, and 10% higher average purity than CSPA, MCLA, HBGF, EAC, LWEA, and LWGA algorithms. The experimental results demonstrate that MECE algorithm is a valid approach to deal with the clustering ensemble problems.

An interpretable clustering approach to safety climate analysis: Examining driver group distinctions

The transportation industry, particularly the trucking sector, is prone to workplace accidents and fatalities. Accidents involving large trucks accounted for a considerable percentage of overall traffic fatalities. Recognizing the crucial role of safety climate in accident prevention, researchers have sought to understand its factors and measure its impact within organizations. While existing data-driven safety climate studies have made remarkable progress, clustering employees based on their safety climate perception is innovative and has not been extensively utilized in research. Identifying clusters of drivers based on their safety climate perception allows the organization to profile its workforce and devise more impactful interventions. The lack of utilizing the clustering approach could be due to difficulties interpreting or explaining the factors influencing employees’ cluster membership. Moreover, existing safety-related studies did not compare multiple clustering algorithms, resulting in potential bias. To address these problems, this study introduces an interpretable clustering approach for safety climate analysis. This study compares five algorithms for clustering truck drivers based on their safety climate perceptions. It also proposes a novel method for quantitatively evaluating partial dependence plots (QPDP). Then, to better interpret the clustering results, this study introduces different interpretable machine learning measures (Shapley additive explanations, permutation feature importance, and QPDP). The Python code used in this study is available at https://github.com/NUS-DBE/truck-driver-safety-climate. This study explains the clusters based on the importance of different safety climate factors. Drawing on data collected from more than 7,000 American truck drivers, this study significantly contributes to the scientific literature. It highlights the critical role of supervisory care promotion in distinguishing various driver groups. Moreover, it showcases the advantages of employing machine learning techniques, such as cluster analysis, to enrich the scientific knowledge in this field. Future studies could involve experimental methods to assess strategies for enhancing supervisory care promotion, as well as integrating deep learning clustering techniques with safety climate evaluation.

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Target maneuver recognition is a prerequisite for air combat situation awareness, trajectory prediction, threat assessment and maneuver decision. To get rid of the dependence of the current target maneuver recognition method on empirical criteria and sample data, and automatically and adaptively complete the task of extracting the target maneuver pattern, in this paper, an air combat maneuver pattern extraction based on time series segmentation and clustering analysis is proposed by combining autoencoder, G-G clustering algorithm and the selective ensemble clustering analysis algorithm. Firstly, the autoencoder is used to extract key features of maneuvering trajectory to remove the impacts of redundant variables and reduce the data dimension; Then, taking the time information into account, the segmentation of Maneuver characteristic time series is realized with the improved FSTS-AEGG algorithm, and a large number of maneuver primitives are extracted; Finally, the maneuver primitives are grouped into some categories by using the selective ensemble multiple time series clustering algorithm, which can prove that each class represents a maneuver action. The maneuver pattern extraction method is applied to small scale air combat trajectory and can recognize and correctly partition at least 71.3% of maneuver actions，indicating that the method is effective and satisfies the requirements for engineering accuracy. In addition, this method can provide data support for various target maneuvering recognition methods proposed in the literature, greatly reduce the workload and improve the recognition accuracy.

Road Intersection Extraction Based on Low-Frequency Vehicle Trajectory Data

Global navigation satellite system (GNSS) vehicle trajectory data play an important role in obtaining timely urban road information. However, most models cannot effectively extract road information from low-frequency trajectory data. In this study, we aimed to accurately extract urban road network intersections and central locations from low-frequency GNSS trajectory data, and we developed a method for accurate road intersection identification based on filtered trajectory sequences and multiple clustering algorithms. Our approach was founded on the following principles. (1) We put in place a rigorous filtering rule to account for the offset characteristics of low-frequency trajectory data. (2) To overcome the low density and weak connection features of vehicle turning points, we adopted the CDC clustering algorithm. (3) By combining the projection features of orientation values in 2D coordinates, a mean solving method based on the DBSCAN algorithm was devised to obtain intersection center coordinates with greater accuracy. Our method could effectively identify urban road intersections and determine the center position and more effectively apply low-frequency trajectory data. Compared with remote sensing images, the intersection identification accuracy was 96.4%, the recall rate was 89.6%, and the F-value was 92.88% for our method; the intersection center position’s root mean square error (RMSE) was 10.39 m, which was 14.9% higher than that of the mean value method.

ClusterBMA: Bayesian model averaging for clustering.

Various methods have been developed to combine inference across multiple sets of results for unsupervised clustering, within the ensemble clustering literature. The approach of reporting results from one 'best' model out of several candidate clustering models generally ignores the uncertainty that arises from model selection, and results in inferences that are sensitive to the particular model and parameters chosen. Bayesian model averaging (BMA) is a popular approach for combining results across multiple models that offers some attractive benefits in this setting, including probabilistic interpretation of the combined cluster structure and quantification of model-based uncertainty. In this work we introduce clusterBMA, a method that enables weighted model averaging across results from multiple unsupervised clustering algorithms. We use clustering internal validation criteria to develop an approximation of the posterior model probability, used for weighting the results from each model. From a combined posterior similarity matrix representing a weighted average of the clustering solutions across models, we apply symmetric simplex matrix factorisation to calculate final probabilistic cluster allocations. In addition to outperforming other ensemble clustering methods on simulated data, clusterBMA offers unique features including probabilistic allocation to averaged clusters, combining allocation probabilities from 'hard' and 'soft' clustering algorithms, and measuring model-based uncertainty in averaged cluster allocation. This method is implemented in an accompanying R package of the same name. We use simulated datasets to explore the ability of the proposed technique to identify robust integrated clusters with varying levels of separation between subgroups, and with varying numbers of clusters between models. Benchmarking accuracy against four other ensemble methods previously demonstrated to be highly effective in the literature, clusterBMA matches or exceeds the performance of competing approaches under various conditions of dimensionality and cluster separation. clusterBMA substantially outperformed other ensemble methods for high dimensional simulated data with low cluster separation, with 1.16 to 7.12 times better performance as measured by the Adjusted Rand Index. We also explore the performance of this approach through a case study that aims to identify probabilistic clusters of individuals based on electroencephalography (EEG) data. In applied settings for clustering individuals based on health data, the features of probabilistic allocation and measurement of model-based uncertainty in averaged clusters are useful for clinical relevance and statistical communication.

Development and application of comprehensive evaluation model based on correlation analysis algorithm, multiple logistic regression and clustering algorithm - Taking glass composition analysis and identification as an example

Take glass as an example. In ancient times, when firing glass, in addition to the raw material quartz sand, chemical components such as stabilizer and flux need to be added. During the storage process of glass, the internal elements will exchange with the environmental elements in a large amount, resulting in the weathering of the glass surface. In this paper, the composition of ancient glass is analyzed and identified by establishing correlation analysis model, multiple logistic regression and cluster model. The results show that: (1) There is a high correlation between glass type and surface weathering. Lead barium glass is easy to be weathered, and high potassium glass is not easy to be weathered. (2) High-potassium glass and lead-barium glass have the highest classification accuracy when they are divided into two sub-categories. Therefore, glass can be divided into four categories: high-potassium I, high-potassium II, lead-barium I, and lead-barium II.

Fuzzy-Rough induced spectral ensemble clustering

Ensemble clustering helps achieve fast clustering under abundant computing resources by constructing multiple base clusterings. Compared with the standard single clustering algorithm, ensemble clustering integrates the advantages of multiple clustering algorithms and has stronger robustness and applicability. Nevertheless, most ensemble clustering algorithms treat each base clustering result equally and ignore the difference of clusters. If a cluster in a base clustering is reliable/unreliable, it should play a critical/uncritical role in the ensemble process. Fuzzy-rough sets offer a high degree of flexibility in enabling the vagueness and imprecision present in real-valued data. In this paper, a novel fuzzy-rough induced spectral ensemble approach is proposed to improve the performance of clustering. Specifically, the significance of clusters is differentiated, and the unacceptable degree and reliability of clusters formed in base clustering are induced based on fuzzy-rough lower approximation. Based on defined cluster reliability, a new co-association matrix is generated to enhance the effect of diverse base clusterings. Finally, a novel consensus spectral function is defined by the constructed adjacency matrix, which can lead to significantly better results. Experimental results confirm that the proposed approach works effectively and outperforms many state-of-the-art ensemble clustering algorithms and base clustering, which illustrates the superiority of the novel algorithm.

VSFCM: A Novel Viewpoint-Driven Subspace Fuzzy C-Means Algorithm

Nowadays, most fuzzy clustering algorithms are sensitive to the initialization results of clustering algorithms and have a weak ability to handle high-dimensional data. To solve these problems, we developed the viewpoint-driven subspace fuzzy c-means (VSFCM) algorithm. Firstly, we propose a new cut-off distance. Based on this, we establish the cut-off distance-induced clustering initialization (CDCI) method and use it as a new strategy for cluster center initialization and viewpoint selection. Secondly, by taking the viewpoint obtained by CDCI as the entry point of knowledge, a new fuzzy clustering strategy driven by knowledge and data is formed. Based upon these, we put forward the VSFCM algorithm combined with viewpoints, separation terms, and subspace fuzzy feature weights. Moreover, compared with the symmetric weights obtained by other subspace clustering algorithms, the weights of the VSFCM algorithm exhibit significant asymmetry. That is, they assign greater weights to features that contribute more, which is validated on the artificial dataset DATA2 in the experimental section. The experimental results compared with multiple advanced clustering algorithms on the three types of datasets validate that the proposed VSFCM algorithm has the best performance in five indicators. It is demonstrated that the initialization method CDCI is more effective, the feature weight allocation of VSFCM is more consistent with the asymmetry of experimental data, and it can achieve better convergence speed while displaying better clustering efficiency.