Partitional Clustering Research Articles

SimpleMKKM: Simple Multiple Kernel K-Means.

We propose a simple yet effective multiple kernel clustering algorithm, termed simple multiple kernel k-means (SimpleMKKM). It extends the widely used supervised kernel alignment criterion to multi-kernel clustering. Our criterion is given by an intractable minimization-maximization problem in the kernel coefficient and clustering partition matrix. To optimize it, we equivalently rewrite the minimization-maximization formulation as a minimization of an optimal value function, prove its differenentiablity, and design a reduced gradient descent algorithm to decrease it. Furthermore, we prove that the resultant solution of SimpleMKKM is the global optimum. We theoretically analyze the performance of SimpleMKKM in terms of its clustering generalization error. After that, we develop extensive experiments to investigate the proposed SimpleMKKM from the perspectives of clustering accuracy, advantage on the formulation and optimization, variation of the learned consensus clustering matrix with iterations, clustering performance with varied number of samples and base kernels, analysis of the learned kernel weight, the running time and the global convergence. The experimental study demonstrates the effectiveness of the proposed SimpleMKKM by considerably and consistently outperforming state of the art multiple kernel clustering alternatives. In addition, the ablation study shows that the improved clustering performance is contributed by both the novel formulation and new optimization. Our work provides a more effective approach to integrate multi-view data for clustering, and this could trigger novel research on multiple kernel clustering. The source code and data for SimpleMKKM are available at https://github.com/xinwangliu/SimpleMKKMcodes/.

Cluster Partitions Optimization Using the Latent Class Analysis Technique

The paper explores the formation of a cluster partition using a system of bundles of vectors. Each object is represented by a vector with coordinates corresponding to its characteristic factors. The study focuses on the principles of changes in the total density of these bundles when objects move from one cluster to another. Based on the findings, an algorithm is proposed to reduce the total intracluster variance of the initial cluster partition.
 The proposed algorithm is based on a modern technique for analyzing latent classes. This technique emphasizes that the forming factors of objects within each cluster should be highly correlated. Since the number of forming factors is arbitrary, the algorithm replaces this requirement with the maximum possible proximity of object factor vectors to the average vector within each cluster. The degree of this closeness is referred to as the tightness of the cluster vector bundle.
 The paper also introduces a new method for quantifying clusters in the improvable clustering constructed using the proposed algorithm. A practical example of the algorithm's application to process medical data is presented. The study discusses the reasons for the dependence of the algorithm's outcome on the choice of initial cluster partition.

An Atmospheric Phase Correction Method Based on Normal Vector Clustering Partition in Complicated Conditions for GB-SAR

Atmospheric phase is the main factor affecting the accuracy of ground-based synthetic aperture radar. The atmospheric phase screen (APS) may be very complicated, due to the drastic changes in atmospheric conditions, and the conventional correction methods based on regression models cannot fit and correct it effectively. Partition correction is a feasible path to improve atmospheric phase correction (APC) accuracy for complicated APS, but the overfitting problem cannot be ignored. In this article, we propose a clustering partition method, based on the normal vector of APS, which can partition the complicated APS more reasonably, and then perform APC based on the partition results. APC, and simulation experiments on measurement data, suggests that the proposed method achieves higher accuracy than the conventional model-based methods for complicated APS and avoids severe overfitting, realizing the balance between accuracy and credibility. This article verifies the feasibility and effectiveness of using APS distribution information to guide the partition and conduct APC.

The impact of isolation kernel on agglomerative hierarchical clustering algorithms

Agglomerative hierarchical clustering (AHC) is one of the popular clustering approaches. AHC generates a dendrogram that provides richer information and insights from a dataset than partitioning clustering. However, a major problem with existing distance-based AHC methods is: it fails to effectively identify adjacent clusters with varied densities, regardless of the cluster extraction methods applied to the resultant dendrogram. This paper aims to reveal the root cause of this issue and provides a solution by using a data-dependent kernel. We analyse the condition under which existing AHC methods fail to effectively extract clusters, and give the reason why the data-dependent kernel is an effective remedy. This leads to a new approach to kernerlise existing hierarchical clustering algorithms including the traditional AHC algorithms, HDBSCAN, GDL, PHA and HC-OT. Our extensive empirical evaluation shows that the recently introduced Isolation Kernel produces a higher quality or purer dendrogram than distance, Gaussian Kernel and adaptive Gaussian Kernel in all the above mentioned AHC algorithms.

A Taxonomy of Machine Learning Clustering Algorithms, Challenges, and Future Realms

In the field of data mining, clustering has shown to be an important technique. Numerous clustering methods have been devised and put into practice, and most of them locate high-quality or optimum clustering outcomes in the field of computer science, data science, statistics, pattern recognition, artificial intelligence, and machine learning. This research provides a modern, thorough review of both classic and cutting-edge clustering methods. The taxonomy of clustering is presented in this review from an applied angle and the compression of some hierarchical and partitional clustering algorithms with various parameters. We also discuss the open challenges in clustering such as computational complexity, refinement of clusters, speed of convergence, data dimensionality, effectiveness and scalability, data object representation, evaluation measures, data streams, and knowledge extraction; scientists and professionals alike will be able to use it as a benchmark as they strive to advance the state-of-the-art in clustering techniques.

A survey on the utilization of Superpixel image for clustering based image segmentation.

Superpixel become increasingly popular in image segmentation field as it greatly helps image segmentation techniques to segment the region of interest accurately in noisy environment and also reduces the computation effort to a great extent. However, selection of proper superpixel generation techniques and superpixel image segmentation techniques play a very crucial role in the domain of different kinds of image segmentation. Clustering is a well-accepted image segmentation technique and proved their effective performance over various image segmentation field. Therefore, this study presents an up-to-date survey on the employment of superpixel image in combined with clustering techniques for the various image segmentation. The contribution of the survey has four parts namely (i) overview of superpixel image generation techniques, (ii) clustering techniques especially efficient partitional clustering techniques, their issues and overcoming strategies, (iii) Review of superpixel combined with clustering strategies exist in literature for various image segmentation, (iv) lastly, the comparative study among superpixel combined with partitional clustering techniques has been performed over oral pathology and leaf images to find out the efficacy of the combination of superpixel and partitional clustering approaches. Our evaluations and observation provide in-depth understanding of several superpixel generation strategies and how they apply to the partitional clustering method.

Robust two-layer partition clustering of sparse multivariate functional data

A novel elastic time distance for sparse multivariate functional data is proposed and used to develop a robust distance-based two-layer partition clustering method. With this proposed distance, the new approach not only can detect correct clusters for sparse multivariate functional data under outlier settings but also can detect those outliers that do not belong to any clusters. Classical distance-based clustering methods such as density-based spatial clustering of applications with noise (DBSCAN), agglomerative hierarchical clustering, and K-medoids are extended to the sparse multivariate functional case based on the newly-proposed distance. Numerical experiments on simulated data highlight that the performance of the proposed algorithm is superior to the performances of existing model-based and extended distance-based methods. The effectiveness of the proposed approach is demonstrated using Northwest Pacific cyclone tracks data as an example.

Algorithm Comparison on Email Spam Filtering Task

Email has long been a major form of communication among organizations and individual users. In recent years, with the rise of internet use, email spamming has become increasingly common. Spamming has raised security concerns as it causes potential loss to the users with fake advertisements, invalid information, undetected virus, and other harmful information. Various techniques have been developed to facilitate spam filtering, using classifying algorithms to characterize emails into different categories. This article presents an investigation of how machine-learning-based algorithms are used in email spam filtering by providing some previous researches that have shown to be successful. The algorithms are in the range from supervised learning, including the Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Naïve Bayes (NB) to unsupervised learning such as artificial neural networks (ANN) and partitional clustering. This article also presents an experiment that compares different implementations of such algorithms, including the SVM, NB, and K-Nearest Neighbor (KNN). The results reflect that NB gave the highest accuracy. A second attempt of the same experiment was conducted, with an improved data cleaning procedure and larger testing sets. The data collected from the second attempt again show that the NB implementation gave the highest accuracy in detecting spam emails.

Biotypes of major depressive disorder identified by a multiview clustering framework

BackgroundThe advances in resting-state functional magnetic resonance imaging techniques motivate parsing heterogeneity in major depressive disorder (MDD) through neurophysiological subtypes (i.e., biotypes). Based on graph theories, researchers have observed the functional organization of the human brain as a complex system with modular structures and have found wide-spread but variable MDD-related abnormality regarding the modules. The evidence implies the possibility of identifying biotypes using high-dimensional functional connectivity (FC) data in ways that suit the potentially multifaceted biotypes taxonomy. MethodsWe proposed a multiview biotype discovery framework that involves theory-driven feature subspace partition (i.e., “view”) and independent subspace clustering. Six views were defined using intra- and intermodule FC regarding three MDD focal modules (i.e., the sensory-motor system, default mode network, and subcortical network). For robust biotypes, the framework was applied to a large multisite sample (805 MDD participants and 738 healthy controls). ResultsTwo biotypes were stably obtained in each view, respectively characterized by significantly increased and decreased FC compared to healthy controls. These view-specific biotypes promoted the diagnosis of MDD and showed different symptom profiles. By integrating the view-specific biotypes into biotype profiles, a broad spectrum in the neural heterogeneity of MDD and its separation from symptom-based subtypes was further revealed. LimitationsThe power of clinical effects is limited and the cross-sectional nature cannot predict the treatment effects of the biotypes. ConclusionsOur findings not only contribute to the understanding of heterogeneity in MDD, but also provide a novel subtyping framework that could transcend current diagnostic boundaries and data modality.

An Ensemble and Multi-View Clustering Method Based on Kolmogorov Complexity

The ability to build more robust clustering from many clustering models with different solutions is relevant in scenarios with privacy-preserving constraints, where data features have a different nature or where these features are not available in a single computation unit. Additionally, with the booming number of multi-view data, but also of clustering algorithms capable of producing a wide variety of representations for the same objects, merging clustering partitions to achieve a single clustering result has become a complex problem with numerous applications. To tackle this problem, we propose a clustering fusion algorithm that takes existing clustering partitions acquired from multiple vector space models, sources, or views, and merges them into a single partition. Our merging method relies on an information theory model based on Kolmogorov complexity that was originally proposed for unsupervised multi-view learning. Our proposed algorithm features a stable merging process and shows competitive results over several real and artificial datasets in comparison with other state-of-the-art methods that have similar goals.

Unsupervised algorithms to identify potential under-coding of secondary diagnoses in hospitalisations databases in Portugal.

Quantifying and dealing with lack of consistency in administrative databases (namely, under-coding) requires tracking patients longitudinally without compromising anonymity, which is often a challenging task. This study aimed to (i) assess and compare different hierarchical clustering methods on the identification of individual patients in an administrative database that does not easily allow tracking of episodes from the same patient; (ii) quantify the frequency of potential under-coding; and (iii) identify factors associated with such phenomena. We analysed the Portuguese National Hospital Morbidity Dataset, an administrative database registering all hospitalisations occurring in Mainland Portugal between 2011-2015. We applied different approaches of hierarchical clustering methods (either isolated or combined with partitional clustering methods), to identify potential individual patients based on demographic variables and comorbidities. Diagnoses codes were grouped into the Charlson an Elixhauser comorbidity defined groups. The algorithm displaying the best performance was used to quantify potential under-coding. A generalised mixed model (GML) of binomial regression was applied to assess factors associated with such potential under-coding. We observed that the hierarchical cluster analysis (HCA) + k-means clustering method with comorbidities grouped according to the Charlson defined groups was the algorithm displaying the best performance (with a Rand Index of 0.99997). We identified potential under-coding in all Charlson comorbidity groups, ranging from 3.5% (overall diabetes) to 27.7% (asthma). Overall, being male, having medical admission, dying during hospitalisation or being admitted at more specific and complex hospitals were associated with increased odds of potential under-coding. We assessed several approaches to identify individual patients in an administrative database and, subsequently, by applying HCA + k-means algorithm, we tracked coding inconsistency and potentially improved data quality. We reported consistent potential under-coding in all defined groups of comorbidities and potential factors associated with such lack of completeness. Our proposed methodological framework could both enhance data quality and act as a reference for other studies relying on databases with similar problems.

A MIMO–ANFIS-Controlled Solar-Fuel-Cell-Based Switched Capacitor Z-Source Converter for an Off-Board EV Charger

The efficiency of a nation’s progress is determined by a variety of factors; however, transportation plays a critical role in boosting progress because it facilitates trade and communication between countries. The majority of transportation is powered by fossil fuels such as gasoline or diesel, which will be depleted in less than 50 years. Another option is to operate transportation systems after replacing conventional vehicles with electric vehicles (EV). Powering these vehicles with green electricity contributes zero carbon emissions from production to the final product. Together with the controller, an efficient charger ensures that the entire system is reliable and stable. The current work focuses on charging an off-board EV from greener energy sources (both a fuel cell and PV array forming a micro-grid) based on their availability via an efficient converter controlled by an adaptive multi-objective controller. A novel multi-output-based adaptive neuro fuzzy inference system (ANFIS) controller for charging the off-board EV at a constant current and voltage for both line and load regulations is proposed, in the current work. A comparison study of grid partitioning and subtractive clustering was conducted in order to select an optimized algorithm for generating FIS. Novelty is achieved by ensuring closed-loop stability is the main aim of the work. The entire work was created with the MATLAB/Simulink software.

Voltage Regulation Strategy of Distribution Network with Decentralized Wind Power Based on Cluster Partition

Background: With the access to large-scale distributed energy, the voltage off-limit problem of the distribution network has become more and more serious, and the traditional centralized voltage control method has been difficult to meet the demand of distribution network control and market development. Methods: In this paper, a voltage regulation strategy based on decentralized wind power clustering is proposed. Considering the active and reactive power regulation ability of decentralized wind power, based on the cluster division of distributed energy, the goal is to minimize power regulation and voltage fluctuation within the cluster. Combining the K-means clustering algorithm and optimized PSO algorithm for voltage regulation within the cluster ensures that the voltagecrossing problem is solved. After the voltage regulation of all clusters, the auxiliary service transaction of voltage regulation is implemented to complete the voltage regulation of the whole network. Results: Taking ieee33 bus system as an example, the MATLAB simulation shows that the decentralized wind power access triggered voltage overrun. Using the proposed cluster division method to cluster the model, the clustered voltage regulation of the nodes with voltage overrun or reaching the limit resulted in greater voltage down-regulation than the overall voltage regulation. Conclusion: The proposed voltage regulation strategy has better advantages. It is verified that the proposed voltage regulation strategy not only solves the voltage overrun problem, but also reduces voltage fluctuations and the amount of power regulation used for voltage regulation, increasing the utilization of distributed energy with better superiority.

Simplifying Clustering with Graph Neural Networks

The objective functions used in spectral clustering are generally composed of two terms: i) a term that minimizes the local quadratic variation of the cluster assignments on the graph and; ii) a term that balances the clustering partition and helps avoiding degenerate solutions. This paper shows that a graph neural network, equipped with suitable message passing layers, can generate good cluster assignments by optimizing only a balancing term.Results on attributed graph datasets show the effectiveness of the proposed approach in terms of clustering performance and computation time.

Performance analysis of Kmeans with modified initial centroid selection algorithms and developed Kmeans9+ model

Partition the sample space to the user requirement is an easy and efficient cluster method in many applications. Kmeans is one of the simple and efficient partition algorithms used in many cluster solutions. We suggest Multivariant Silhouette method to predict the cluster count value for Kmeans algorithm. The work proposes three new algorithms for initial seed selection. The Kmeans algorithm is modified using statistical measure Mean, Median, Partition centre for the cluster centroid calculation. In conventional Kmeans algorithm, the samples are compared with all the partition centroids to decide the inclusion of a sample in a cluster. In Kmeans9+ algorithm, the samples are compared only with the centroids of current and eight nearest neighbouring cluster partitions. It reduces the needless comparisons of samples with cluster centroids and improve the efficiency of the algorithm. The performance of created algorithms is evaluated using data from UCI and KBPE SSLC. Cluster efficiency analysis is done using the cluster evaluation indices such as Silhouette and Dunn Index. Nine nearest neighbour uniform partition cluster model Kmeans9+ improve the performance of the K means algorithm and obtain the natural cluster results with minimum iterations.

Online risk-based portfolio allocation on subsets of crypto assets applying a prototype-based clustering algorithm

Mean-variance portfolio optimization models are sensitive to uncertainty in risk-return estimates, which may result in poor out-of-sample performance. In particular, the estimates may suffer when the number of assets considered is high and the length of the return time series is not sufficiently long. This is precisely the case in the cryptocurrency market, where there are hundreds of crypto assets that have been traded for a few years. We propose enhancing the mean-variance (MV) model with a pre-selection stage that uses a prototype-based clustering algorithm to reduce the number of crypto assets considered at each investment period. In the pre-selection stage, we run a prototype-based clustering algorithm where the assets are described by variables representing the profit-risk duality. The prototypes of the clustering partition are automatically examined and the one that best suits our risk-aversion preference is selected. We then run the MV portfolio optimization with the crypto assets of the selected cluster. The proposed approach is tested for a period of 17 months in the whole cryptocurrency market and two selections of the cryptocurrencies with the higher market capitalization (175 and 250 cryptos). We compare the results against three methods applied to the whole market: classic MV, risk parity, and hierarchical risk parity methods. We also compare our results with those from investing in the market index CCI30. The simulation results generally favor our proposal in terms of profit and risk-profit financial indicators. This result reaffirms the convenience of using machine learning methods to guide financial investments in complex and highly-volatile environments such as the cryptocurrency market.

A hybrid machine learning approach for prediction of conversion from mild cognitive impairment to dementia

Mild cognitive impairment (MCI) represents a precursor to dementia for many individuals; however, some forms of MCI tend to remain stable over time and do not progress to dementia. In fact, conversion rates vary substantially depending on the diagnostic criteria used and the nature of the analytic sample and clinical setting. To identify personalized strategies to prevent or slow the progression of dementia and to support the clinical development of novel treatments, we need to develop new approaches for modelling disease progression that can differentiate between progressive and non-progressive MCI subjects. The aim of this study was to develop a novel prognostic machine learning (ML) framework utilising longitudinal information encoded in efficient, cost-effective, and non-invasive markers to identify MCI subjects that are at risk for developing dementia. Our approach was developed using the dataset from the National Alzheimer’s Coordinating Center. We built two prognostic models based on the patient data from 3 (n = 768) (Model 1) and 4 (n = 409) (Model 2) assessment visits. A novel hybrid prognostic approach, using cognitive trajectory classes, generated through unsupervised learning (Stage 1), as input in supervised MLmodels (Stage 2), was developed and systematically tested. Our unsupervised learning approach (Stage 1) involved: (i) the implementation of the longitudinal data partitioning method allowing for clustering trajectories based on their shapes; (ii) validation of the optimal number of clusters using three different Clustering Validity Indices (CVIs), and (iii) application of the fusion-based methods for combining CVIs into the fused normalized CVI scores, averaged for each cluster partition to determine the final number of trajectory classes for each type of clinical scores. In Stage 2, we built four types of prognostic models based on random forest (RF), Support Vector Machines (SVM), logistic regression (LR), and kNN ensemble approaches. Classification models incorporating both clinical scores and cognitive trajectory classes input showed up to 6.5 % higher accuracy than models based only on clinical scores (p < 0.05 in all cases). Given the patient data from three time points (Model 1), the highest recorded prediction accuracy was achieved for the ensemble and RF model, i.e., 85.0 % (standard deviation: 3.1 %) and 84.6 % (4.1 %) respectively. Using the patient data from four time points (Model 2), the highest accuracy was reported for RF and ensemble models, i.e., 87.5 % (6.1 %) and 86.8 % (3.7 %) respectively. We showed that the incorporation of the output of unsupervised learning significantly improved the performance of supervised ML models. Our prognostic framework can be applied to improve recruitment in clinical trials and to select early interventions for individuals at high risk of developing dementia.

Identifying the partition of port clusters and hierarchical structure for port networks

Identifying the partition of port clusters and hierarchical structure for port networks

Hybrid Deep Learning Enabled Load Prediction for Energy Storage Systems

Recent economic growth and development have considerably raised energy consumption over the globe. Electric load prediction approaches become essential for effective planning, decision-making, and contract evaluation of the power systems. In order to achieve effective forecasting outcomes with minimum computation time, this study develops an improved whale optimization with deep learning enabled load prediction (IWO-DLELP) scheme for energy storage systems (ESS) in smart grid platform. The major intention of the IWO-DLELP technique is to effectually forecast the electric load in SG environment for designing proficient ESS. The proposed IWO-DLELP model initially undergoes pre-processing in two stages namely min-max normalization and feature selection. Besides, partition clustering approach is applied for the decomposition of data into distinct clusters with respect to distance and objective functions. Moreover, IWO with bidirectional gated recurrent unit (BiGRU) model is applied for the prediction of load and the hyperparameters are tuned by the use of IWO algorithm. The experiment analysis reported the enhanced results of the IWO-DLELP model over the recent methods interms of distinct evaluation measures.

Detection of False Data Injection Attacks in Smart Grid Based on Joint Dynamic and Static State Estimation

Power system state estimation is an essential component of the modern power system energy management system (EMS), and accurate state estimation is an indispensable basis for subsequent work. However, the attacker can inject biases into measurements to launch false data injection attacks (FDIAs) in smart grids, which ultimately cause state estimates to deviate from security values. This paper proposed the joint use of static state estimation and dynamic state estimation to detect the FDIA, i.e. the joint use of weighted least squares (WLS) and extended Kalman filter (EKF) with exponential weighting function (WEKF), which improves the robustness of state estimation. Since the WLS estimation considers only the measurements at the current moment, the recursive feature of the WEKF enables the estimation process to involve both historical state and current measurements. Therefore, consistency tests and residual tests were performed using the estimations of WLS and WEKF to effectively detect FDIA. In addition, a cluster partitioning approach with approximate equal redundancy of subsystems is proposed to locate the FDIA. The detection of FDIA triggers the partitioning of the network system, and then the chi-square test is used separately in each sub-network to determine the location of FDIA. Finally, the experimental results in the IEEE-14 bus system and the IEEE-30 bus system demonstrate that the approach can effectively detect and locate FDIAs.