Kernel K-means Research Articles

Scalable kernel k-means clustering with Nyström approximation

Kernel k-means clustering can correctly identify and extract a far more varied collection of cluster structures than the linear k-means clustering algorithm. However, kernel k- means clustering is ...

Efficient Multiple Kernel k-Means Clustering With Late Fusion

The recently proposed multiple-kernel clustering algorithms have demonstrated promising performance in various applications. However, most of the existing methods suffer from high computational complexity and intensive time cost. To address this issue, we propose to fulfill multiple kernel k-means clustering via a late fusion manner. In specific, we design two multiple kernel k-means algorithms with late fusion, whose computational complexities linearly grow with the number of samples. The proposed algorithms integrally optimize the various clustering matrices into the optimal consensus clustering results iteratively. Furthermore, we analyze the computational complexities of the proposed algorithms and theoretically prove their convergence. As demonstrated by the experiments on six benchmark datasets, our algorithms achieve comparable or better clustering performance to state-of-the-art ones with less time cost, which demonstrates the advantages of the late fusion in multiple kernel k-means.

Consensus Multiple Kernel K-Means Clustering With Late Fusion Alignment and Matrix-Induced Regularization

Multiple kernel clustering (MKC) attracts considerable attention due to its competitive performance in unsupervised learning. However, we observe that most of the existing MKC approaches do not sufficiently consider the correlation between different clustering partitions. As a result, the existing methods would cause redundant and low diversity of selected clustering partitions which deteriorate clustering performance. To address these issues, we propose an effective and efficient multiple kernel $k$ -means clustering method termed Consensus Multiple Kernel Clustering with Late Fusion Alignment and Matrix-Induced Regularization (CMKC-LFA-MR) . Specifically, the correlations between different clustering partitions are calculated as a matrix-induced regularization to encourage the diversity of clustering results. Moreover, we propose to maximally align the consensus partition with the weighted base partitions. The proposed algorithm jointly optimizes the basic clustering partitions and the optimal consensus clustering result. To solve the resultant optimization problem, a three-step alternate algorithm is proposed with both theoretically and experimentally proved convergence. As demonstrated by the experiments on six benchmark datasets, our algorithm outperforms the existing state-of-the-art multi-kernel methods in clustering performance with less time complexity, which demonstrates the effectiveness and efficiency of our proposed algorithm.

Synthetic direct demodulation method and its applications in Insight-HXMT data analysis

Aims. A modulation equation relates the observed data to the object where the observation is approximated by a linear system. Reconstructing the object from the observed data is therefore equivalent to solving the modulation equation. In this work we present the synthetic direct demodulation (synDD) method to reduce the dimensionality of a general modulation equation and solve the equation in its sparse representation. Methods. A principal component analysis is used to reduce the dimensionality of the kernel matrix and k-means clustering is applied to its sparse representation in order to decompose the kernel matrix into a weighted sum of a series of circulant matrices. The matrix-vector and matrix-matrix multiplication complexities are therefore reduced from polynomial time to linear-logarithmic time. A general statistical solution of the modulation equation in sparse representation is derived. Several data-analysis pipelines are designed for the Hard X-ray modulation Telescope (Insight-HXMT) based on the synDD method. Results. In this approach, a large set of data originating from the same object but sampled irregularly and/or observed with different instruments in multiple epochs can be reduced simultaneously in a synthetic observation model. We suggest using the proposed synDD method in Insight-HXMT data analysis especially for the detection of X-ray transients and monitoring time-varying objects with scanning observations.

Late Fusion Incomplete Multi-View Clustering.

Incomplete multi-view clustering optimally integrates a group of pre-specified incomplete views to improve clustering performance. Among various excellent solutions, multiple kernel $k$k-means with incomplete kernels forms a benchmark, which redefines the incomplete multi-view clustering as a joint optimization problem where the imputation and clustering are alternatively performed until convergence. However, the comparatively intensive computational and storage complexities preclude it from practical applications. To address these issues, we propose Late Fusion Incomplete Multi-view Clustering (LF-IMVC) which effectively and efficiently integrates the incomplete clustering matrices generated by incomplete views. Specifically, our algorithm jointly learns a consensus clustering matrix, imputes each incomplete base matrix, and optimizes the corresponding permutation matrices. We develop a three-step iterative algorithm to solve the resultant optimization problem with linear computational complexity and theoretically prove its convergence. Further, we conduct comprehensive experiments to study the proposed LF-IMVC in terms of clustering accuracy, running time, advantages of late fusion multi-view clustering, evolution of the learned consensus clustering matrix, parameter sensitivity and convergence. As indicated, our algorithm significantly and consistently outperforms some state-of-the-art algorithms with much less running time and memory.

Incomplete Multiview Clustering via Late Fusion

In real-world applications of multiview clustering, some views may be incomplete due to noise, sensor failure, etc. Most existing studies in the field of incomplete multiview clustering have focused on early fusion strategies, for example, learning subspace from multiple views. However, these studies overlook the fact that clustering results with the visible instances in each view could be reliable under the random missing assumption; accordingly, it seems that learning a final clustering decision via late fusion of the clustering results from incomplete views would be more natural. To this end, we propose a late fusion method for incomplete multiview clustering. More specifically, the proposed method performs kernel k-means clustering on the visible instances in each view and then performs a late fusion of the clustering results from different views. In the late fusion step of the proposed method, we encode each view's clustering result as a zero-one matrix, of which each row serves as a compressed representation of the corresponding instance. We then design an alternate updating algorithm to learn a unified clustering decision that can best group the visible compressed representations in each view according to the k-means clustering objective. We compare the proposed method with several commonly used imputation methods and a representative early fusion method on six benchmark datasets. The superior clustering performance observed validates the effectiveness of the proposed method.

Predicting clinical outcomes in neuroblastoma with genomic data integration

BackgroundNeuroblastoma is a heterogeneous disease with diverse clinical outcomes. Current risk group models require improvement as patients within the same risk group can still show variable prognosis. Recently collected genome-wide datasets provide opportunities to infer neuroblastoma subtypes in a more unified way. Within this context, data integration is critical as different molecular characteristics can contain complementary signals. To this end, we utilized the genomic datasets available for the SEQC cohort patients to develop supervised and unsupervised models that can predict disease prognosis.ResultsOur supervised model trained on the SEQC cohort can accurately predict overall survival and event-free survival profiles of patients in two independent cohorts. We also performed extensive experiments to assess the prediction accuracy of high risk patients and patients without MYCN amplification. Our results from this part suggest that clinical endpoints can be predicted accurately across multiple cohorts. To explore the data in an unsupervised manner, we used an integrative clustering strategy named multi-view kernel k-means (MVKKM) that can effectively integrate multiple high-dimensional datasets with varying weights. We observed that integrating different gene expression datasets results in a better patient stratification compared to using these datasets individually. Also, our identified subgroups provide a better Cox regression model fit compared to the existing risk group definitions.ConclusionAltogether, our results indicate that integration of multiple genomic characterizations enables the discovery of subtypes that improve over existing definitions of risk groups. Effective prediction of survival times will have a direct impact on choosing the right therapies for patients.ReviewersThis article was reviewed by Susmita Datta, Wenzhong Xiao and Ziv Shkedy.

Does deep learning help topic extraction? A kernel k-means clustering method with word embedding

Abstract Topic extraction presents challenges for the bibliometric community, and its performance still depends on human intervention and its practical areas. This paper proposes a novel kernel k-means clustering method incorporated with a word embedding model to create a solution that effectively extracts topics from bibliometric data. The experimental results of a comparison of this method with four clustering baselines (i.e., k-means, fuzzy c-means, principal component analysis, and topic models) on two bibliometric datasets demonstrate its effectiveness across either a relatively broad range of disciplines or a given domain. An empirical study on bibliometric topic extraction from articles published by three top-tier bibliometric journals between 2000 and 2017, supported by expert knowledge-based evaluations, provides supplemental evidence of the method’s ability on topic extraction. Additionally, this empirical analysis reveals insights into both overlapping and diverse research interests among the three journals that would benefit journal publishers, editorial boards, and research communities.

Non-Exhaustive, Overlapping Clustering.

Traditional clustering algorithms, such as K-Means, output a clustering that is disjoint and exhaustive, i.e., every single data point is assigned to exactly one cluster. However, in many real-world datasets, clusters can overlap and there are often outliers that do not belong to any cluster. While this is a well-recognized problem, most existing algorithms address either overlap or outlier detection and do not tackle the problem in a unified way. In this paper, we propose an intuitive objective function, which we call the NEO-K-Means (Non-Exhaustive, Overlapping K-Means) objective, that captures the issues of overlap and non-exhaustiveness in a unified manner. Our objective function can be viewed as a reformulation of the traditional K-Means objective, with easy-to-understand parameters that capture the degrees of overlap and non-exhaustiveness. By considering an extension to weighted kernel K-Means, we show that we can also apply our NEO-K-Means idea to overlapping community detection, which is an important task in network analysis. To optimize the NEO-K-Means objective, we develop not only fast iterative algorithms but also more sophisticated algorithms using low-rank semidefinite programming techniques. Our experimental results show that the new objective and algorithms are effective in finding ground-truth clusterings that have varied overlap and non-exhaustiveness; for the case of graphs, we show that our method outperforms state-of-the-art overlapping community detection algorithms.

Comparison assessment of low rank sparse-PCA based-clustering/classification for automatic mineral identification in long wave infrared hyperspectral imagery

The developments in hyperspectral technology in different applications are known in many fields particularly in remote sensing, airborne imagery, mineral identification and core logging. The automatic mineral identification system provides considerable assistance in geology to identify mineral automatically. Here, the proposed approach addresses an automated system for mineral (i.e. pyrope, olivine, quartz) identification in the long-wave infrared (7.7–11.8 μm - LWIR) ground-based spectroscopy. A low-rank Sparse Principal Component Analysis (Sparse-PCA) based spectral comparison methods such as Spectral Angle Mapper (SAM), Spectral Information Divergence (SID), Normalized Cross Correlation (NCC) have been used to extract the features in the form of false colors composite. Low-rank Sparse-PCA is used to extract the spectral reference which and showed high similarity to the ASTER (JPL/NASA) spectral library. For decision making step, two methods used to establish a comparison between a kernel Extreme Learning Machine (ELM) and Principal Component Analysis (PCA) kernel K-means clustering. ELM yields classification accuracy up to 76.69% using SAM based polynomial kernel ELM for pyrope mixture, and 70.95% using SAM based sigmoid kernel ELM for olivine mixture. This accuracy is slightly lower as compared to clustering which yields an identification accuracy of 84.91% (NCC) and 69.9% (SAM). However, the supervised classification significantly depends on the number of training samples and is considerably more difficult as compared to clustering due to labeling and training limitations. Moreover, the results indicate considerable similarity between the spectra from low rank approximation from the spectra of pure sample and the spectra from the ASTER spectral library.

A new model order reduction method based on global kernelk-means clustering: Application in health monitoring of plate using Lamb wave propagation and impedance method

In this paper, a new method based on global kernel k-means clustering (GKKMC) has been developed as model order reduction to model electromechanical impedance and Lamb wave in a plate structure. First, a model, based on the spectral finite element method, is developed to simulate piezoelectric wafer active sensor (PWAS) induced electromechanical impedance and Lamb wave propagation. Second, the related coupled electromechanical field equations are solved in 3 dimensions, then an electric voltage signal is applied to PWAS actuator, and finally the produced voltage in PWAS sensor is calculated. In reality, high frequency impedance and Lamb wave simulation in a plate need high degrees of freedom, which leads to a very slow simulation in time and frequency domain calculations. To overcome this problem, GKKMC as a model order reduction approach is proposed and applied. For this purpose, in Lamb wave modeling, a time snapshot method based on the Rayleigh damping effect has been introduced. This proposed method not only has a high accuracy and less time to simulate the problem but also needs lower memory than a full order model. A comparison between the results obtained by GKKMC, the conventional balanced proper orthogonal decomposition method, and a full order method with experimental results have been carried out demonstrating a good agreement. The results show that using this new model order reduction based on GKKMC, simulation time speedups can be reached up to 6 times faster compared with applying a conventional balanced proper orthogonal decomposition method.

Efficient similarity-based data clustering by optimal object to cluster reallocation.

We present an iterative flat hard clustering algorithm designed to operate on arbitrary similarity matrices, with the only constraint that these matrices be symmetrical. Although functionally very close to kernel k-means, our proposal performs a maximization of average intra-class similarity, instead of a squared distance minimization, in order to remain closer to the semantics of similarities. We show that this approach permits the relaxing of some conditions on usable affinity matrices like semi-positiveness, as well as opening possibilities for computational optimization required for large datasets. Systematic evaluation on a variety of data sets shows that compared with kernel k-means and the spectral clustering methods, the proposed approach gives equivalent or better performance, while running much faster. Most notably, it significantly reduces memory access, which makes it a good choice for large data collections. Material enabling the reproducibility of the results is made available online.

Big Data Clustering with Kernel k-Means: Resources, Time and Performance

Data clustering is an unsupervised learning task that has found many applications in various scientific fields. The goal is to find subgroups of closely related data samples (clusters) in a set of unlabeled data. A classic clustering algorithm is the so-called k-Means. It is very popular, however, it is also unable to handle cases in which the clusters are not linearly separable. Kernel k-Means is a state of the art clustering algorithm, which employs the kernel trick, in order to perform clustering on a higher dimensionality space, thus overcoming the limitations of classic k-Means regarding the non-linear separability of the input data. With respect to the challenges of Big Data research, a field that has established itself in the last few years and involves performing tasks on extremely large amounts of data, several adaptations of the Kernel k-Means have been proposed, each of which has different requirements in processing power and running time, while also incurring different trade-offs in performance. In this paper, we present several issues and techniques involving the usage of Kernel k-Means for Big Data clustering and how the combination of each component in a clustering framework fares in terms of resources, time and performance. We use experimental results, in order to evaluate several combinations and provide a recommendation on how to approach a Big Data clustering problem.

CPU and GPU parallelized kernel K-means

K-means is one of the most commonly used clustering algorithms, with diverse scope for implementation in the signal processing, artificial intelligence and image processing fields, among others. Different variations and improvements of K-means exist, with kernel K-means being the most famous. K-means has been the subject of many studies aiming to improve its hardware and software implementations. Several of these studies have focused on the parallelization of K-means. Kernelization mainly transforms the data into a feature space of high dimensionality by computing the inner product between each possible data pair. As a result of kernelization, kernel K-means involves several computational steps and has additional computational requirements. As a result, kernel K-means has not seen the same interest and much can still be done in terms of its parallelized and robust implementations. This original research studies and develops different parallel implementations of kernel K-means on both the CPU and the GPU. The proposed CPU implementations use OpenMP and BLAS, while for the developed GPU implementation, the concentration is on CUDA available on Nvidia GPUs. Several datasets of a varying number of features and patterns are used. The results show that CUDA generally provides the best run-times with speedups varying between two to more than two hundred times over a single-core CPU implementation according to the used dataset.

Educational Data Clustering in a Weighted Feature Space Using Kernel K-Means and Transfer Learning Algorithms

Educational data clustering on the students’ data collected with a program can find several groups of the students sharing the similar characteristics in their behaviors and study performance. For some programs, it is not trivial for us to prepare enough data for the clustering task. Data shortage might then influence the effectiveness of the clustering process and thus, true clusters can not be discovered appropriately. On the other hand, there are other programs that have been well examined with much larger data sets available for the task. Therefore, it is wondered if we can exploit the larger data sets from other source programs to enhance the educational data clustering task on the smaller data sets from the target program. Thanks to transfer learning techniques, a transfer-learning-based clustering method is defined with the kernel k-means and spectral feature alignment algorithms in our paper as a solution to the educational data clustering task in such a context. Moreover, our method is optimized within a weighted feature space so that how much contribution of the larger source data sets to the clustering process can be automatically determined. This ability is the novelty of our proposed transfer learning-based clustering solution as compared to those in the existing works. Experimental results on several real data sets have shown that our method consistently outperforms the other methods using many various approaches with both external and internal validations.

The Spatial–Temporal Trend Analysis of Schistosomiasis from 1997 to 2010 in Anhui Province, Eastern China

Schistosomiasis is still prevalent in some parts of China. A shift in strategy from morbidity control to elimination has led to great strides in the past several decades. The objective of this study was to explore the spatial and temporal characteristics of schistosomiasis in Anhui, an eastern province of China. In this study, township-based parasitological data were collected from annual cross-sectional surveys during 1997-2010. The kernel k-means method was used to identify spatial clusters of schistosomiasis, and an empirical mode decomposition technique was used to analyze the temporal trend for Schistosoma japonicum in each clustered region. Overall, the prevalence of schistosomiasis remained at a low level except for the resurgence in 2005. According to the Caliński-Harabas index, all the townships were classified into three different clusters (median prevalence: 3.6 per 10,100, 1.8 per 10,000 and 1.7 per 10,000), respectively representing high-, median-, and low-risk clusters. There was an increasing tendency observed for the disease over time. The prevalence increased rapidly from 2003 to 2005, peaked in 2006, and then decreased afterward in the high-risk cluster. A moderate increase was observed in the median-risk cluster from 1998 to 2006, but there was an obvious decreasing tendency in the low-risk cluster after the year 2000. The spatial and temporal patterns of schistosomiasis were nonsynchronous across the three clusters. Disease interventions may be adjusted according to the risk levels of the clusters.

Falling the Risk of Consumer Relocation by Using Big Data Clustering Algorithm

In the world of survival of the fittest the market contests are increasing and organization of the consumers has become all the focus for business. For the same, different consumer migration or mix predicting representation are been developed and used by many Bigdata industries which are struggling to emerge. At the latest, there exits many clustering algorithms which focus on the same. And also many clustering algorithms also have been put forward for the same. One among these algorithms are SCM and SCSCM whose major focus is on churn predictions. They make use Hadoop map Reduce framework. A More efficient and resourceful clustering algorithm is parallel kernel k-means is proposed in this paper. Our proposed system design efficiently and effectively performs clustering huge amount of heterogeneous dataset. Proposed algorithm can cluster structured, unstructured and semi-structured data considered simultaneously for clustering. The kernels run in parallel accepting datasets and balancing the load of huge data. It’s time consumption compared to other, is expected to be very less, faster execution of clustering and enhances the profit to a large extent.

MMKK++ algorithm for clustering heterogeneous images into an unknown number of clusters

In this paper we present a suggested automatic clustering procedure with the main aim to predict the number of clusters of unknown, heterogeneous images. We used the state-of-the-art Fisher-vector for mathematical representation of the images and these vectors were considered as input data points for the clustering algorithm. We implemented a novel variant of K-means, the kernel K-means++, furthermore the min-max kernel K-means plusplus (MMKK++) as clustering method. The proposed approach examines some candidate cluster numbers and uses the law of large numbers in order to choose the optimal cluster size. We conducted experiments on four image sets to demonstrate the efficiency of our solution. The first two image sets are subsets of different popular collections; the third is their union; the fourth is the complete Caltech101 image set.

The Classification of Diabetes Mellitus Using Kernel k-means

Diabetes Mellitus is a metabolic disorder which is characterized by chronicle hypertensive glucose. Automatics detection of diabetes mellitus is still challenging. This study detected diabetes mellitus by using kernel k-Means algorithm. Kernel k-means is an algorithm which was developed from k-means algorithm. Kernel k-means used kernel learning that is able to handle non linear separable data; where it differs with a common k-means. The performance of kernel k-means in detecting diabetes mellitus is also compared with SOM algorithms. The experiment result shows that kernel k-means has good performance and a way much better than SOM.

Kernel Clustering: Density Biases and Solutions.

Kernel methods are popular in clustering due to their generality and discriminating power. However, we show that many kernel clustering criteria have density biases theoretically explaining some practically significant artifacts empirically observed in the past. For example, we provide conditions and formally prove the density mode isolation bias in kernel K-means for a common class of kernels. We call it Breiman's bias due to its similarity to the histogram mode isolation previously discovered by Breiman in decision tree learning with Gini impurity. We also extend our analysis to other popular kernel clustering methods, e.g.,average/normalized cut or dominant sets, where density biases can take different forms. For example, splitting isolated points by cut-based criteria is essentially the sparsest subset bias, which is the opposite of the density mode bias. Our findings suggest that a principled solution for density biases in kernel clustering should directly address data inhomogeneity. We show that density equalization can be implicitly achieved using either locally adaptive weights or locally adaptive kernels. Moreover, density equalization makes many popular kernel clustering objectives equivalent. Our synthetic and real data experiments illustrate density biases and proposed solutions. We anticipate that theoretical understanding of kernel clustering limitations and their principled solutions will be important for a broad spectrum of data analysis applications across the disciplines.