Kernel-based Clustering Algorithm Research Articles

Kernel-based clustering via Isolation Distributional Kernel

Clustering has become one of the widely used automatic data-labeling techniques applied in a variety of disciplines. Kernel-based clustering is a technique designed to identify non-linearly separable clusters with irregular shapes. However, existing kernel-based clustering algorithms usually produce weaker clustering outcomes than density-based clustering, and they have high computational cost which limits their applications to small datasets only. In this paper, we contend that these limitations are mainly due to the use of (a) the Expectation-and-Maximization algorithm as an optimization procedure, and (b) a non-adaptive kernel. In addition, to address the limitations of current kernel-based algorithms, we propose the first clustering algorithm that employs an adaptive distributional kernel without any optimization, while achieving a similar optimization objective function. We demonstrate its superior performance of identifying complex clusters on massive datasets under different real-world application scenarios.

Fast kernel [formula omitted]-means clustering using incomplete Cholesky factorization

Kernel-based clustering algorithms can identify and capture the nonlinear structure in datasets, thereby achieving better performance than linear clustering. However, the large amount of memory involved in computing and storing the entire kernel matrix makes it difficult for kernel-based clustering to handle large-scale datasets. In this study, we prove that an incomplete Cholesky factorization (InCF) can generate an approximate matrix with rank s for a kernel matrix after s iterations. We also show that the approximation error decreases exponentially as the number of iterations increases when the exponential decay of the kernel matrix eigenvalues is sufficiently fast. We therefore employ InCF to accelerate kernel clustering and save memory space. The key idea of the proposed kernel k-means clustering using InCF is that the entire kernel matrix is approximated as the product of a low-rank matrix and its transpose. Then, linear k-means clustering is applied to the columns of the transpose of the low-rank matrix. We prove that the clustering error achieved by this method decreases exponentially as the rank of the approximate matrix increases. Experimental results show that the performance of the proposed algorithm is similar to that of the kernel k-means clustering algorithm, but the method can be applied to large-scale datasets.

RKT2FCM: RBF Kernel-Based Type-2 Fuzzy Clustering

Fuzzy clustering is used to cluster data such that each point may pertain to a number of clusters. Fuzzy clustering has applications in the areas like pattern recognition, medical segmentation, recommender systems etc. Kernel based clustering algorithms are performing better as in Kernel based algorithms, clustering is performed by mapping data to high dimension space. RBF kernel type-2 fuzzy-c-means (RKT2FCM) algorithm has been proposed using the RBF function in this paper. A few standard datasets are used to better analysis and compare performance of proposed algorithm with Fuzzy-C-Means (FCM), Type-2 Fuzzy-C-Means (T2FCM), Kernalized Fuzzy-C-Means (KFCM), FCM with new distance metric (FCM-σ). KT2FCM algorithm uses RBF mapping function and hence gives better performance. Experimental analysis of the proposed algorithm has been given in this paper.

Graph Kernel Based Clustering Algorithm in MANETs

The mobile ad hoc network (MANET) is a kind of dynamic, easy to construct and universal network, which has been widely concerned by a large number of researchers. Graph theory provides an effective theoretical tool for MANETs modeling and analysis. Clustering is one of the most effective methods to measure network performance with different attributes. This paper gives the basic concept of graph kernel and discusses the principle of optimizing graph kernel and multi-graph kernel. In this paper, we propose a Graph Kernel based Clustering Algorithm in MANETs (GKCA). The GKCA algorithm gives the basic concept of graph kernel, discusses the principle of optimizing graph kernel and multi-graph kernel, and proposes the basic principle based on $d$ -hop graph kernel. GKCA algorithm uses shortest path (SP) to connect different cluster head nodes for packet transmission. The performance of GKCA algorithm, such as the control packets ratio, packets loss ratio, and average end-to-end delay are experimentally evaluated using network simulation (NS2) software. Experimental analysis shows that the proposed approach is efficient, and its performance advantage in dynamic mobile networks is promising.

Autoencoder-based cluster ensembles for single-cell RNA-seq data analysis

BackgroundSingle-cell RNA-sequencing (scRNA-seq) is a transformative technology, allowing global transcriptomes of individual cells to be profiled with high accuracy. An essential task in scRNA-seq data analysis is the identification of cell types from complex samples or tissues profiled in an experiment. To this end, clustering has become a key computational technique for grouping cells based on their transcriptome profiles, enabling subsequent cell type identification from each cluster of cells. Due to the high feature-dimensionality of the transcriptome (i.e. the large number of measured genes in each cell) and because only a small fraction of genes are cell type-specific and therefore informative for generating cell type-specific clusters, clustering directly on the original feature/gene dimension may lead to uninformative clusters and hinder correct cell type identification.ResultsHere, we propose an autoencoder-based cluster ensemble framework in which we first take random subspace projections from the data, then compress each random projection to a low-dimensional space using an autoencoder artificial neural network, and finally apply ensemble clustering across all encoded datasets to generate clusters of cells. We employ four evaluation metrics to benchmark clustering performance and our experiments demonstrate that the proposed autoencoder-based cluster ensemble can lead to substantially improved cell type-specific clusters when applied with both the standard k-means clustering algorithm and a state-of-the-art kernel-based clustering algorithm (SIMLR) designed specifically for scRNA-seq data. Compared to directly using these clustering algorithms on the original datasets, the performance improvement in some cases is up to 100%, depending on the evaluation metric used.ConclusionsOur results suggest that the proposed framework can facilitate more accurate cell type identification as well as other downstream analyses. The code for creating the proposed autoencoder-based cluster ensemble framework is freely available from https://github.com/gedcom/scCCESS

Parallel Computation Performingkernel-Based Clustering Algorithm using Particle Swarm Optimization for the Big Data Analytics

Digital data has been accelerating day by day with a bulk of dimensions. Analysis of such an immense quantity of data popularly termed as big data, which requires tremendous data analysis scalable techniques. Clustering is an appropriate tool for data analysis to observe hidden similar groups inside the data. Clustering distinct datasets involve both Linear Separable and Non-Linear Separable clustering algorithms by defining and measuring their inter-point similarities as well as non-linear similarity measures. Problem Statement: Yet there are many productive clustering algorithms to cluster linearly; they do not maintain quality clusters.Kernel-based algorithms make use of non-linear similarity measures to define similarity while forming clusters specifically with arbitrary shapes and frequencies. Existing System:Current Kernel-based clustering algorithms have few restraints concerning complexity, memory, and performance. Time and Memory will increase equally when the size of the dataset increase. It is challenging to elect kernel similarity function for different datasets. We have classical random sampling and low-rank matrix approximation linear clustering algorithms with high cluster quality and low memory essentials. Proposed work: in our research, we have introduced a parallel computation performing Kernel-based clustering algorithm using Particle Swarm Optimization approach. This methodology can cluster large datasets having maximum dimensional values accurately and overcomes the issues of high dimensional datasets.

Effect of Kernel Learning in Unsupervised Learning for Clustering High Dimensional Databases

This paper reviews the effectiveness of kernel learning in unsupervised data analysis using clustering. Cluster analysis is an explorative data analysis tool that assists in discovering hidden patterns or natural grouping and has many effective applications in various disciplines. The unison of kernel learning with the objective of unsupervised clustering algorithms facilitates in recognizing non linear structures in high dimensional data containing outliers with heavy noise. The recent kernel clustering methods considered in this paper are the kernelized versions of K-Means, Fuzzy C-Means, Possibilistic C-Means and Intuitionistic Fuzzy C-Means. Computational complexities in kernel based clustering algorithms are quiet prominent and our objective is to understand the performance gains while using kernels in clustering. Experimental studies of this paper substantiate that kernel based clustering algorithms yields significant improvements over their traditional counterparts.

Impact of similarity metrics on single-cell RNA-seq data clustering.

Advances in high-throughput sequencing on single-cell gene expressions [single-cell RNA sequencing (scRNA-seq)] have enabled transcriptome profiling on individual cells from complex samples. A common goal in scRNA-seq data analysis is to discover and characterise cell types, typically through clustering methods. The quality of the clustering therefore plays a critical role in biological discovery. While numerous clustering algorithms have been proposed for scRNA-seq data, fundamentally they all rely on a similarity metric for categorising individual cells. Although several studies have compared the performance of various clustering algorithms for scRNA-seq data, currently there is no benchmark of different similarity metrics and their influence on scRNA-seq data clustering. Here, we compared a panel of similarity metrics on clustering a collection of annotated scRNA-seq datasets. Within each dataset, a stratified subsampling procedure was applied and an array of evaluation measures was employed to assess the similarity metrics. This produced a highly reliable and reproducible consensus on their performance assessment. Overall, we found that correlation-based metrics (e.g. Pearson's correlation) outperformed distance-based metrics (e.g. Euclidean distance). To test if the use of correlation-based metrics can benefit the recently published clustering techniques for scRNA-seq data, we modified a state-of-the-art kernel-based clustering algorithm (SIMLR) using Pearson's correlation as a similarity measure and found significant performance improvement over Euclidean distance on scRNA-seq data clustering. These findings demonstrate the importance of similarity metrics in clustering scRNA-seq data and highlight Pearson's correlation as a favourable choice. Further comparison on different scRNA-seq library preparation protocols suggests that they may also affect clustering performance. Finally, the benchmarking framework is available at http://www.maths.usyd.edu.au/u/SMS/bioinformatics/software.html.

Gaussian kernel c-means hard clustering algorithms with automated computation of the width hyper-parameters

Conventional Gaussian kernel c-means clustering algorithms are widely used in applications. However, Gaussian kernel functions have an important parameter, the width hyper-parameter, which needs to be tuned. Usually this parameter is tuned once and for all and it is the same for all variables. Thus, implicitly, all the variables are equally rescaled and therefore, they have equal importance on the clustering task. This paper presents Gaussian kernel c-means hard clustering algorithms with automated computation of the width hyper-parameters. In these kernel-based clustering algorithms, the hyper-parameters change at each iteration of the algorithm, they differ from variable to variable and can differ from cluster to cluster. Because each variable is rescaled differently according to its own hyper-parameter, these algorithms can select the important variables in the clustering process. Experiments using synthetic data sets and using UCI machine learning repository data sets corroborate the usefulness of the proposed algorithms.

A new kernel-based possibilistic intuitionistic fuzzy c-means clustering

Fuzzy c-means and its derivatives such as possibilistic c-means and possibilistic fuzzy c-means are the most widely used clustering algorithms in the literature. Though efficient, these clustering algorithms do not achieve high cluster quality on real-world datasets, which are not linearly separable. Kernel-based clustering algorithms employ nonlinear similarity measures to define the inter-point similarities. As a result, they are able to identify clusters of arbitrary shapes and densities. Comparative analysis over standard datasets has established the superiority of kernel methods over its corresponding standard algorithms. In this paper, we propose a kernel-based Atanassov's possibilistic intuitionistic fuzzy clustering for data clustering and image segmentation. The paper explores the performance of the proposed methodology with respect to various internal and external indices for various real datasets and it is found to perform better than other clustering techniques in the sequel, i.e., normal as well as kernel-based algorithms. Experimental results on noisy image datasets also show the competence of the proposed approach.

Fuzzy clustering using multiple Gaussian kernels with optimized-parameters

In this paper, we propose a new kernel-based fuzzy clustering algorithm which tries to find the best clustering results using optimal parameters of each kernel in each cluster. It is known that data with nonlinear relationships can be separated using one of the kernel-based fuzzy clustering methods. Two common fuzzy clustering approaches are: clustering with a single kernel and clustering with multiple kernels. While clustering with a single kernel doesn’t work well with “multiple-density” clusters, multiple kernel-based fuzzy clustering tries to find an optimal linear weighted combination of kernels with initial fixed (not necessarily the best) parameters. Our algorithm is an extension of the single kernel-based fuzzy c-means and the multiple kernel-based fuzzy clustering algorithms. In this algorithm, there is no need to give “good” parameters of each kernel and no need to give an initial “good” number of kernels. Every cluster will be characterized by a Gaussian kernel with optimal parameters. In order to show its effective clustering performance, we have compared it to other similar clustering algorithms using different databases and different clustering validity measures.

A new kernel-based possibilistic intuitionistic fuzzy c-means clustering

Fuzzy c-means and its derivatives such as possibilistic c-means and possibilistic fuzzy c-means are the most widely used clustering algorithms in the literature. Though efficient, these clustering algorithms do not achieve high cluster quality on real-world datasets, which are not linearly separable. Kernel-based clustering algorithms employ nonlinear similarity measures to define the inter-point similarities. As a result, they are able to identify clusters of arbitrary shapes and densities. Comparative analysis over standard datasets has established the superiority of kernel methods over its corresponding standard algorithms. In this paper, we propose a kernel-based Atanassov's possibilistic intuitionistic fuzzy clustering for data clustering and image segmentation. The paper explores the performance of the proposed methodology with respect to various internal and external indices for various real datasets and it is found to perform better than other clustering techniques in the sequel, i.e., normal as well as kernel-based algorithms. Experimental results on noisy image datasets also show the competence of the proposed approach.

DOFL: Kernel Based Directive Operative Fractional Lion Optimisation Algorithm for Data Clustering

In recent years, clustering finds various applications in most of the fields like networking, telecommunications and medical domain. So, various clustering algorithms are developed by researchers to improve the clustering performance. Among this, optimisation based clustering algorithm is one of the recently developed algorithms for clustering process to discover the optimal clusters based on the objective function. Accordingly, in this paper, we have proposed directive operative fractional lion optimization using kernel based clustering algorithm. Initially, three objective functions are developed based on the kernel function. Then, the result of objective function is selected as the fitness function for the proposed optimisation algorithm called directive operative based fractional lion algorithm to select the rapid centroid of the data cluster. Finally, the performance of the proposed kernel based directive operative fractional lion algorithm (DOFL) algorithm is evaluated based on the various evaluation metrics such as cluster accuracy, jaccard coefficient and rand coefficient using the both benchmarked iris and wine data sets. From the research outcome, we can prove that, the maximum clustering accuracy of 79% is obtained by the proposed optimization clustering algorithm compared to other existing clustering algorithms such as Particle Swarm clustering algorithm (PSC), modified Particle Swarm clustering algorithm (mPSC), lion algorithm and fractional lion algorithm.

Distance metric learning for soft subspace clustering in composite kernel space

Soft subspace clustering algorithms have been successfully used for high dimensional data in recent years. However, the existing algorithms often utilize only one distance function to evaluate the distance between data items on each feature, which cannot deal with datasets with complex inner structures. In this paper, a composite kernel space (CKS) is constructed based on a set of basis kernels and a novel framework of soft subspace clustering is proposed by integrating distance metric learning in the CKS. Two soft subspace clustering algorithms, i.e., entropy weighting fuzzy clustering in CKS for kernel space (CKS-EWFC-K) and feature space (CKS-EWFC-F) are thus developed. In both algorithms, the prototype in the feature space is mapped into the CKS by multiple simultaneous mappings, one mapping for each cluster, which is distinct from existing kernel-based clustering algorithms. By evaluating the distance on each feature in the CKS, both CKS-EWFC-K and CKS-EWFC-F learn the distance function adaptively during the clustering process. Experimental results have demonstrated that the proposed algorithms in general outperform classical clustering algorithms and are immune to ineffective kernels and irrelevant features in soft subspace.

A Novel Graph-Matching-Based Approach for Domain Adaptation in Classification of Remote Sensing Image Pair

This paper addresses the problem of land-cover classification of remotely sensed image pairs in the context of domain adaptation. The primary assumption of the proposed method is that the training data are available only for one of the images (source domain), whereas for the other image (target domain), no labeled data are available. No assumption is made here on the number and the statistical properties of the land-cover classes that, in turn, may vary from one domain to the other. The only constraint is that at least one land-cover class is shared by the two domains. Under these assumptions, a novel graph theoretic cross-domain cluster mapping algorithm is proposed to detect efficiently the set of land-cover classes which are common to both domains as well as the additional or missing classes in the target domain image. An interdomain graph is introduced, which contains all of the class information of both images, and subsequently, an efficient subgraph-matching algorithm is proposed to highlight the changes between them. The proposed cluster mapping algorithm initially clusters the target domain data into an optimal number of groups given the available source domain training samples. To this end, a method based on information theory and a kernel-based clustering algorithm is proposed. Considering the fact that the spectral signature of land-cover classes may overlap significantly, a postprocessing step is applied to refine the classification map produced by the clustering algorithm. Two multispectral data sets with medium and very high geometrical resolution and one hyperspectral data set are considered to evaluate the robustness of the proposed technique. Two of the data sets consist of multitemporal image pairs, while the remaining one contains images of spatially disjoint geographical areas. The experiments confirm the effectiveness of the proposed framework in different complex scenarios.

Dynamic Scene Understanding for Behavior Analysis Based on String Kernels

This paper aims at dynamically understanding the properties of a scene from the analysis of moving object trajectories. Two different applications are proposed: the former is devoted to identify abnormal behaviors, while the latter allows to extract the k, most of the similar trajectories to the one hand-drawn by an human operator. A set of normal trajectories' models is extracted using a novel unsupervised learning technique: the scene is adaptively partitioned into zones using the distribution of the training set and each trajectory is represented as a sequence of symbols by considering positional information (the zones crossed in the scene), speed, and shape. The main novelty is the use of a kernel-based approach for evaluating the similarity between the trajectories. Furthermore, we define a novel and efficient kernel-based clustering algorithm, aimed at obtaining groups of normal trajectories. Experimentations, conducted over three standard data sets, confirm the effectiveness of the proposed approach.

Fault Diagnosis of Metro Vehicle Auxiliary Inverter Based on PSO-KPCM Algorithm

In this paper, the principle of kernel-based possibilistic clustering algorithm and particle swarm optimization algorithm are introduced and the application of the algorithms in fault diagnosis of auxiliary inverter is studied. Several common fault types are simulated by MATALB software. By initializing clustering center of samples based on PSO algorithm, calculating the final membership matrix and the final clustering center matrix based on the KPCM algorithm, the fault samples can be classified finally. The simulation results show that the PSO-KPCM algorithm can be used in the field of fault diagnosis. The PSO-KPCM algorithm even has better results and faster convergence rate than FCM algorithm.

Selective sampling-based training schemes for core vector machines

Support vector machines (SVMs) are hyperplane classifiers defined in a kernel induced feature space. The high computational and space requirements for solving the conventional SVM problem prohibit its use in applications involving large datasets. Core vector machine (CVM) is a suitable technique for scaling an SVM for large-scale pattern classification problems. But in applications where the datasets are unbalanced, the performance of CVM is observed to be poor both in terms of generalisation and training time. In such scenarios, the CVM performance highly depends on the orderings of data points belonging to the two classes within the dataset. In this paper, we propose two training schemes which improve the performance of CVM irrespective of the orderings of patterns belonging to different classes within the dataset. These methods employ a selective sampling-based training of CVM using novel kernel-based clustering algorithms. Empirical studies made on several synthetic and real world datasets show that the proposed strategies improve the performance of CVM on large datasets.

Rule extraction algorithm from support vector machines and its application to credit screening

Developing rule extraction algorithms from machine learning techniques such as artificial neural networks and support vector machines (SVMs), which are considered incomprehensible black-box models, is an important topic in current research. This study proposes a rule extraction algorithm from SVMs that uses a kernel-based clustering algorithm to integrate all support vectors and genetic algorithms into extracted rule sets. This study uses measurements of accuracy, sensitivity, specificity, coverage, fidelity and comprehensibility to evaluate the performance of the proposed method on the public credit screening data sets. Results indicate that the proposed method performs better than other rule extraction algorithms. Thus, the proposed algorithm is an essential analysis tool that can be effectively used in data mining fields.

Multi-object detection and tracking by stereo vision

This paper presents a new stereo vision-based model for multi-object detection and tracking in surveillance systems. Unlike most existing monocular camera-based systems, a stereo vision system is constructed in our model to overcome the problems of illumination variation, shadow interference, and object occlusion. In each frame, a sparse set of feature points are identified in the camera coordinate system, and then projected to the 2D ground plane. A kernel-based clustering algorithm is proposed to group the projected points according to their height values and locations on the plane. By producing clusters, the number, position, and orientation of objects in the surveillance scene can be determined for online multi-object detection and tracking. Experiments on both indoor and outdoor applications with complex scenes show the advantages of the proposed system.