Kernel Fuzzy C-means Research Articles

Training Data Reduction and Classification Based on Greedy Kernel Principal Component Analysis and Fuzzy C-Means Algorithm

Nonlinear feature extraction used standard Kernel Principal Component Analysis (KPCA) method has large memories and high computational complexity in large datasets. A Greedy Kernel Principal Component Analysis (GKPCA) method is applied to reduce training data and deal with the nonlinear feature extraction problem for training data of large data in classification. First, a subset, which approximates to the original training data, is selected from the full training data using the greedy technique of the GKPCA method. Then, the feature extraction model is trained by the subset instead of the full training data. Finally, FCM algorithm classifies feature extraction data of the GKPCA, KPCA and PCA methods, respectively. The simulation results indicate that the feature extraction performance of both the GKPCA, and KPCA methods outperform the PCA method. In addition of retaining the performance of the KPCA method, the GKPCA method reduces computational complexity due to the reduced training set in classification.

Relational Fuzzy c-Means and Kernel Fuzzy c-Means Using an Object-Wise β-Spread Transformation

Clustering methods of relational data are often based on the assumption that a given set of relational data is Euclidean, and kernelized clustering methods are often based on the assumption that a given kernel is positive semidefinite. In practice, non-Euclidean relational data and an indefinite kernel may arise, and a β-spread transformation was proposed for such cases, which modified a given set of relational data or a give a kernel Gram matrix such that the modified β value is common to all objects. In this paper, we propose an object-wise β-spread transformation for use in both relational and kernelized fuzzy c-means clustering. The proposed system retains the given data better than conventional methods, and numerical examples show that our method is efficient for both relational and kernel fuzzy c-means.

A Segmentation Method for Multiple Sclerosis White Matter Lesions on Conventional Magnetic Resonance Imaging Based on Kernel Fuzzy Clustering

Multiple sclerosis (MS) is a chronic disease that affects the central nervous system and impacts substantially on patients. MS lesions are visible in conventional magnetic resonance imaging (cMRI) and the automatic segmentation of MS lesions enables the efficient processing of images for research studies and in clinical trials. A new method for the segmentation of MS white matter lesions (WML) on cMRI is presented in this paper. Firstly the Kernel Fuzzy C-Means Clustering (KFCM) is applied to the preprocessed T1-weight (T1-w) image for extracting the white matter (WM) region. Then region growing algorithm is applied to the WM region image to make a binary mask which is then superimposed on the corresponding T2-weight (T2-w) image to yield a masked image only containing WM structures and lesions. The KFCM is then reapplied to the masked image to obtain MS lesions. The testing results show that the proposed method is able to segment WML on cMRI automatically and effectively.

An Improved Faults Diagnosis Strategy of Analog Circuit Online Using Kernel Fuzzy C-Means for the Unknown Fault

To solve the unknown faults diagnosis online for analog circuit, a novel faults diagnosis strategy based on improved kernel fuzzy c-means (IKFCM) is proposed. simultaneously, the high-performance recognition tree structure of the improved kernel fuzzy c-means itself can decrease the train sample and eliminate wild value, the training speed and precision of classifier can be done well in this way. The realizing of precision fault diagnosis, firstly, via confirming exact class centers from the data of known faults, and then the mean value can be obtained relying on the faults data of each class, meanwhile, setting this mean values as the thresholds for judging faults and each data point is issued with a class label. During the whole faults diagnosis, each detection data will be compared with the thresholds, the high similarity detection data will be fall into the known faults classes while the low similarity detection data will be labeled as unknown faults. Simulation proved the well performance effectively of the proposed IKFCM method .

Kernel fuzzy c-means with automatic variable weighting

This paper presents variable-wise kernel fuzzy c-means clustering methods in which dissimilarity measures are obtained as sums of Euclidean distances between patterns and centroids computed individually for each variable by means of kernel functions. The advantage of the proposed approach over the conventional kernel clustering methods is that it allows us to use adaptive distances which change at each algorithm iteration and can either be the same for all clusters or different from one cluster to another. This kind of dissimilarity measure is suitable to learn the weights of the variables during the clustering process, improving the performance of the algorithms. Another advantage of this approach is that it allows the introduction of various fuzzy partition and cluster interpretation tools. Experiments with synthetic and benchmark datasets show the usefulness of the proposed algorithms and the merit of the fuzzy partition and cluster interpretation tools.

Efficient Hardware Architecture for Kernel Fuzzy C-Means Algorithm

A novel VLSI architecture for kernel fuzzy c-means algorithm is presented in this paper. The architecture consists of efficient circuits for the computation of kernel functions, membership coefficients and cluster centers. In addition, the usual iterative operations for updating the membership matrix and cluster centers are merged into one single updating process to evade the large storage requirement. The circuit is used as a hardware accelerator of a softcore processor in a system-on-programmable chip for physical performance measurement. Experimental results show that the proposed solution is an effective alternative for cluster analysis with low computational cost and high performance.

Semi-supervised clustering of large data sets with kernel methods

Labelling real world data sets is a difficult problem. Often, the human expert is unsure about a class label of a specific sample point or, in case of very large data sets, it is impractical to label them manually. In semi-supervised clustering, the sample labels, which are external informations, are used to find better matching cluster partitions. Further, kernel-based clustering methods are able to partition the data with nonlinear boundaries in feature space. While these methods improve the clustering results, they have a quadratic computation time. In this paper, we propose a meta-algorithm that processes small-sized subsets of a large data set, clusters them with the sample labels and merges the points close to the resulting prototypes with the next points, until the whole data set has been processed. It has a linear computation time. The error function that this meta-algorithm minimizes is presented. Although we applied this meta-algorithm to Kernel Fuzzy C-Means, Relational Neural Gas and Kernel K-Means, it can be applied to a broad range of kernel-based clustering methods. The proposed method has been empirically evaluated on two real world benchmark data sets.

Unsupervised classification of polarimetric synthetic aperture radar images using kernel fuzzy C-means clustering

In this article, an unsupervised clustering approach, known as the kernel fuzzy C-means (KFCM), which is based on Cloude's decomposition, is used for polarimetric synthetic aperture radar (POLSAR) image classification. The KFCM algorithm is an improved version of the fuzzy C-means (FCM) algorithm. It replaces the original Euclidean distance measure by the kernel-induced distance. The original sample space, coherency matrix T, is mapped to a higher-dimensional feature space so as to simplify the complex POLSAR data by using the Gaussian kernel function. The method has three main steps: first, the eight initial centres are obtained by averaging the coherency matrix within each partition according to the classical H/α plane so as to preserve the polarimetric property reasonably well; second, the related parameters of the KFCM algorithm are iteratively refined and third, the membership matrix is defuzzified by using the maximum membership decision rule. The distance measure used in the algorithm is derived from the complex Wishart distribution of the pixel data presented in the coherence data. The KFCM method not only takes advantage of the polarimetric scattering properties but also utilises the kernel method to cluster the nonlinear structure data with noise. The feasibility of this approach was tested by using two JPL/AIRSAR polarimetric SAR images. The experimental results showed that the KFCM clustering algorithm was better than the FCM clustering algorithm at classifying the POLSAR images.

Power transformer fault diagnosis under measurement originated uncertainties

This paper addresses the problem of diagnosing the fault symptoms of power transformers with measurement originated uncertainties, which arise from the imprecision of samples (i.e. due to noises and outliers) and the effect of class imbalance (i.e. samples are unequally distributed between different fault types) in a training dataset used to identify different fault types. Two fuzzy support vector machine (FSVM) algorithms namely fuzzy c-means clustering-based FSVM (FCM-FSVM) and kernel fuzzy c-means clustering-based FSVM (KFCM-FSVM) have been applied in this paper to deal with any noises and outliers in training dataset. In order to reduce the effect of class imbalance in training dataset, two approaches including between-class weighting and random oversampling have been adopted and integrated with FCM-FSVM and KFCM-FSVM. The case studies show that KFCM-FSVM algorithm and its variants have consistent tendency to attain satisfied classification accuracy in transformer fault diagnosis using dissolved gas analysis (DGA) measurements.

Fuzzy c-Means Algorithms for Very Large Data

Very large (VL) data or big data are any data that you cannot load into your computer's working memory. This is not an objective definition, but a definition that is easy to understand and one that is practical, because there is a dataset too big for any computer you might use; hence, this is VL data for you. Clustering is one of the primary tasks used in the pattern recognition and data mining communities to search VL databases (including VL images) in various applications, and so, clustering algorithms that scale well to VL data are important and useful. This paper compares the efficacy of three different implementations of techniques aimed to extend fuzzy c-means (FCM) clustering to VL data. Specifically, we compare methods that are based on 1) sampling followed by noniterative extension; 2) incremental techniques that make one sequential pass through subsets of the data; and 3) kernelized versions of FCM that provide approximations based on sampling, including three proposed algorithms. We use both loadable and VL datasets to conduct the numerical experiments that facilitate comparisons based on time and space complexity, speed, quality of approximations to batch FCM (for loadable data), and assessment of matches between partitions and ground truth. Empirical results show that random sampling plus extension FCM, bit-reduced FCM, and approximate kernel FCM are good choices to approximate FCM for VL data. We conclude by demonstrating the VL algorithms on a dataset with 5 billion objects and presenting a set of recommendations regarding the use of different VL FCM clustering schemes.

MRI Brain Image Segmentation based on Wavelet and FCM Algorithm

Image segmentation plays a preliminary and indispensable step in medical image processing. Magnetic resonance (MR) segmentation used for brain tissues extraction white matter (WM), gray matter (GM) and cerebrospinal fluids (CSF). These tissues help in many medical image segmentation applications such as radiotherapy planning, clinical diagnosis, treatment planning and Alzheimer disease. This paper presents a novel manipulation or utilization of Fuzzy CMeans (FCM) Clustering by using wavelet Decomposition for feature extraction and feature vector treat as input to FCM. This algorithm is called Wavelet Fuzzy Cmeans (WFCM), the algorithm results are compared with standard FCM and Kernelized Fuzzy CMeans (KFCM). The performance of the proposed segmentation algorithm provides satisfactory results compared with the other two algorithms. General Terms Pattern Recognition, image segmentation.

Review and Comparison of Kernel Based Fuzzy Image Segmentation Techniques

This paper presents a detailed study and comparison of some Kernelized Fuzzy C-means Clustering based image segmentation algorithms Four algorithms have been used Fuzzy Clustering, Fuzzy CMeans(FCM) algorithm, Kernel Fuzzy CMeans(KFCM), Intuitionistic Kernelized Fuzzy CMeans(KIFCM), Kernelized Type-II Fuzzy CMeans(KT2FCM).The four algorithms are studied and analyzed both quantitatively and qualitatively. These algorithms are implemented on synthetic images in case of without noise along with Gaussian and salt and pepper noise for better review and comparison. Based on outputs best algorithm is suggested. Index Terms —Fuzzy Clustering, Fuzzy CMeans(FCM) algorithm, Kernel Fuzzy CMeans(KFCM),Intuitionistic Kernelized Fuzzy CMeans(KIFCM) ,Kernelized Type-II Fuzzy CMeans(KT2FCM),kernel width.

Determining the number of clusters for kernelized fuzzy C-means algorithms for automatic medical image segmentation

In this paper, we determine the suitable validity criterion of kernelized fuzzy C-means and kernelized fuzzy C-means with spatial constraints for automatic segmentation of magnetic resonance imaging (MRI). For that; the original Euclidean distance in the FCM is replaced by a Gaussian radial basis function classifier (GRBF) and the corresponding algorithms of FCM methods are derived. The derived algorithms are called as the kernelized fuzzy C-means (KFCM) and kernelized fuzzy C-means with spatial constraints (SKFCM). These methods are implemented on eighteen indexes as validation to determine whether indexes are capable to acquire the optimal clusters number. The performance of segmentation is estimated by applying these methods independently on several datasets to prove which method can give good results and with which indexes. Our test spans various indexes covering the classical and the rather more recent indexes that have enjoyed noticeable success in that field. These indexes are evaluated and compared by applying them on various test images, including synthetic images corrupted with noise of varying levels, and simulated volumetric MRI datasets. Comparative analysis is also presented to show whether the validity index indicates the optimal clustering for our datasets.

Multiple Kernel Fuzzy Clustering

While fuzzy c-means is a popular soft-clustering method, its effectiveness is largely limited to spherical clusters. By applying kernel tricks, the kernel fuzzy c-means algorithm attempts to address this problem by mapping data with nonlinear relationships to appropriate feature spaces. Kernel combination, or selection, is crucial for effective kernel clustering. Unfortunately, for most applications, it is uneasy to find the right combination. We propose a multiple kernel fuzzy c-means (MKFC) algorithm that extends the fuzzy c-means algorithm with a multiple kernel-learning setting. By incorporating multiple kernels and automatically adjusting the kernel weights, MKFC is more immune to ineffective kernels and irrelevant features. This makes the choice of kernels less crucial. In addition, we show multiple kernel k-means to be a special case of MKFC. Experiments on both synthetic and real-world data demonstrate the effectiveness of the proposed MKFC algorithm.

On Kernel Fuzzy c-Means for Data with Tolerance Using Explicit Mapping for Kernel Data Analysis

While explicit mapping is generally unknown for kernel data analysis, its inner product should be known. Although we proposed a kernel fuzzy c-means algorithm for data with tolerance, cluster centers and tolerance in higher dimensional space have not been seen. Contrary to this common assumption, explicit mapping has been introduced and the situation of kernel fuzzy c-means in higher dimensional space has been described via kernel principal component analysis using explicit mapping. In this paper, cluster centers and the tolerance of kernel fuzzy c-means for data with tolerance are described via kernel principal component analysis using explicit mapping.

Gaussian Kernelized Fuzzy c-means with Spatial Information Algorithm for Image Segmentation

FCM is used for image segmentation in some applications. It is based on a specific distance norm and does not use spatial information of the image, so it has some drawbacks. Various kinds of improvements have been developed to extend the adaptability, such as BFCM, SFCM and KFCM. These methods extend FCM from two aspects, one is replacing the Euclidean norm, and the other is considering the spatial information constraints for clustering. Kernel distance can improve the robustness for multi-distribution data sets. Spatial information can help eliminate the sensitivity to noises and outliers. In this paper, Gaussian kernel-based fuzzy c-means algorithm with spatial information (KSFCM) is proposed. KSFCM is more robust and adaptive. The experiment results showd that KSFCM has the better performance.

Fuzzy Clustering Algorithms for Image Segmentation

In this document we present the implementation of three fuzzy clustering algorithms using the Insight Toolkit ITK. Firstly, we developed the conventional Fuzzy C-Means that will serve as the basis for the rest of the proposed algorithms. The next algorithms are the FCM with spatial constraints based on kernel-induced distance and the Modified Spatial Kernelized Fuzzy C-Means. Both of these introduce a Kernel function, replacing the Euclidean distance of the FCM, and spatial information into the membership function.These algorithms have been implemented in a threaded version to take advantage of the multicore processors. Moreover, providing an useful implementation make it possible that classes work with 2D/3D images, different kernels and spatial shapes.We included the source code as well as different 2D/3D examples, using several input parameters for the algorithms and obtaining the results generated on 2D/3D CT lung studies.

A novel KFCM based fault diagnosis method for unknown faults in satellite reaction wheels

Reaction wheels are one of the most critical components of the satellite attitude control system, therefore correct diagnosis of their faults is quintessential for efficient operation of these spacecraft. The known faults in any of the subsystems are often diagnosed by supervised learning algorithms, however, this method fails to work correctly when a new or unknown fault occurs. In such cases an unsupervised learning algorithm becomes essential for obtaining the correct diagnosis. Kernel Fuzzy C-Means (KFCM) is one of the unsupervised algorithms, although it has its own limitations; however in this paper a novel method has been proposed for conditioning of KFCM method (C-KFCM) so that it can be effectively used for fault diagnosis of both known and unknown faults as in satellite reaction wheels. The C-KFCM approach involves determination of exact class centers from the data of known faults, in this way discrete number of fault classes are determined at the start. Similarity parameters are derived and determined for each of the fault data point. Thereafter depending on the similarity threshold each data point is issued with a class label. The high similarity points fall into one of the ‘known-fault’ classes while the low similarity points are labeled as ‘unknown-faults’. Simulation results show that as compared to the supervised algorithm such as neural network, the C-KFCM method can effectively cluster historical fault data (as in reaction wheels) and diagnose the faults to an accuracy of more than 91%.

Audio watermarking scheme robust against desynchronization attacks based on kernel clustering

In this paper, we propose an adaptive audio watermarking scheme based on kernel fuzzy c-means (KFCM) clustering algorithm, which possesses robust ability against common signal processing and desynchronization attacks. The original audio signal is partitioned into audio frames and then each audio frame is further divided as two sub-frames. In order to resist desynchronization attacks, we embed a synchronization code into first sub-frame of each audio frame by using a mean quantization technique in temporal domain. Moreover, watermark signal is hid into DWT coefficients of second sub-frame of each audio frame by using an energy quantization technique. A local audio feature data set extracted from all audio frames is used to train a KFCM. The well-trained KFCM is used to adaptively control quantization steps in above two quantization techniques. The experimental results show the proposed scheme is robust to common signal processing (such as MP3 lossy compression, noise addition, filtering, re-sampling, re-quantizing) and desynchronization attacks (random cropping, pitch shifting, amplitude variation, time-scale modification, jittering).

Smoothing approach to alleviate the meager rating problem in collaborative recommender systems

In this paper, an integrated recommendation approach using Radial Basis Function Network (RBFN) and Collaborative Filtering (CF) is proposed. Radial basis function network is a neural network approximation method used to improve the accuracy of the recommendations. The proposed system RBFN_KFCM has offline and online phases. During offline, the system uses RBFN for smoothing and kernel fuzzy c-means (KFCM) method for clustering. During online recommendation, KFCM based approach is proposed. At the end of each session the clusters are updated by replacing the smoothed rating by the original rating. A comparison is made with benchmark recommender systems such as item-based, user-based systems, singular value decomposition (SVD) and also popular machine learning techniques such as Support Vector Machine (SVM), Multilayer Perceptron (MLP) using backpropagation algorithm, in terms of accuracy, decision-support measures and computational time. Empirical evaluation is carried out by using movielens dataset.