Gustafson-Kessel Clustering Research Articles

A Parallel Implementation of the Gustafson-Kessel Clustering Algorithm with CUDA

Despite the benefits of the Gustafson-Kessel (GK) clustering algorithm, it becomes computationally inefficient when applied to high-dimensional data. In this letter, a parallel implementation of the GK algorithm on the GPU with CUDA is proposed. Using an optimized matrix multiplication algorithm with fast access to shared memory, the CUDA version achieved a maximum 240-fold speedup over the single-CPU version.

Recursive clustering based on a Gustafson–Kessel algorithm

In this paper an on-line fuzzy identification of Takagi Sugeno fuzzy model is presented. The presented method combines a recursive Gustafson–Kessel clustering algorithm and the fuzzy recursive least squares method. The on-line Gustafson–Kessel clustering method is derived. The recursive equations for fuzzy covariance matrix, its inverse and cluster centers are given. The use of the method is presented on two examples. First example demonstrates the use of the method for monitoring of the waste water treatment process and in the second example the method is used to develop an adaptive fuzzy predictive functional controller for a pH process. The results for the Mackey–Glass time series prediction are also given.

Identifying the Mastery Concepts in Linear Algebra by Using FCM-CM Algorithm

Euclidean distance function based fuzzy clustering algorithms can only be used to detect spherical structural clusters. Gustafson-Kessel (GK) clustering algorithm and Gath-Geva (GG) clustering algorithm were developed to detect non-spherical structural clusters by employing Mahalanobis distance in objective function, however, both of them need to add some constrains for Mahalanobis distance. In this paper, the authors’ improved Fuzzy C-Means algorithm based on common Mahalanobis distance (FCM-CM) is used to identify the mastery concepts in linear algebra, for comparing the performances with other four partition algorithms; FCM-M, GG, GK, and FCM. The result shows that FCM-CM has better performance than others.

Fuzzy clustering algorithms for unsupervised change detection in remote sensing images

In this paper, we propose a context-sensitive technique for unsupervised change detection in multitemporal remote sensing images. The technique is based on fuzzy clustering approach and takes care of spatial correlation between neighboring pixels of the difference image produced by comparing two images acquired on the same geographical area at different times. Since the ranges of pixel values of the difference image belonging to the two clusters ( changed and unchanged) generally have overlap, fuzzy clustering techniques seem to be an appropriate and realistic choice to identify them (as we already know from pattern recognition literatures that fuzzy set can handle this type of situation very well). Two fuzzy clustering algorithms, namely fuzzy c-means (FCM) and Gustafson–Kessel clustering (GKC) algorithms have been used for this task in the proposed work. For clustering purpose various image features are extracted using the neighborhood information of pixels. Hybridization of FCM and GKC with two other optimization techniques, genetic algorithm (GA) and simulated annealing (SA), is made to further enhance the performance. To show the effectiveness of the proposed technique, experiments are conducted on two multispectral and multitemporal remote sensing images. A fuzzy cluster validity index (Xie–Beni) is used to quantitatively evaluate the performance. Results are compared with those of existing Markov random field (MRF) and neural network based algorithms and found to be superior. The proposed technique is less time consuming and unlike MRF does not require any a priori knowledge of distributions of changed and unchanged pixels.

Parameter estimation of K-distributed sea clutter based on fuzzy inference and Gustafson–Kessel clustering

The detection performance of maritime radars is restricted by the unwanted sea echo or clutter. Although the number of these target-like data is small, they may cause false alarm and perturb the target detection. K-distribution is known as the best fit probability density function for the radar sea clutter. This paper proposes a novel approach to estimate the parameters of K-distribution, based on fuzzy Gustafson–Kessel clustering and fuzzy Takagi–Sugeno Kang modelling. The main contribution of the proposed method is the ability to estimate the parameters, given a small number of data which will usually be the case in practical applications. This is achieved by a pre-estimation using fuzzy clustering that provides a prior knowledge and forms a rough model to be fine tuned using the least square method. The algorithm also improves the calculations of shape and width of membership functions by means of clustering in order to improve the accuracy. The resultant estimator then acts to overcome the bottleneck of the existing methods in which it achieves a higher performance and accuracy in spite of small number of data.

Desulfurization process using Takagi–Sugeno–Kang fuzzy modeling

Two Takagi–Sugeno–Kang fuzzy models for the prediction of the amount of reagents for desulfurization in steel processing are developed from experimental data. For the design of the models, an algorithm was proposed to be used in the procedures of the two phases: structure building and parametric identification. In the first phase, the Gustafson–Kessel clustering algorithm with the cluster validity index was proposed to find the number of fuzzy rules and an initial fuzzy model. In the second phase, a gradient-descent-based approach was developed and used for optimized tuning of membership functions of the fuzzy model. The numerical results were compared with a conventional statistical model and neural networks and adaptive network-based fuzzy inference system.

Automatic classification of underwater targets using fuzzy-cluster-based wavelet signatures.

A novel active acoustic underwater target recognition approach is applied to simulated mine-like targets. A feature extraction and integration method is introduced based on the continuous wavelet transform (CWT) and the Gustafson-Kessel (GK) clustering algorithm. CWT projects the acoustic signals onto the time-scale space, and the time-scale signatures are represented by a number of clusters. A modified GK clustering algorithm allows for specific enhancements: (i) autoadjustment of the shape of the clusters according to the time-scale patterns and (ii) autovalidation of the optimum number of clusters. The feature integration encompasses the application of the clustering analysis N times. The resulting cluster centers are combined and overlaps are eliminated. Using this method, the target of interest is characterized by an integrated clustering distribution, or a feature template. A classification scheme is developed by matching the clustering distributions of the echoes received from an unknown target with the existing feature templates. White Gaussian noise has been added to the signals to simulate the interface of an ocean floor. The recognition rate varies from 72–98% for a SNR between 5–15 dB.

Fuzzy Clustering for Image Segmentation Using Generic Shape Information

The performance of clustering algorithms for image segmentation are highly sensitive to the features used and types of objects in the image, which ultimately limits their generalization capability. This provides strong motivation to investigate integrating shape information into the clustering framework to improve the generality of these algorithms. Existing shape-based clustering techniques mainly focus on circular and elliptical clusters and so are unable to segment arbitrarily-shaped objects. To address this limitation, this paper presents a new shape-based algorithm called fuzzy clustering for image segmentation using generic shape information (FCGS), which exploits the B-spline representation of an object’s shape in combination with the Gustafson-Kessel clustering algorithm. Qualitative and quantitative results for FCGS confirm its superior segmentation performance consistently compared to well-established shape-based clustering techniques, for a wide range of test images comprising various regular and arbitrary-shaped objects.

A semi-supervised method to detect seismic random noise with fuzzy GK clustering

We present a new method to detect random noise in seismic data using fuzzy Gustafson–Kessel (GK) clustering. First, using an adaptive distance norm, a matrix is constructed from the observed seismic amplitudes. The next step is to find centres of ellipsoidal clusters and construct a partition matrix which determines the soft decision boundaries between seismic events and random noise. The GK algorithm updates the cluster centres in order to iteratively minimize the cluster variance. Multiplication of the fuzzy membership function with values of each sample yields new sections; we name them ‘clustered sections’. The seismic amplitude values of the clustered sections are given in a way to decrease the level of noise in the original noisy seismic input. In pre-stack data, it is essential to study the clustered sections in a f–k domain; finding the quantitative index for weighting the post-stack data needs a similar approach. Using the knowledge of a human specialist together with the fuzzy unsupervised clustering, the method is a semi-supervised random noise detection. The efficiency of this method is investigated on synthetic and real seismic data for both pre- and post-stack data. The results show a significant improvement of the input noisy sections without harming the important amplitude and phase information of the original data. The procedure for finding the final weights of each clustered section should be carefully done in order to keep almost all the evident seismic amplitudes in the output section. The method interactively uses the knowledge of the seismic specialist in detecting the noise.

Data mining in forecasting PVT correlations of crude oil systems based on Type1 fuzzy logic inference systems

Pressure–volume–temperature properties are very important in the reservoir engineering computations. There are many empirical approaches for predicting various PVT properties based on empirical correlations and statistical regression models. Last decade, researchers utilized neural networks to develop more accurate PVT correlations. These achievements of neural networks open the door to data mining techniques to play a major role in oil and gas industry. Unfortunately, the developed neural networks correlations are often limited, and global correlations are usually less accurate compared to local correlations. Recently, adaptive neuro-fuzzy inference systems have been proposed as a new intelligence framework for both prediction and classification based on fuzzy clustering optimization criterion and ranking. This paper proposes neuro-fuzzy inference systems for estimating PVT properties of crude oil systems. This new framework is an efficient hybrid intelligence machine learning scheme for modeling the kind of uncertainty associated with vagueness and imprecision. We briefly describe the learning steps and the use of the Takagi Sugeno and Kang model and Gustafson–Kessel clustering algorithm with K-detected clusters from the given database. It has featured in a wide range of medical, power control system, and business journals, often with promising results. A comparative study will be carried out to compare their performance of this new framework with the most popular modeling techniques, such as neural networks, nonlinear regression, and the empirical correlations algorithms. The results show that the performance of neuro-fuzzy systems is accurate, reliable, and outperform most of the existing forecasting techniques. Future work can be achieved by using neuro-fuzzy systems for clustering the 3D seismic data, identification of lithofacies types, and other reservoir characterization.

Unsupervised classification of hyperspectral-image data using fuzzy approaches that spatially exploit membership relations

This letter presents unsupervised hyperspectral-image classification based on fuzzy-clustering algorithms that spatially exploit membership relations. Not only is the conventional fuzzy c-means approach used to demonstrate the advantage of using membership relations but also Gustafson-Kessel clustering, which uses an adaptive distance norm, is, for the first time, used for the segmentation of hyperspectral images. A novel approach to include spatial information in the segmentation process is achieved by making use of spatial relations of fuzzy-membership functions among neighbor pixels. Two- and three-dimensional Gaussian filtering of fuzzy-membership degrees is utilized for this purpose. A novel phase-correlation-based similarity measure is used to further enhance the performance of the proposed approach by taking spatial relations into account for pixels with similar spectral characteristics only. It is shown that the proposed approach provides superior clustering performance for hyperspectral images.

Fuzzy logic application to drying kinetics modelling

A Fick's model that includes a Takagi-Sugeno fuzzy model to estimate the effective diffusivity was analyzed. The modelling of drying kinetics on mango trough this diffusional-fuzzy model was compared with the theoretical Fick's model and the empirical Peleg and Weibull models. The identification and validation was performed from experimental drying curves of ripe mango slices (Mangifera indica L. cv. Tommy Atkins) at constant air velocity (4 m/s) and different drying temperatures (40, 50, 60 and 70 °C). The fuzzy sets for the antecedent of Takagi-Sugeno system were identified by the Gustafson-Kessel clustering algorithm and approximated by membership functions of piecewise exponential form. On identification and validation, the diffusional-fuzzy model showed best results than the Fick's model, whereas it showed little difference with the Weibull and Peleg models. The diffusional-fuzzy model keeps the interpretability of Fick's model, improves the process simulation and avoids phenomenon and property considerations which require additional experimental and modelling work.

Individual leaf extractions from young canopy images using Gustafson–Kessel clustering and a genetic algorithm

The extraction of individual concealed leaves from images of complex plant canopies is a necessary step for taxonomic feature acquisition, species identification, and mapping using a modern personal computer. A new system for individual leaflet extraction was developed and tested, based on connected components, fuzzy clustering and a genetic optimization algorithm. Color images were taken of young, but sparse green canopies, grown in both greenhouse and field conditions. Some images contained individual leaves as connected components, which were readily apparent after separation of the vegetation from its background. Fragments of all other leaves imbedded in the canopy were obtained using the Gustafson–Kessel (GK) clustering algorithm. Each leaf fragment was labeled and placed in a variable length data structure called a chromosome, which represented selected leaf fragments and its neighbors. A genetic algorithm was then used to systematically reassemble the fragments of non-occluded, individual leaves. System performance was evaluated by comparing the number of individual leaves extracted by the computer per plant or plant canopy connected component for various soil/residue backgrounds and time after emergence. 83.5% of the plants in the second week produced at least one computer-extracted leaf for identification. Ninty-two percent of the plants had at least one computer extracted leaf by the third week. 84.7% had more than three computer extracted leaves for identification in the third week. Images of young field plants in multiple species clusters resulted in a 46% leaf extraction rate, but with at least one leaf per connected canopy component. Soybean and velvetleaf leaflets were the easiest to extract. Once individual leaves are extracted, they can be classified using traditional shape and textural feature methods. Computerized individual leaf extraction could assist plant identification and mapping, needed for weed control and crop management.

Detecting clusters of different geometrical shapes in microarray gene expression data

Clustering has been used as a popular technique for finding groups of genes that show similar expression patterns under multiple experimental conditions. Many clustering methods have been proposed for clustering gene-expression data, including the hierarchical clustering, k-means clustering and self-organizing map (SOM). However, the conventional methods are limited to identify different shapes of clusters because they use a fixed distance norm when calculating the distance between genes. The fixed distance norm imposes a fixed geometrical shape on the clusters regardless of the actual data distribution. Thus, different distance norms are required for handling the different shapes of clusters. We present the Gustafson-Kessel (GK) clustering method for microarray gene-expression data. To detect clusters of different shapes in a dataset, we use an adaptive distance norm that is calculated by a fuzzy covariance matrix (F) of each cluster in which the eigenstructure of F is used as an indicator of the shape of the cluster. Moreover, the GK method is less prone to falling into local minima than the k-means and SOM because it makes decisions through the use of membership degrees of a gene to clusters. The algorithmic procedure is accomplished by the alternating optimization technique, which iteratively improves a sequence of sets of clusters until no further improvement is possible. To test the performance of the GK method, we applied the GK method and well-known conventional methods to three recently published yeast datasets, and compared the performance of each method using the Saccharomyces Genome Database annotations. The clustering results of the GK method are more significantly relevant to the biological annotations than those of the other methods, demonstrating its effectiveness and potential for clustering gene-expression data. The software was developed using Java language, and can be executed on the platforms that JVM (Java Virtual Machine) is running. It is available from the authors upon request. Supplementary data are available at http://dragon.kaist.ac.kr/gk.

An Approach to Building Rough Data Model Through Supervised Fuzzy Clustering

A new method for fast building the rough data model (RDM) by means of supervised fuzzy clustering in the product space of input and output variables is proposed. The approach incorporates the RDM’s classification quality performance index with Gustafson-Kessel (GK) clustering algorithm and is of many good properties. The way to convert the fuzzy cluster models to rough data models by introducing the concept of putative membership degree of a data point to a fuzzy cluster is suggested. Hence, an efficient algorithm that can obtain RDMs by just iteratively computing two necessary condition equations is worked out. It minimizes the objective function and turns the multi-dimensional search process of the Kowalczyk’s method to one dimensional search strategy (in terms of the number of clusters). This technique reduces the searching time greatly. Compared with the traditional rough set theory and the Kowalczyk’s method, the approach has more powerful ability to handle data contaminated by noise and better generalization ability. Finally, different examples of data sets illustrate the effectiveness of the approach.

Extended fuzzy GK clustering with application to identification ofan automatic voltage regulation loop dynamics

A new strategy for non linear identification which combines an extended fuzzy Gustafson-Kessel (GK) clustering technique using competitive agglomeration with locally weighted least-squares is proposed. This is applied to nonlinear identification of the highly nonlinear loop dynamics of an automatic voltage regulation (AVR) loop across a wide operating range. A Takagi-Sugeno fuzzy model is automatically constructed from plant input-output response data set. The validity of the resultant TS fuzzy model representation is confirmed on a validated simulation of a 3kVA laboratory micromachine.

Open loop dynamic trajectory generator for a fuzzy gain scheduler

This paper presents a dynamic trajectory generator for a nonlinear system to be controlled by a fuzzy gain scheduler composed of a set of local linear Takagi–Sugeno (TS) fuzzy controllers. The local control laws are designed for the error system including the desired state and the corresponding desired control input. The task of the open loop dynamic trajectory generator is the generation of a sequence of control inputs along a predefined dynamic trajectory of the nominal nonlinear system. While the desired state is normally given, the corresponding desired control input may not always be computable in an explicit or unique way. With the proposed method the desired control input is approximated by an inverse fuzzy model of the nominal system. The model is built on the basis of a combination of c-elliptotype and Gustafson–Kessel clustering and a subsequent identification of local linear and affine TS models. In a next one-step ahead optimization loop the approximated control input is corrected by an analytical forward model of the nominal system.