Fuzzy C-means Objective Function Research Articles

Double-Constraint Fuzzy Clustering Algorithm

Given a set of data objects, the fuzzy c-means (FCM) partitional clustering algorithm is favored due to easy implementation, rapid response, and feasible optimization. However, FCM fails to reflect either the importance degree of the individual data objects or that of the clusters. Numerous variants of FCM have been proposed to address these issues. However, most of them cannot effectively apply the available information on data objects or clusters. In this paper, a double-constraint fuzzy clustering algorithm is proposed to reflect the importance degrees of both individual data objects and clusters. By incorporating double constraints into each data object and cluster, the objective function of FCM is reformulated and its realization equation is mathematically conducted. Consequently, the clustering accuracy of FCM is improved by applying the available information on both data objects and clusters. Especially, the proposed algorithm effectively addresses the limitations inherent in the existing variants of FCM. The experimental results validate the effectiveness, implementation, and robustness of the new fuzzy clustering algorithm.

Fuzzy subspace clustering noisy image segmentation algorithm with adaptive local variance & non-local information and mean membership linking

The Fuzzy C-means (FCM) clustering algorithm is an effective method for image segmentation. Non-local spatial information considers more redundant information of the image thus is more robust to noise. However, under-segmentation of non-local spatial information may exist with higher noise density. The number of iteration steps is also significant in FCM, and employing membership linking can effectively reduce the number of iteration steps. Nonetheless, when there are outliers in the membership degree, the membership linking can make the algorithm converge prematurely before reaching the optimum, affecting segmentation performance. This paper presents a fuzzy subspace clustering noisy image segmentation algorithm with adaptive local variance & non-local information and mean membership linking (FSC_LNML). Firstly, local variance templates are utilized to eliminate the under-segmentation of non-local information, and local variance & non-local information are integrated into the FCM objective function to improve robustness. Secondly, the mean membership linking is employed as the denominator of the objective function to reduce the number of iterations and solve the problem that the algorithm converges early before reaching the optimum when the membership has an outlier. Thirdly, the absolute intensity difference between the original image and the local variance & non-local information and its inverse are used to adaptively constrain the original image and the local variance & non-local information. Finally, the concept of the subspace is introduced to adaptively assign appropriate weights to each dimension of the image to improve the segmentation performance of color images. The simulation results on noisy grayscale images and noisy color images show that the efficiency of the proposed method FSC_LNML is better than other fuzzy-based clustering algorithms. The convergence proof of the algorithm is also presented.

Sparse Regularization-Based Fuzzy C-Means Clustering Incorporating Morphological Grayscale Reconstruction and Wavelet Frames

The conventional fuzzy C-means (FCM) algorithm is not robust to noise and its rate of convergence is generally impacted by data distribution. Consequently, it is challenging to develop FCM-related algorithms that have good performance and require less computing time. In this article, we elaborate on a comprehensive FCM-related algorithm for image segmentation. To make FCM robust, we first utilize a morphological grayscale reconstruction (MGR) operation to filter observed images before clustering, which guarantees noise-immunity and image detail-preservation. Since real images can generally be approximated by sparse coefficients in a tight wavelet frame system, feature spaces of observed and filtered images can be obtained. Taking such features to be clustered, we investigate an improved FCM model in which a sparse regularization term is introduced into the objective function of FCM. We design a three-step iterative algorithm to solve the sparse regularization-based FCM model, which is constructed by the Lagrangian multiplier method, hard-threshold operator, and normalization operator, respectively. Such an algorithm can not only perform well for image segmentation, but also come with high computational efficiency. To further enhance the segmentation accuracy, we use MGR to filter the label set generated by clustering. Finally, a large number of supporting experiments and comparative studies with other FCM-related algorithms available in the literature are provided. The obtained results for synthetic, medical and color images indicate that the proposed algorithm has good ability for multiphase image segmentation, and performs better than other alternative FCM-related algorithms. Moreover, the proposed algorithm requires less time than most of the existing algorithms.

Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size

In the last two decades, information entropy measures have been relevantly applied in fuzzy clustering problems in order to regularize solutions by avoiding the formation of partitions with excessively overlapping clusters. Following this idea, relative entropy or divergence measures have been similarly applied, particularly to enable that kind of entropy-based regularization to also take into account, as well as interact with, cluster size variables. Particularly, since Rényi divergence generalizes several other divergence measures, its application in fuzzy clustering seems promising for devising more general and potentially more effective methods. However, previous works making use of either Rényi entropy or divergence in fuzzy clustering, respectively, have not considered cluster sizes (thus applying regularization in terms of entropy, not divergence) or employed divergence without a regularization purpose. Then, the main contribution of this work is the introduction of a new regularization term based on Rényi relative entropy between membership degrees and observation ratios per cluster to penalize overlapping solutions in fuzzy clustering analysis. Specifically, such Rényi divergence-based term is added to the variance-based Fuzzy C-means objective function when allowing cluster sizes. This then leads to the development of two new fuzzy clustering methods exhibiting Rényi divergence-based regularization, the second one extending the first by considering a Gaussian kernel metric instead of the Euclidean distance. Iterative expressions for these methods are derived through the explicit application of Lagrange multipliers. An interesting feature of these expressions is that the proposed methods seem to take advantage of a greater amount of information in the updating steps for membership degrees and observations ratios per cluster. Finally, an extensive computational study is presented showing the feasibility and comparatively good performance of the proposed methods.

Learning fuzzy clustering for SPECT/CT segmentation via convolutional neural networks.

Quantitative bone single-photon emission computed tomography (QBSPECT) has the potential to provide a better quantitative assessment of bone metastasis than planar bone scintigraphy due to its ability to better quantify activity in overlapping structures. An important element of assessing the response of bone metastasis is accurate image segmentation. However, limited by the properties of QBSPECT images, the segmentation of anatomical regions-of-interests (ROIs) still relies heavily on the manual delineation by experts. This work proposes a fast and robust automated segmentation method for partitioning a QBSPECT image into lesion, bone, and background. We present a new unsupervised segmentation loss function and its semi- and supervised variants for training a convolutional neural network (ConvNet). The loss functions were developed based on the objective function of the classical Fuzzy C-means (FCM) algorithm. The first proposed loss function can be computed within the input image itself without any ground truth labels, and is thus unsupervised; the proposed supervised loss function follows the traditional paradigm of the deep learning-based segmentation methods and leverages ground truth labels during training. The last loss function is a combination of the first and the second and includes a weighting parameter, which enables semi-supervised segmentation using deep learning neural network. We conducted a comprehensive study to compare our proposed methods with ConvNets trained using supervised, cross-entropy and Dice loss functions, and conventional clustering methods. The Dice similarity coefficient (DSC) and several other metrics were used as figures of merit as applied to the task of delineating lesion and bone in both simulated and clinical SPECT/CT images. We experimentally demonstrated that the proposed methods yielded good segmentation results on a clinical dataset even though the training was done using realistic simulated images. On simulated SPECT/CT, the proposed unsupervised model's accuracy was greater than the conventional clustering methods while reducing computation time by 200-fold. For the clinical QBSPECT/CT, the proposed semi-supervised ConvNet model, trained using simulated images, produced DSCs of and for lesion and bone segmentation in SPECT, and a DSC of bone segmentation of CT images. These DSCs were larger than that for standard segmentation loss functions by for SPECT segmentation, and for CT segmentation with P-values from a paired t-test. A ConvNet-based image segmentation method that uses novel loss functions was developed and evaluated. The method can operate in unsupervised, semi-supervised, or fully-supervised modes depending on the availability of annotated training data. The results demonstrated that the proposed method provides fast and robust lesion and bone segmentation for QBSPECT/CT. The method can potentially be applied to other medical image segmentation applications.

Using fuzzy C-means Clustering to Identify Heavy Metal Polluted Soil in a Certain Area of Shanghai

Fuzzy clustering is an important branch of fuzzy pattern recognition, which has been widely used in many fields such as data mining, image processing, big data analysis and so on. Among many fuzzy clustering algorithms, fuzzy c-means algorithm is the most widely used. In this paper, the fuzzy c-means clustering is applied to the identification of polluted soil. To solve the problem of determining the optimal number of clusters for this method, the method we choose is obtaining an initial clustering result firstly and then merging. Based on the important characteristic of fuzzy c-means method that the objective function of fuzzy c-means clustering decreases rapidly as the number of clusters increases, and the rate become slow after exceeding the optimal number of clusters, we choose the clustering result whose objective function is reduced to a certain degree as the initial clustering result. Then use the parameter estimation method in statistics to estimate the distance between classes, determine the reasonable range of distance between classes, combine the initial classification results, and finally perform the final clustering results according to the clustering validity function. Evaluation and experiments prove the feasibility and effectiveness of the scheme.

Noise distance driven fuzzy clustering based on adaptive weighted local information and entropy-like divergence kernel for robust image segmentation

Kernel method is an effective way to solve the problem of nonlinear mode analysis, and its key is the selection or construction of kernel function. This paper firstly induced entropy-like divergence by combining Jensen-Shannon/Bregman divergence with convex function, its mercer kernel function called entropy-like divergence kernel is also constructed. Secondly, an adaptive noise distance based on entropy-like divergence kernel and a novel fuzzy weighted local factor of robust fuzzy clustering are presented, and they are also embedded into the objective function of fuzzy C-means clustering with noise cluster. In the end, a novel noise-resistant fuzzy weighed local information clustering based on entropy-like divergence kernel (NEKWFLICM) is proposed, and its convergence is strictly proved by convergence theorem of alternating iteration. Many experimental results delicate that the proposed algorithm has more robust and accurate than a series of existing state-of-the-art Gaussian kernel-based fuzzy clustering-related segmentation algorithms in the presence of high noise.

Supervised Fuzzy C-Means Techniques to Solve the Capacitated Vehicle Routing Problem

Fuzzy C-Means (FCM) clustering technique is among the most effective partitional clustering algorithms available in the literature. The Capacitated Vehicle Routing Problem (CVRP) is an important industrial logistics and managerial NP-hard problem. Cluster-First Route-Second Method (CFRS) is one of the efficient techniques used to solve CVRP. In CFRS technique, customers are first divided into clusters in the first phase, then each cluster is solved independently as a Traveling Salesman Problem (TSP) in the second phase. This research is concerned with the clustering phase of CFRS, and TSP is then solved using a traditional optimization method. Three supervised FCM based techniques are proposed to solve the clustering phase at reduced cost via centroids (pre-FCM) initialization phase. The proposed pre-FCM initialization techniques are developed to be problem dependent. Hence, three initialization techniques are first developed using K-means technique, spatially equally distributed, and demand weighted center of mass. Then, a modified demand weighted fuzzy c-means objective function is employed to assign customers to clusters. To compare the performance of the proposed supervised FCM techniques, forty-two CVRP benchmark problems are solved using the traditional fuzzy C-means algorithm and the developed algorithms. Extensive comparisons are conducted between the traditional fuzzy C-means algorithm, the three proposed initialization techniques, and other fuzzy C-means techniques available in the literature. Results show that the proposed three initialization techniques are efficient in terms of solution quality and computational cost.

Image clustering algorithm using superpixel segmentation and non‐symmetric Gaussian–Cauchy mixture model

In this study, an unsupervised clustering algorithm is proposed to label superpixel density images. Firstly, the authors propose a novel superpixel segmentation algorithm driven by a modified fuzzy C-means objective function, Kullback–Leibler (KL) divergence, and an entropy term, which generate superpixels with good boundary adherence and intensity homogeneity. In this model, the logarithm of Gaussian distribution as a new distance metric is used to improve the accuracy of boundary pixel classification, the KL divergence is applied to regularise the fuzzy objective function. Based on this model, the generated superpixel intensity images with a highly distinctive background colour from the colour of the target are obtained. Grouping cues generated by superpixels can affect the performance of image clustering greatly. Next, according to the small amount of clustering data generated by the superpixel intensity images, they construct a non-symmetric mixture model based on a mixture of Gaussian distribution and Cauchy distribution for implementing image clustering. Thus, clustering of colour images is transformed into clustering of these newly generated data. The advantage of this model is its well adaption to different shapes of observed data. Experimental results on publicly available data sets are provided to demonstrate the effectiveness of the proposed algorithm.

Pairwise Constrained Fuzzy Clustering: Relation, Comparison and Parallelization

Although clustering with pairwise constraints through penalty regularization has been widely adopted in existing semi-supervised clustering approaches, little work has been done on theoretical comparison of these pairwise constrained approaches with respect to difference in penalties. In this paper, we first propose two types of penalties in the context of pairwise constrained fuzzy clustering. The first one accounts for the overall consistency of assignments in terms of fuzzy memberships regarding constrained pairs. The second one is the total Euclidean distance between membership vectors of constrained pairs. After analytical discussion, we establish the connection between different penalties to provide a unified view as well as a better understanding of each of them. Following the idea of penalty regularization, variants of pairwise constrained fuzzy c-means are formulated by incorporating the consistency-type and distance-type penalties, respectively, into the objective function of fuzzy c-means. We also extend this idea to co-clustering by considering pairwise constraints of two types of objects to produce fuzzy co-clusters. Efficient and scalable algorithms have been proposed for parallel implementation. The experimental results with real-world datasets show good performance of the proposed approaches with respect to effectiveness, efficiency and scalability.

Feature selection strategy based on hybrid crow search optimization algorithm integrated with chaos theory and fuzzy c-means algorithm for medical diagnosis problems

Powerful knowledge acquisition tools and techniques have the ability to increase both the quality and the quantity of knowledge-based systems for real-world problems. In this paper, we designed a hybrid crow search optimization algorithm integrated with chaos theory and fuzzy c-means algorithm denoted as CFCSA for feature selection problems of medical diagnosis. In the proposed CFCSA framework, the crow search algorithm adopts the global optimization technique to avoid the sensitivity of local optimization. The fuzzy c-means (FCM) objective function is used as a cost function for the chaotic crow search optimization algorithm. The proposed algorithm CFCSA is benchmarked against the binary crow search algorithm (BCSA), chaotic ant lion optimization algorithm (CALO), binary ant lion optimization algorithm (BALO) and bat algorithm relevant methods. The proposed CFCSA algorithm vs. BCSA, CALO, BALO and bat algorithm is tested on diabetes, heart, Radiopaedia CT liver, breast cancer, lung cancer, cardiotocography, ILPD, liver disorders, hepatitis and arrhythmia. Experimental results show the proposed method CFCSA is better against comparative models in feature selection on the medical diagnosis data sets.

A Feature-Reduction Fuzzy Clustering Algorithm Based on Feature-Weighted Entropy

Fuzzy clustering algorithms generally treat data points with feature components under equal importance. However, there are various datasets with irrelevant features involved in clustering process that may cause bad performance for fuzzy clustering algorithms. That is, different feature components should take different importance. In this paper, we present a novel method for improving fuzzy clustering algorithms that can automatically compute individual feature weight, and simultaneously reduce these irrelevant feature components. In fuzzy clustering, the fuzzy c-means (FCM) algorithm is the best known. We first consider the FCM objective function with feature-weighted entropy, and construct a learning schema for parameters, and then reduce these irrelevant feature components. We call it a feature-reduction FCM (FRFCM). During FRFCM processes, a new procedure for eliminating irrelevant feature(s) with small weight(s) is created for feature reduction. The computational complexity of FRFCM is also analyzed. Some numerical and real datasets are used to compare FRFCM with various feature-weighted FCM methods in the literature. Experimental results and comparisons actually demonstrate these good aspects of FRFCM with its effectiveness and usefulness in practice.

A fuzzy image clustering method based on an improved backtracking search optimization algorithm with an inertia weight parameter

In this paper, we introduced a novel image clustering method based on combination of the classical Fuzzy C-Means (FCM) algorithm and Backtracking Search optimization Algorithm (BSA). The image clustering was achieved by minimizing the objective function of FCM with BSA. In order to improve the local search ability of the new algorithm, an inertia weight parameter (w) was proposed for BSA. The improvement was accomplished by using w in the steps of the determination of the search-direction matrix of BSA and the new algorithm was named as w-BSAFCM. In order to show the effectiveness of the new algorithm, FCM was also combined with the general forms of BSA in the same manner and three benchmark images were clustered by utilizing these algorithms. The obtained results were analyzed according to the objective function and Davies-Bouldin index values to compare the performances of the algorithms. According to the results, it was shown that w-BSAFCM can be effectively be used for solving image clustering problem.

Sparse Regularization in Fuzzy c-Means for High-Dimensional Data Clustering.

In high-dimensional data clustering practices, the cluster structure is commonly assumed to be confined to a limited number of relevant features, rather than the entire feature set. However, for high-dimensional data, identifying the relevant features and discovering the cluster structure are still challenging problems. To solve these problems, this paper proposes a novel fuzzy c-means (FCM) model with sparse regularization (ℓq(0<q≤1)-norm regularization), by reformulating the FCM objective function into the weighted between-cluster sum of square form and imposing the sparse regularization on the weights. An algorithm is also developed to explicitly solve the proposed model. Compared with the existing clustering models, the proposed model can shrink the weights of irrelevant features (noisy features) to exact zero, and also can be efficiently solved in analytic forms when q = 1,1/2. Experiments on both synthetic and real-world data sets show that the proposed approach outperforms the existing clustering approaches.

Image Guided Fuzzy C-Means for Image Segmentation

Prior knowledge has been considered as valuable supplementary information in many image processing techniques. In this paper, we take the input image itself as the guidance prior and develop a novel fuzzy clustering algorithm to segment it by adding a new term to the objective function of Fuzzy C-Means. The new term comes from Guided Filter for its capability in noise suppression and edge-preserving smoothing. As a result, the memberships derived from the new objective function incorporate the guidance information from the image to be segmented. In this way, the segmentation result retains more subtle details on the boundaries of segments. According to experimental results, the proposed method shows good performance in image segmentation tasks especially for images with high noise rates.

A fuzzy clustering image segmentation algorithm based on Hidden Markov Random Field models and Voronoi Tessellation

In this paper, we present new results related to the Voronoi Tessellation (VT) and Hidden Markov Random Field (HMRF) based Fuzzy C-Means (FCM) algorithm (VTHMRF-FCM) for texture image segmentation. In the VTHMRF-FCM algorithm, a VTHMRF model is established by using VT to partition an image domain into sub-regions (Voronoi polygons) and HMRF to describe the relationship of neighbor sub-regions. Based on the VTHMRF model, the objective function of VTHMRF-FCM is defined by adding a regularization term of Kullback–Leibler (KL) divergence information to FCM objective function. The proposed algorithm combines the benefits stemming from robust regional HMRF and FCM based clustering segmentation. Segmentation experiments on synthetic and real images by the proposed and other improved FCM algorithms are performed. Their results demonstrate that the proposed algorithm can obtain much better segmentation results than other FCM based methods.

A Modified Fuzzy C-Means Algorithm for Brain MR Image Segmentation and Bias Field Correction

In quantitative brain image analysis, accurate brain tissue segmentation from brain magnetic resonance image (MRI) is a critical step. It is considered to be the most important and difficult issue in the field of medical image processing. The quality of MR images is influenced by partial volume effect, noise, and intensity inhomogeneity, which render the segmentation task extremely challenging. We present a novel fuzzy c-means algorithm (RCLFCM) for segmentation and bias field correction of brain MR images. We employ a new gray-difference coefficient and design a new impact factor to measure the effect of neighbor pixels, so that the robustness of anti-noise can be enhanced. Moreover, we redefine the objective function of FCM (fuzzy c-means) by adding the bias field estimation model to overcome the intensity inhomogeneity in the image and segment the brain MR images simultaneously. We also construct a new spatial function by combining pixel gray value dissimilarity with its membership, and make full use of the space information between pixels to update the membership. Compared with other state-of-the-art approaches by using similarity accuracy on synthetic MR images with different levels of noise and intensity inhomogeneity, the proposed algorithm generates the results with high accuracy and robustness to noise.

A Spatial Fuzzy Clustering Algorithm With Kernel Metric Based on Immune Clone for SAR Image Segmentation

The fuzzy c-means (FCM) clustering algorithm has been widely used in image segmentation. However, FCM exhibits poor robustness to noise, often leading to unsatisfactory segmentations on noisy images. Additionally, the FCM algorithm is sensitive to the choice of initial cluster centers. In order to solve these problems, this paper proposes clone kernel spatial FCM (CKS_FCM), which improves segmentation performance in several ways. First, in CKS_FCM, an immune clone algorithm is used to generate the initial cluster centers, which helps prevent the algorithm from converging on local optima. Second, CKS_FCM improves the robustness to noise by incorporating spatial information into the objective function of FCM. Third, CKS_FCM uses a non-Euclidean distance based on a kernels metric, instead of the Euclidean distance conventionally used in FCM, to enhance the segmentation accuracy (SA). We present experimental results on both real and synthetic SAR images, which suggest that the proposed method can generate higher accuracy, and obtain more robustness to noise, as compared against six state-of-the-art methods from the literatures.

Efficient kernel induced fuzzy c-means based on Gaussian function for imagedata analyzing

In this paper we introduce a new fuzzy c-means objective function called kernel induced fuzzy c-means based on Gaussian function for the purpose of segmentation of medical images. The originality of this algorithm is based on the fact that the conventional FCM-based algorithm considers no spatial c ontent information, which makes it sensitive to noise. The new algorithm is formulated by incorporating the spatial neighbourhood information into the spatial neighbourhood into the original FCM algorithm by apriori probability and initialized by a kernel function induced histogram based FCM algorithm. The probability in the algorithm that indicates the spatial influence of the neighbouring pixels on the centre pixel plays a key role in this algorithm and it obtains efficient method for calculating membership and updating prototypes by minimizing the new objective function of Gaussian based fuzzy c-means. The performance of proposed algorithm has been tested with medical images to reduce the inhomogeneities and to allow the labelling of a pixel to be influenced by the labels in its immediate neighbourhood. Also this paper compares the results of proposed method with the results of existing basic fuzzy c-means.

Exploring the uniform effect of FCM clustering: A data distribution perspective

Fuzzy c-means (FCM) is a well-known and widely used fuzzy clustering method. Though there have been considerable studies that focused on the improvement of FCM algorithm or its applications, it is still necessary to understand the effect of data distributions on the performance of FCM. In this paper, we present an organized study of FCM clustering from the perspective of data distribution. We first analyze the structure of the objective function of FCM and find that FCM has the same uniform effect as K-means. Namely, FCM also tends to produce clusters of relatively uniform sizes. The coefficient of variation (CV) is introduced to measure the variation of cluster sizes in a given data set. Then based on the change of CV values between the original “true” cluster sizes and the cluster sizes partitioned by FCM clustering, a necessary but not sufficient criterion for the validation of FCM clustering is proposed from the data distribution perspective. Finally, our experiments on six synthetic data sets and ten real-world data sets further demonstrate the uniform effect of FCM. It tends to reduce the variation in cluster sizes when the CV value of the original data distribution is larger than 0.88, and increase the variation when the variation of original “true” cluster sizes is low.