Fuzzy Clustering Problem Research Articles

Fuzzy clustering with capacity constraints: Algorithm, convergence analysis and numerical experiments

In this paper we study the fuzzy clustering problem with capacity constraints. Despite of the fact that the fuzzy clustering approach is widely encountered in the literature, the inclusion of capacity constraints is recent and has several practical applications. We propose a general formulation of the clustering problem, where each point has an associated weight and the sum of the weights of the points that compose each group is established a priori. We discuss existence of solutions of the involved problems, providing a mathematical foundation for the established formulas. Besides, we propose a practical algorithm for solving this problem and present its convergence analysis. This algorithm follows an alternate minimization scheme, wherein a given iteration addresses the problem first in terms of the probabilities of each point xj belonging to each cluster i, denoted as uij, finding subsequently the position of the centroids, ci,i=1,…,g,j=1,…,n. This procedure is K-means-like, with the distinction that, as a point does not exclusively belong to a group, the computation of uij requires optimization techniques. In our case, this involves solving a linear system derived from the Karush–Kuhn–Tucker (KKT) conditions. With the aim of validating our algorithm, we present numerical tests with synthetic and real-world data to demonstrate its performance for the given problems. Since the proposal successfully solved these numerical tests within a reasonable computational time, it can be considered a valuable resource for addressing real-world applications.

A New Criterion for Improving Convergence of Fuzzy C-Means Clustering

One of the most used algorithms to solve the fuzzy clustering problem is Fuzzy C-Means; however, one of its main limitations is its high computational complexity. It is known that the efficiency of an algorithm depends, among other factors, on the strategies for its initialization and convergence. In this research, a new convergence strategy is proposed, which is based on the difference of the objective function values, in two consecutive iterations, expressed as a percentage of its value in the next to the last one. Additionally, a new method is proposed to optimize the selection of values of the convergence or stop threshold of the algorithm, which is based on the Pareto principle. To validate our approach, a collection of real datasets was solved, and a significant reduction in the number of iterations was observed, without affecting significantly the solution quality. Based on the proposed method and the experiments carried out, we found it is convenient to use threshold values equal to 0.73 and 0.35 if a decrease in the number of iterations of approximately 75.2% and 64.56%, respectively, is wanted, at the expense of a reduction in solution quality of 2% and 1%, respectively. It is worth mentioning that, as the size of the datasets is increased, the proposed approach tends to obtain better results, and therefore, its use is suggested for datasets found in Big Data and Data Science.

CREDIBILISTIC ROBUST ONLINE FUZZY CLUSTERING IN DATA STREAM MINING TASKS

Context. The task of clustering-classification without a teacher of data arrays occupies an important place in the general problem of Data Mining, and for its solution there exists currently many approaches, methods and algorithms. There are quite a lot of situations where the real data to be clustered are corrupted with anomalous outliers or disturbances with non-Gaussian distributions. It is clear that “classical” methods of artificial intelligence (both batch and online) are ineffective in this situation. The goal of the paper is to develop a credibilistic robust online fuzzy clustering method that combines the advantages of credibilistic and robust approaches in fuzzy clustering tasks.
 Objective. The goal of the work is online credibilistic fuzzy clustering of distorted data, using of credibility theory in data stream mining.
 Method. The procedure of fuzzy clustering of data using credibilistic approach based on the use of both robust goal functions of a special type, insensitive to outliers and designed to work both in batch and its recurrent online version designed to solve Data Stream Mining problems when data are fed to processing sequentially in real time.
 Results. Analyzing the obtained results overall accuracy of clustering methods and algorithm, proposed method similar with result of credibilistic fuzzy clustering method, but has time superiority regardless of the number observations that fed on clustering process.
 Conclusions. The problem of fuzzy clustering of data streams contaminated by anomalous non-Gaussian distributions is considered. A recurrent credibilistic online algorithm based on the objective function of a special form is introduced, which suppresses these outliers by using the hyperbolic tangent function, which, in addition to neural networks, is used in robust estimation tasks. The proposed algorithm is quite simple in numerical implementation and is a generalization of some well-known online fuzzy clustering procedures intended for solving Data Stream Mining problems.

On fuzzification and optimization problems of clustering indices

Results of clustering are qualitatively evaluated by quantities called clustering indices. While many clustering indices are proposed, in B. Desgraupes [Clustering Indices (2016)], Desgraupes reviewed 27 indices, most of which are applicable only to crisp clustering and only one of which is applicable to fuzzy clustering. In the previous article [D. R. Sope and M. Fujio, On a fuzzification of clustering indices, in Proc. Fuzzy System Symposium of the Japan Intelligent Information Fuzzy Society, Vol. 35 (2002), pp. 443–448], the authors gazed on analytic 3 indices among them and modified them to fit fuzzy clustering by regarding the membership degrees as distribution functions of objects over clusters. In this paper, this method called fuzzification, is applied to all the indices of Desgraupes. This investigation also includes optimization of indices. A significant benefit of fuzzy clustering is that the membership degrees allow the optimization problems to be treated as continuous, whereas the ones in the crisp case are discrete, making it generally easier to solve. After giving a precise description of fuzzification which is briefly given in D. R. Sope and M. Fujio [On a fuzzification of clustering indices, in Proc. Fuzzy System Symposium of the Japan Intelligent Information Fuzzy Society, Vol. 35 (2002), pp. 443–448], the authors fuzzify all of 27 indices of Desgraupes and then solve the fuzzy clustering problems having 13 analytic indices among them as the objective functions by the gradient method.

Fuzzy Cluster-based Group-wise Point Set Registration with Quality Assessment.

This article studies group-wise point set registration and makes the following contributions: "FuzzyGReg", which is a new fuzzy cluster-based method to register multiple point sets jointly, and "FuzzyQA", which is the associated quality assessment to check registration accuracy automatically. Given a group of point sets, FuzzyGReg creates a model of fuzzy clusters and equally treats all the point sets as the elements of the fuzzy clusters. Then, the group-wise registration is turned into a fuzzy clustering problem. To resolve this problem, FuzzyGReg applies a fuzzy clustering algorithm to identify the parameters of the fuzzy clusters while jointly transforming all the point sets to achieve an alignment. Next, based on the identified fuzzy clusters, FuzzyQA calculates the spatial properties of the transformed point sets and then checks the alignment accuracy by comparing the similarity degrees of the spatial properties of the point sets. When a local misalignment is detected, a local re-alignment is performed to improve accuracy. The proposed method is cost-efficient and convenient to be implemented. In addition, it provides reliable quality assessments in the absence of ground truth and user intervention. In the experiments, different point sets are used to test the proposed method and make comparisons with state-of-the-art registration techniques. The experimental results demonstrate the effectiveness of our method. The code is available at https://gitsvn-nt.oru.se/qianfang.liao/FuzzyGRegWithQA.

Two effective heuristic methods of determining the numbers of fuzzy clustering centers based on bilevel programming

Fuzzy clustering method plays an increasingly important role in data analysis, image segmentation and many other fields. How to determine the number of clustering centers is a technical problem faced by most clustering methods. Starting from the index-based clustering method and the idea based on the fusion of multiple clustering techniques, this paper designs two methods of determining the numbers of fuzzy clustering centers based on bilevel programming respectively. Firstly, based on the bilevel fuzzy clustering model, an evolutionary algorithm is designed based on the absorptive criterion of fuzzy clustering method(EA-Ac-FCM), and its effectiveness in solving bilevel fuzzy clustering problem is verified by empirical analysis. Secondly, we propose an alternate minimization clustering by a collaborative strategy(AM-CC), the mean shift and fuzzy clustering complement each other and guide the clustering together. Compared with the traditional clustering methods, all of the numerical tests show that the two proposed methods could identify the number of cluster centers of the objective to be clustered correctly and handle the data analysis and image segmentation tasks better.

Adaptive neuro-fuzzy clustering of distorted data based on prototype-centroid strategy using evolutionary procedures

The problem of clustering is a very relevant area in Data Mining of different nature. To solve this problem, there are a large number of known methods and algorithms, most of which work in batch mode, in conditions when the entire of data set is known in advance and does not change over the time. These methods are complex in software implementa-tion and are not without drawbacks. The aim of the work is to develop an adaptive neuro-fuzzy clustering method of distorted data based on proto-type-centroid strategy using evolutionary procedures, that solves the problem in online mode, when data are fed se-quentially in real time and is characterized by numerical simplicity and high speed. The problem of adaptive fuzzy clustering of distorted data by outliers and emissions, which are presented in the form of vector arrays, based on the strategy of the nearest prototype - centroid using evolutionary procedures, is con-sidered. The proposed approach is based on the online probabilistic fuzzy clustering procedure with the membership function of special type and the evolutionary cat swarm algorithm. Proposed adaptive neuro-fuzzy clustering method of distorted data based on prototype-centroid strategy using evolutionary procedures characterized by computational simplicity, high speed and accuracy of the results based on experimental studies. The modification of optimization procedure that based on cat swarm algorithm was propose. The proposed method is simple in numerical implementation, workable in the case when the data is distorted and are fed sequentially in online mode, that is confirmed experimentally.

Land-Cover Change Detection for SAR Images Based on Biobjective Fuzzy Local Information Clustering Method With Decomposition

The existence of a speckle noise significantly affects the accuracy of land-cover change detection results for synthetic aperture radar (SAR) images. This letter proposes a biobjective fuzzy local information clustering method with decomposition (BIFLICM/D) to address this problem. SAR images change detection is described as a biobjective fuzzy local information clustering problem from the aspects of preserving image details and removing noise. To improve the ability to extract the original information detail, the log-mean ratio method is used to generate a first difference image in BIFLICM/D. The second difference image is achieved by combining the homomorphic filtering and saliency detection, which effectively removes the speckle noise. Fuzzy clustering objective functions are then constructed for the two difference images to recognize the changed and unchanged pixels under different requirements. A new fuzzy membership degree-updating method is adopted to optimize the two objective functions, which can balance the influences of the two objectives and improve the robustness of the proposed method. The experimental result demonstrates that the proposed method is more effective and superior to the comparison algorithms.

A Novel Cluster Prediction Approach Based on Locality-Sensitive Hashing for Fuzzy Clustering of Categorical Data

This paper addresses the problem of fuzzy clustering for categorical data. During the last two decades, many attempts have been made to extend the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means algorithm, making it applicable to clustering for categorical data, due to its simplicity and efficiency. However, as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means-like algorithms are local optimization methods, their clustering results are highly sensitive to initialization. In this paper, we propose to use Locality-Sensitive Hashing (LSH) to reduce the categorical data dimensions and predict the initial fuzzy clusters in low-dimensional space. Different from the existing cluster initialization optimization methods that aim to create only crisp initial clusters, the proposed method aims at predicting ‘high quality’ fuzzy clusters at the initialization step before proceeding in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means-like fashion. The numerical results show that the proposed method yields relatively accurate results on 16 UCI datasets and outperforms all other related approaches in terms of both crisp and fuzzy clustering effectiveness.

Fuzzy clustering of Acute Lymphoblastic Leukemia images assisted by Eagle strategy and morphological reconstruction

Patients with Acute Lymphoblastic Leukemia (ALL) require prompt diagnosis since it has the chance to become fatal if neglected for a few weeks. The microscopic image of lymphocyte cells creates doubts even for expert pathologists because normal lymphocytes cells and ALL blast cells are both very smooth. Therefore, proper segmentation of White Blood Cells (WBC) is a very crucial aspect of the task. Consequently, the focus of the study is on segmenting the WBCs in Acute Lymphoblastic Leukemia (ALL) images utilizing classical crisp and fuzzy clustering approaches like K-means (KM) and Fuzzy C-means (FCM). But these classical clustering approaches are very sensitive to noise and initial cluster center initialization and hence trapped in local optima. As a result, these techniques may produce incorrect cluster centers. Firstly, researchers are employing Nature-Inspired Optimization Algorithms (NIOAs) as an alternate methodology for both crisp and fuzzy clustering problems to solve the initial cluster center initialization issue. Therefore, using a two-stage Eagle Strategy based on Stochastic Fractal Search (SFS) method, this research proposes a fuzzy clustering methodology. Secondly, morphological reconstruction has been employed for filtering the membership matrix to guarantee noise-immunity. A scrupulous parallel study is performed among the proposed eagle strategy based fuzzy clustering depending on morphological reconstruction with some well-known NIOA based fuzzy clustering and crisp clustering approaches, and classical clustering methodologies like KM and FCM in view of a collection of color ALL images and regular performance metrics. Experimental results demonstrate that recommended ES-SFS based fuzzy clustering technique with morphological reconstruction surpasses the most of utilized approaches in words of computation effort, quality metrics, and robustness. Additionally, to get rid of the random effect in the achieved numerical results, a non-parametric strategy is used for statistical validation.

Multi-view fuzzy clustering of deep random walk and sparse low-rank embedding

Multi-view clustering aims to improve the learning performance by exploiting discriminative information from heterogeneous data sources. It has been capturing growing research attention due to its wide utilization in the areas of data mining and computer vision. However, the full use of complementarity and consistency from multi-view data is still a challenging task. Here, we propose a novel multi-view fuzzy clustering, which adopts the joint learning of deep random walk and sparse low-rank embedding. First, a deep random walk is employed to acquire a robust similarity matrix of data points and convert fuzzy membership matrix learning to adaptive graph learning. Second, the adaptive graph is restricted with sparse low-rank constraints, which ensures its strong discriminative ability and effective cluster assignments. Third, by exploiting the sparse low-rank property, the multi-view fuzzy clustering problem is formulated as optimizing a regularized graph adjacency matrix with distance metric learning and spectral norm minimization simultaneously. The distance metric learning is embedded to scatter all data points so that any two samples are projected onto a low-dimensional subspace with a large margin. Finally, an efficient algorithm is developed for the formulated problem and its convergence is also guaranteed. In order to demonstrate the superiority of the proposed method, extensive experiments are conducted by comparing state-of-the-arts on real-world databases.

Credibilistic fuzzy clustering based on evolutionary method of crazy cats

The problem of fuzzy clustering of large datasets that are sent for processing in both batch and online modes, based on a credibilistic approach, is considered. To find the global extremum of the credibilistic fuzzy clustering goal function, the modification of the swarm algorithm of crazy cats swarms was introduced, that combined the advantages of evolutionary algorithms and a global random search. It is shown that different search modes are generated by a unified mathematical procedure, some cases of which are known algorithms for both local and global optimizations. The proposed approach is easy to implement and is characterized by the high speed and reliability in problems of multi-extreme fuzzy clustering.

Building fuzzy time series model from unsupervised learning technique and genetic algorithm.

This paper proposes a new model to interpolate time series and forecast it effectively for the future. The important contribution of this study is the combination of optimal techniques for fuzzy clustering problem using genetic algorithm and forecasting model for fuzzy time series. Firstly, the proposed model finds the suitable number of clusters for a series and optimizes the clustering problem by the genetic algorithm using the improved Davies and Bouldin index as the objective function. Secondly, the study gives the method to establish the fuzzy relationship of each element to the established clusters. Finally, the developed model establishes the rule to forecast for the future. The steps of the proposed model are presented clearly and illustrated by the numerical example. Furthermore, it has been realized positively by the established MATLAB procedure. Performing for a lot of series (3007 series) with the differences about characteristics and areas, the new model has shown the significant performance in comparison with the existing models via some parameters to evaluate the built model. In addition, we also present an application of the proposed model in forecasting the COVID-19 victims in Vietnam that it can perform similarly for other countries. The numerical examples and application show potential in the forecasting area of this research.

Fuzzy Clustering Methods with Rényi Relative Entropy and Cluster Size

In the last two decades, information entropy measures have been relevantly applied in fuzzy clustering problems in order to regularize solutions by avoiding the formation of partitions with excessively overlapping clusters. Following this idea, relative entropy or divergence measures have been similarly applied, particularly to enable that kind of entropy-based regularization to also take into account, as well as interact with, cluster size variables. Particularly, since Rényi divergence generalizes several other divergence measures, its application in fuzzy clustering seems promising for devising more general and potentially more effective methods. However, previous works making use of either Rényi entropy or divergence in fuzzy clustering, respectively, have not considered cluster sizes (thus applying regularization in terms of entropy, not divergence) or employed divergence without a regularization purpose. Then, the main contribution of this work is the introduction of a new regularization term based on Rényi relative entropy between membership degrees and observation ratios per cluster to penalize overlapping solutions in fuzzy clustering analysis. Specifically, such Rényi divergence-based term is added to the variance-based Fuzzy C-means objective function when allowing cluster sizes. This then leads to the development of two new fuzzy clustering methods exhibiting Rényi divergence-based regularization, the second one extending the first by considering a Gaussian kernel metric instead of the Euclidean distance. Iterative expressions for these methods are derived through the explicit application of Lagrange multipliers. An interesting feature of these expressions is that the proposed methods seem to take advantage of a greater amount of information in the updating steps for membership degrees and observations ratios per cluster. Finally, an extensive computational study is presented showing the feasibility and comparatively good performance of the proposed methods.

A forecasting model for time series based on improvements from fuzzy clustering problem

This article proposes a new fuzzy time series model that can interpolate historical data, and forecast effectively for the future. It is combination of the improved steps from the existing models. There are problems to use the percentage variations of series between consecutive periods of time as a universal set, to divide the universal set into clusters by the automatic algorithm based on the similarity between elements, to determine the relationships between elements in the series and the divided clusters by the improved fuzzy cluster analysis algorithm, and to interpolate the historical data and to forecast for future by new principle. The proposed algorithm is performed quickly and efficiently by the established Matlab procedure. It is illustrated by an example, and tested for many other data sets, especially for 3003 series in M3-Competition data. Comparing to the existing models, the proposed model always gives the best result. We also apply the proposed model in forecasting the salty peak for a coastal province of Vietnam. Examples and application show the potential of the studied problem.

A New Validity Index Based on Fuzzy Energy and Fuzzy Entropy Measures in Fuzzy Clustering Problems.

Two well-known drawbacks in fuzzy clustering are the requirement of assigning in advance the number of clusters and random initialization of cluster centers. The quality of the final fuzzy clusters depends heavily on the initial choice of the number of clusters and the initialization of the clusters, then, it is necessary to apply a validity index to measure the compactness and the separability of the final clusters and run the clustering algorithm several times. We propose a new fuzzy C-means algorithm in which a validity index based on the concepts of maximum fuzzy energy and minimum fuzzy entropy is applied to initialize the cluster centers and to find the optimal number of clusters and initial cluster centers in order to obtain a good clustering quality, without increasing time consumption. We test our algorithm on UCI (University of California at Irvine) machine learning classification datasets comparing the results with the ones obtained by using well-known validity indices and variations of fuzzy C-means by using optimization algorithms in the initialization phase. The comparison results show that our algorithm represents an optimal trade-off between the quality of clustering and the time consumption.

Novel coupled DP system for fuzzy C-means clustering and image segmentation

This study proposed a novel fuzzy c-means clustering method which calculates the density of the data points based on the weighted mean distance (WMFCM). A novel coupled DNA-GA and P system (DP system) is introduced to realize the clustering process. The evolutional rules in the coupled DP system can help the WMFCM algorithm jump out of the local optimum and get the initial clustering centers. The performance of the coupled DP system in dealing with fuzzy clustering problems is measured by conducting experimental analysis on UCI datasets and BSDS300 image datasets. And the experimental results are compared with several popular algorithms. Experimental results show that our algorithm can perform better than other algorithms.

Interval Intuitionistic Fuzzy Clustering Algorithm Based on Symmetric Information Entropy

Based on the continuous optimal aggregation operator, a novel distance measure is proposed to deal with interval intuitionistic fuzzy clustering problems. The optimal ordered weighted intuitionistic fuzzy quasi-averaging (OOWIFQ) operator and the continuous OOWIFQ operator are presented to aggregate all the values in an interval intuitionistic fuzzy number. Some of their desirable properties are also studied. The OOWIFQ operator can describe the fuzzy state of things more realistically and present the fuzzy properties more accurately. The opinions of experts are very important, the OOWIFQ operators take expert preferences into account to reduce systematic errors. Considering the hesitation of things and avoiding distortion of information, we put forward the distance measure for interval intuitionistic fuzzy numbers by using symmetric information entropy. Based on the continuous OOWIFQ operator and proposed distance measure, a new interval intuitionistic fuzzy clustering (IIFC) algorithm is proposed. The application in soil clustering shows the validity and practicability of the IIFC algorithm.

A Two-Stage Evolutionary Fuzzy Clustering Framework for Noisy Image Segmentation

This article presents a two-stage evolutionary fuzzy clustering framework for noisy image segmentation. It is a bi-stage system comprising a multi-objective optimization stage and a fuzzy clustering segmentation stage. In the multi-objective optimization stage, the fuzzy clustering on pixels in inhomogeneous regions is converted into a multi-objective problem, which can preserve image details while restraining noise. The multi-objective problem is decomposed into several sub-problems by the Tchebycheff approach. A trade-off can be obtained by optimizing these sub-problems simultaneously. In the fuzzy clustering segmentation stage, fuzzy clustering with the trade-off between preserving image details and restraining noise is performed on the whole observed image. To deal with this fuzzy clustering problem, an adaptive evolutionary fuzzy clustering algorithm with spatial information is proposed. Experiment results on synthetic and real images illustrate the effectiveness of the proposed framework for noisy image segmentation.

Modified fuzzy clustering with segregated cluster centroids

In this paper, we firstly propose a novel strategy to fuse k-means and fuzzy c-means objective functions from a multi-objective problem into a unified model, such that extreme difficulty of choosing the optimal level of the cluster fuzziness can be avoided (level of the cluster fuzziness vanishes). Accordingly, fused k-means and fuzzy c-means clustering (FKMFCM) problem is achieved with a brand new form. Based on the proposed FKMFCM, novel degrees of membership could be further derived with the closed forms. Additionally, a penalty problem is further introduced to the proposed FKMFCM, such that the fuzzy cluster centroids are modified to be more segregated from each other. Consequently, FKMFCM with the modified cluster centroid (FKMFCM-MCC) problem is represented as a bi-objective optimization for the efficient clustering. To address the proposed FKMFCM-MCC problem, an original characteristic function is introduced, such that the corresponding algorithm converges to the global optimum of the proposed FKMFCM-MCC problem. Eventually, theoretical analysis along with the empirical result is provided to validate the effectiveness of the proposed FKMFCM-MCC approach.