Cluster Validity Functions Research Articles

Safe semi-supervised clustering based on Dempster–Shafer evidence theory

In this paper, we propose a safe semi-supervised clustering algorithm based on Dempster–Shafer (D–S) evidence theory. The motivation is that D–S evidence theory can be used to fuse multiple base clustering results and obtain robust confidence estimations of mislabeled samples. Firstly, the proposed algorithm constructs multiple base clusters using fuzzy c-means and kernel fuzzy c-means. Base clusters with good performance are selected according to a clustering validity function. Then, D–S evidence theory is used to fuse the results of the selected base clusters, and the confidence of labeled samples is calculated based on the fused results. Finally, we construct a p-nearest neighbor graph to limit the outputs of labeled samples with low confidence to be those of the p nearest unlabeled samples. It is desired to reduce the negative influence of labeled samples with low confidence and achieve safe exploitation. To verify the effectiveness of the proposed algorithm, we compare it to several unsupervised and semi-supervised clustering algorithms. The results demonstrate that our algorithm yields higher accuracy and is more stable.

Component-wise design method of fuzzy C-means clustering validity function based on CRITIC combination weighting

Component-wise design method of fuzzy C-means clustering validity function based on CRITIC combination weighting

Data Processing Method of Mine Wind Speed Monitoring Based on an Improved Fuzzy C-Means Clustering Algorithm

Analyzing and processing mine wind speed monitoring data is the key to realizing intelligent ventilation and real-time calculation of the ventilation network. According to the characteristics of the artificial regulation of a mine ventilation system, a local regression fuzzy C clustering algorithm is proposed in this paper, which combines local outlier processing with global air volume state analysis. Firstly, the algorithm uses the robust local weighted regression principle to analyze and preprocess the data locally, determines the risk degree of the abnormal data according to the identified times of outliers, determines the clustering number according to the clustering validity function, and analyzes the global air volume fluctuation according to the clustering results. The results show that most outliers are identified in data preprocessing. Still, the processing of dense outliers is weak, related to the window width setting and weighting multiple. The number of clusters can represent the fluctuation of the ventilation state and the pre-processed cluster centers are 4.4% lower than the original data because most of the outliers are higher than the average data. According to the law of air volume balance, the clustering results can pave the way for the global deduction of mine wind speed. There is an implicit relationship between data preprocessing and the clustering process, and when intensive outliers are not eliminated, they may be identified as separate clusters. The research of this paper points out the direction of mine wind speed data analysis, which can provide a theoretical basis for intelligent mine ventilation and real-time calculation of the ventilation network.

Exponent and Logarithm Component-Wise Construction Method of FCM Clustering Validity Function Based on Subjective and Objective Weighting

Exponent and Logarithm Component-Wise Construction Method of FCM Clustering Validity Function Based on Subjective and Objective Weighting

Ratio component-wise design method of fuzzy c-means clustering validity function

Fuzzy clustering is an important research field in pattern recognition, machine learning and image processing. The fuzzy C-means (FCM) clustering algorithm is one of the most common fuzzy clustering algorithms. However, it requires a given number of clusters in advance for accurate clustering of data sets, so it is necessary to put forward a better clustering validity index to verify the clustering results. This paper presents a ratio component-wise design method of clustering validity function based on FCM clustering method. By permutation and combination of six clustering validity components representing different meanings in the form of ratio, 49 different clustering validity functions are formed. Then, these functions are verified experimentally under six kinds of UCI data sets, and a clustering validity function with the simplest structure and the best classification effect is selected by comparison. Finally, this function is compared with seven traditional clustering validity functions on eight UCI data sets. The simulation results show that the proposed validity function can better verify the classification results and determine the optimal clustering number of different data sets.

Multi-Target Magnetic Positioning Using SAFCM Clustering and Invariants-Improved Tilt Angle

The positioning of multiple magnetic sources with different depths, distributions and magnetic moments is extremely difficult. The improved tilt angle for target area recognition can be well adapted to the self-adaptive fuzzy c-means (SAFCM) clustering algorithm for grid measurement of magnetic gradient tensors (MGT), which provides a feasible solution to solve the complex problem of multi-target inversion. First, we improve the tilt angle with the normalized source strengths and the tensor contraction, which can accurately identify the two-dimensional area boundary of objects with different buried depths regardless of the magnetization direction. The grid measurements of magnetic gradient tensor systems (MGTS) enable this process. Then, with the tensor spacial invariant relations, we calculate the source initial position coordinates corresponding to each node in the recognition area, thereby forming a dense point cloud in the space near the target real position. Finally, we introduce a penalty function into the clustering validity function to eliminate the monotonous decreasing trend of the cluster number optimization process, thus the SAFCM algorithm can accurately and automatically detect the number and position of centroids. The simulation shows that with the Gaussian noise of 0 mean and 5 nT/m variance, the targets number recognition accuracy of 20 magnetic dipoles is 97.6%, and the horizontal position and buried depth estimation accuracy is greater than 91.7% and 85.6% respectively. In the real survey areas of 2.1m×2.1m and 1.2m×1.2m, the coordinates’ estimation deviation of nine small magnets is less than 0.091m.

Enhanced Image Segmentation: Merging Fuzzy K-Means and Fuzzy C-Means Clustering Algorithms for Medical Applications

Separating an image into regions according to some criterion is called image segmentation. This paper presents an algorithm that combines the fuzzy k-means (FKM) and fuzzy c-means (FCM) clustering strategies. The proposed algorithm combines the FKM and the FCM algorithms mathematical features, which is referred to as (CFKCM). The FKM and FCM clustering algorithms are adopted to compare the performance and hence evaluate the proposed clustering algorithm. Tests are conducted, and performance parameters are calculated for validation. The comparison and assessment analysis are based on metrics related to the image clustering process, such as the Segmentation Accuracy (SA), Clustering Fitness (CF), and cluster Validity function (V_pc and V_pe). A dataset of MR images is used by this research for the application, test, and evaluation of the image clustering. The results for clustering backbone MRI images show that CFKCM algorithm is featured by being more effective, and comparatively independent of the noise, where it can process the

Using fuzzy C-means Clustering to Identify Heavy Metal Polluted Soil in a Certain Area of Shanghai

Fuzzy clustering is an important branch of fuzzy pattern recognition, which has been widely used in many fields such as data mining, image processing, big data analysis and so on. Among many fuzzy clustering algorithms, fuzzy c-means algorithm is the most widely used. In this paper, the fuzzy c-means clustering is applied to the identification of polluted soil. To solve the problem of determining the optimal number of clusters for this method, the method we choose is obtaining an initial clustering result firstly and then merging. Based on the important characteristic of fuzzy c-means method that the objective function of fuzzy c-means clustering decreases rapidly as the number of clusters increases, and the rate become slow after exceeding the optimal number of clusters, we choose the clustering result whose objective function is reduced to a certain degree as the initial clustering result. Then use the parameter estimation method in statistics to estimate the distance between classes, determine the reasonable range of distance between classes, combine the initial classification results, and finally perform the final clustering results according to the clustering validity function. Evaluation and experiments prove the feasibility and effectiveness of the scheme.

Combination Evaluation Method of Fuzzy C-Mean Clustering Validity Based on Hybrid Weighted Strategy

Clustering validity function is an index used to judge the accuracy of clustering results. At present, most studies on clustering validity are based on single clustering validity function. Research shows that no clustering validity function can handle any data and always perform better than other indexes. Therefore, a hybrid weighted combination evaluation method based on fuzzy C-means (FCM) clustering validity functions was proposed. The weighting method combines expert weighting with information entropy weighting to improve the subjective factor influence of expert weighting and the shortcoming of information entropy weighting in the value judgment of each clustering validity function. Four clustering validity function combination methods of linear, exponential, logarithm and proportion was studied. Finally, the proposed fuzzy clustering validity evaluation method is verified by experiments on artificial data sets and UCI data sets. The experimental results show that the proposed fuzzy clustering validity evaluation method can overcome the shortcoming of single clustering validity function, and can get the optimal clustering number more accurately for different data sets.

A new validity function of FCM clustering algorithm based on intra-class compactness and inter-class separation

Fuzzy C-means (FCM) clustering algorithm is a widely used method in data mining. However, there is a big limitation that the predefined number of clustering must be given. So it is very important to find an optimal number of clusters. Therefore, a new validity function of FCM clustering algorithm is proposed to verify the validity of the clustering results. This function is defined based on the intra-class compactness and inter-class separation from the fuzzy membership matrix, the data similarity between classes and the geometric structure of the data set, whose minimum value represents the optimal clustering partition result. The proposed clustering validity function and seven traditional clustering validity functions are experimentally verified on four artificial data sets and six UCI data sets. The simulation results show that the proposed validity function can obtain the optimal clustering number of the data set more accurately, and can still find the more accurate clustering number under the condition of changing the fuzzy weighted index, which has strong adaptability and robustness.

Dynamic Management Zones for Irrigation Scheduling

Irrigation scheduling decision-support tools can improve water use efficiency by matching irrigation recommendations to prevailing soil and crop conditions within a season. Yet, little research is available on how to support real-time precision irrigation that varies within-season in both time and space. We investigate the integration of remotely sensed NDVI time-series, soil moisture sensor measurements, and root zone simulation forecasts for in-season delineation of dynamic management zones (MZ) and for a variable rate irrigation scheduling in order to improve irrigation scheduling and crop performance. Delineation of MZ was conducted in a 5.8-ha maize field during 2018 using Sentinel-2 NDVI time-series and an unsupervised classification. The number and spatial extent of MZs changed through the growing season. A network of soil moisture sensors was used to interpret spatiotemporal changes of the NDVI. Soil water content was a significant contributor to changes in crop vigor across MZs through the growing season. Real-time cluster validity function analysis provided in-season evaluation of the MZ design. For example, the total within-MZ daily soil moisture relative variance decreased from 85% (early vegetative stages) to below 25% (late reproductive stages). Finally, using the Hydrus-1D model, a workflow for in-season optimization of irrigation scheduling and water delivery management was tested. Data simulations indicated that crop transpiration could be optimized while reducing water applications between 11 and 28.5% across the dynamic MZs. The proposed integration of spatiotemporal crop and soil moisture data can be used to support management decisions to effectively control outputs of crop × environment × management interactions.

DELIMITATION OF HOMOGENEOUS ZONES IN VINEYARDS USING GEOSTATISTICS AND MULTIVARIATE ANALYSIS OF DIFFERENT VEGETATION INDICES

Vegetation indices (VIs) are quantitative measures used to describe the distribution and spatial variability of the vegetation cover of natural or cultivated areas. The aim of this study was to delimit homogeneous zones (HZs) of different VIs using geostatistics and multivariate analysis in order to identify vegetation patterns in Cabernet Franc and Cabernet Sauvignon vineyards. The evaluation was performed in two vineyards in the municipality of Espírito Santo do Pinhal in the state of São Paulo, Brazil. Reflectance (ρ) was measured at three wavelengths of the electromagnetic spectrum (670, 730, and 780 nm) at canopy height in georeferenced points along planting rows using the Crop Circle ACS-430 active sensor. Nine VIs were calculated based on the ratios between the ρ values. Geostatistical data analysis allowed the spatial prediction of VIs by ordinary kriging interpolation. Principal component analysis and fuzzy k-means clustering were applied for HZs delimitation and the optimal number of zones was defined according to cluster validity functions. Despite the variations of the VIs spatial distribution patterns, the multivariate analysis resulted in a representative categorization of the grapevine vegetative vigor and delimitation of HZs for this characteristic. This was validated according to the observed significant differences between VIs.

Hydrological characteristic-based methodology for dividing flood seasons: an empirical analysis from China

A hydrological characteristic -based methodology for dividing flood seasons into sub-seasons is proposed to make full use of a reservoir’s flood control storage during the flood season via multi-stage flood-limited water levels. The proposed framework is mainly composed of three parts: the selection of indices depicting flood seasonality, the establishment of segmentation methods that can address clustering problems with high-dimensional time series and unknown numbers of clusters, and multi-scheme comparison and rationality analysis. The reasonability and validity of the proposed framework is illustrated through an empirical case study of China’s Panjiakou Reservoir basin. The results indicated that the dynamic fuzzy c-means method with clustering validity function provided more objective and quantitative divisions than other methods, including the Fisher optimal partition. The flood season of the Panjiakou Reservoir basin (June 1–September 30) is divided into three sub-seasons according to the principle of optimal clustering: a pre-flood season (June 1–July 10), a main flood season (July 11–August 20), and a post-flood season(August 21–September 30).

A Self-Adaptive FCM for the Optimal Fuzzy Weighting Exponent

Fuzzy weighting exponent [Formula: see text] is an important parameter of fuzzy [Formula: see text]-means (FCM), closely related to the performance of the algorithm. First, an improved fuzzy correlation degree was put forward to measure the relevance between the clusters, based on which a new cluster validity function was defined to evaluate the quality of the fuzzy partition. Then a self-adaptive FCM for the optimal value of [Formula: see text] was proposed with the aid of the global search ability of improved particle swarm algorithm to find out both the final clustering centroids and the optimal value of fuzzy weighting exponent automatically. The improved particle swarm algorithm updated the speed and the position based on dynamic inertia weight and learning factors, and introduced mutation of genetic algorithm to keep the diversity of the particles, preventing premature convergence. The experimental results showed that the proposed algorithm automatically calculated the optimal value of [Formula: see text] and meanwhile achieved better clustering results.

Regional flood risk assessment via coupled fuzzy c-means clustering methods: an empirical analysis from China’s Huaihe River Basin

This study contributed to the comprehensive assessment of flood risk in the Huaihongnanpian flood control protected area (simplified as the HHNP) of the Huaihe River Basin in China. Flood risk analyses were performed by incorporating flood hazard and vulnerability. Flood hazard was simulated by a 1D–2D coupled hydrodynamic model. Flow velocity, inundation duration, and inundation depth were taken as hazard indicators, while agricultural population proportions, female population proportions, GDP per unit area, GDP per person, population density, residential density, shelter density, the land-use sensitivity index, road network density, and river network density were used as vulnerability indicators. Based on these indicators, a regional flood risk assessment model was put forward, which coupled fuzzy c-means clustering, factor analysis, and a clustering validity function. As an example, a proposed model was applied to evaluate the degree of flood risk for 15 townships in the HHNP. The research results showed that (1) flood risk in the HHNP was closely related to three main factors: socioeconomic factor, land cover factor, and flood factor; (2) the degree of risk was objectively divided into six zones: especially high, high, relatively high, medium, relatively low and low; and (3) in the 15 townships, Xiaobengbu (XB), Chengguan (CG), and Wuxiaojie (WX) fell into the especially high, high, and relatively high zones, respectively. Xinji (XJ), Toupu (TP), Daxin (DX), Caoguzhang (CGZ), Meiqiao (MQ), Caolaoji (CL), and Mohekou (MH) fell into the medium-risk zone. Linbeihuizu (LB) was categorized into the relatively low-risk zone, and Xinmaqiao (XM), Wangzhuang (WZ), Kuainan (KN), and Weizhuang (WZ) fell into the low-risk zone. The research results revealed the main driving factors and the spatial distribution of flood risk in the HHNP; therefore, it is highly significant for us to understand the main flood risk sources to provide guidance for flood control and management in the HHNP.

Smallest Univalue Segment Assimilating Nucleus approach to Brain MRI Image Segmentation using Fuzzy C-Means and Fuzzy K-Means Algorithms

Image segmentation still remains an important task in image processing and analysis. Sequel to any segmentation process, preprocessing activities carried out on the images have a great effect on the accuracy of the segmentation task. This paper therefore laid emphasis on the preprocessing stage of brain Magnetic Resonance Imaging (MRI) images Smallest Univalue Segment Assimilating Nucleus (SUSAN) and bias field correction algorithms. Subsequently, brain tissue extraction tool was employed in extracting non-brain tissues from the brain image. Afterwards, Fuzzy K-Means (FKM) and Fuzzy C-Means (FCM) segmentation algorithms were employed for segmenting brain MRI images acquired from four different MRI databases into their White Matter (WM), Gray Matter (GM) and Cerebrospinal Fluid (CSF) constituents. Evaluation metrics such as cluster validity functions using partition coefficients and partition entropy; area error metrics such as false positive, true positive, true negative and false negative (FN); similarity index, sensitivity and specificity were used to evaluate the performance of both techniques. A comparative analysis of the experimental results revealed that in most instances, FKM segmentation technique is preferable to FCM segmentation technique for brain MRI segmentation task.

Many-objective fuzzy centroids clustering algorithm for categorical data

Categorical data clustering algorithms, in contrast to numerical ones, are still in their infancy despite some algorithms have been proposed in the literature. It is known that many clustering algorithms are posed as optimization problems, where internal cluster validity functions are utilized as the objectives to find the optimal partitions. However, most of these methods consider a single criterion that can merely be applied to detect the particular structure/distribution of data. To overcome this issue, in this paper, a novel many objective fuzzy centroids clustering algorithms is proposed for categorical data using reference point based non-dominated sorting genetic algorithm, which simultaneously optimizes several cluster validity indices. In our work, an effective fuzzy centroids algorithm is employed to design the proposed approach, which is different from other contestant k-modes-type methods. Here, the fuzzy memberships are used for chromosome representation that combines with a novel genetic operation to produce new solutions. Moreover, a variable-length encoding scheme is developed for the sake of finding the clusters without knowing any prior knowledge. Experiments on several data sets demonstrate the superiority of the proposed algorithm over other state-of-the-art methods in terms of clustering accuracy and stability. On the other hand, our method can detect the cluster number if not predefined along with a desirable clustering solution.

A Modified Intuitionistic Fuzzy Clustering Algorithm for Medical Image Segmentation

Abstract This paper presents a modified intuitionistic fuzzy clustering (IFCM) algorithm for medical image segmentation. IFCM is a variant of the conventional fuzzy C-means (FCM) based on intuitionistic fuzzy set (IFS) theory. Unlike FCM, IFCM considers both membership and nonmembership values. The existing IFCM method uses Sugeno’s and Yager’s IFS generators to compute nonmembership value. But for certain parameters, IFS constructed using above complement generators does not satisfy the elementary condition of intuitionism. To overcome this problem, this paper adopts a new IFS generator. Further, Hausdorff distance is used as distance metric to calculate the distance between cluster center and pixel. Extensive experimentations are carried out on standard datasets like brain, lungs, liver and breast images. This paper compares the proposed method with other IFS based methods. The proposed algorithm satisfies the elementary condition of intuitionism. Further, this algorithm outperforms other methods with the use of various cluster validity functions.

An Optimization Model for Clustering Categorical Data Streams with Drifting Concepts

There is always a lack of a cluster validity function and optimization strategy to find out clusters and catch the evolution trend of cluster structures on a categorical data stream. Therefore, this paper presents an optimization model for clustering categorical data streams. In the model, a cluster validity function is proposed as the objective function to evaluate the effectiveness of the clustering model while each new input data subset is flowing. It simultaneously considers the certainty of the clustering model and the continuity with the last clustering model in the clustering process. An iterative optimization algorithm is proposed to solve an optimal solution of the objective function with some constraints. Furthermore, we strictly derive a detection index for drifting concepts from the optimization model. We propose a detection method that integrates the detection index and the optimization model to catch the evolution trend of cluster structures on a categorical data stream. The new method can effectively avoid ignoring the effect of the clustering validity on the detection result. Finally, using the experimental studies on several real data sets, we illustrate the effectiveness of the proposed algorithm in clustering categorical data streams, compared with existing data-streams clustering algorithms.

Unsupervised Binning of Metagenomic Assembled Contigs Using Improved Fuzzy C-Means Method

Metagenomic contigs binning is a necessary step of metagenome analysis. After assembly, the number of contigs belonging to different genomes is usually unequal. So a metagenomic contigs dataset is a kind of imbalanced dataset and traditional fuzzy c-means method (FCM) fails to handle it very well. In this paper, we will introduce an improved version of fuzzy c-means method (IFCM) into metagenomic contigs binning. First, tetranucleotide frequencies are calculated for every contig. Second, the number of bins is roughly estimated by the distribution of genome lengths of a complete set of non-draft sequenced microbial genomes from NCBI. Then, IFCM is used to cluster DNA contigs with the estimated result. Finally, a clustering validity function is utilized to determine the binning result. We tested this method on a synthetic and two real datasets and experimental results have showed the effectiveness of this method compared with other tools.