Evidential Clustering Research Articles

Enhancements of evidential [formula omitted]-means algorithms: A clustering framework via feature-weight learning

As a core paradigm of the evidential clustering algorithm, evidential c-means (ECM) offers a more flexible credal partition to characterize uncertainty and imprecision in cluster assignment, which generalizes hard and fuzzy partitions. Nonetheless, ECM and its derivatives equally consider the importance of each feature in clustering, which can lead to suboptimal clustering performance, especially in high-dimensional datasets where some features are more informative than others. In this paper, we propose a series of new evidential clustering algorithms by integrating feature-weight learning to simultaneously characterize uncertainty and imprecision in cluster assignment and automatically learn the different importance of distinct features in clustering. We consider ECM with vector feature-weight learning and ECM with matrix feature-weight learning, and introduce two different constraints on feature weights, respectively. We develop four objective functions and derive the corresponding optimization process. Extensive experimental results using synthetic and real-world datasets demonstrate the feasibility and effectiveness of the proposed algorithms.

Adaptive weighted multi-view evidential clustering with feature preference

Multi-view clustering has attracted substantial attention thanks to its ability to integrate information from diverse views. However, the existing methods can only generate hard or fuzzy partitions, which cannot effectively represent the uncertainty and imprecision when facing objects in overlapping clusters, thus increasing the risk of error. To solve the above problems, in this paper, we propose an adaptive weighted multi-view evidential clustering (WMVEC) method based on the theory of belief functions to characterize the uncertainty and imprecision in cluster assignment. Technically, we integrate view weight assignments and credal partition between objects and cluster prototypes into a joint learning framework. The credal partition offers a more comprehensive insight into the data by enabling objects to be associated with not only singleton clusters but also subsets of these clusters (termed meta-clusters) and the empty set, which represents a noise cluster. To avoid the interference of irrelevant and redundant features, we further present a weighted multi-view evidential clustering with feature preference (WMVEC-FP) to learn the importance of each feature under different views. We suggest the objective functions of WMVEC and WMVEC-FP and design alternating optimization schemes to obtain the optimal solutions, respectively. Through an extensive array of experiments, it has been demonstrated that our proposed clustering methods outperform other related and state-of-the-art methods in terms of their advantages and overall effectiveness.

A GMDA clustering algorithm based on evidential reasoning architecture

The traditional clustering algorithm is difficult to deal with the identification and division of uncertain objects distributed in the overlapping region, and aimed at solving this problem, the Evidential Clustering based on General Mixture Decomposition Algorithm (GMDA-EC) is proposed. First, the belief classification of target cluster is carried out, and the sample category of target distribution overlapping region is extended. Then, on the basis of General Mixture Decomposition Algorithm (GMDA) clustering, the fusion model of evidence credibility and evidence relative entropy is constructed to generate the basic probability assignment of the target and achieve the belief division of the target. Finally, the performance of the algorithm is verified by the synthetic dataset and the measured dataset. The experimental results show that the algorithm can reflect the uncertainty of target clustering results more comprehensively than the traditional probabilistic partition clustering algorithm.

Self-filling evidential clustering for partial multi-view data

Partial multi-view clustering (PMVC) is a hot topic in data mining, where each view suffers from the absence of some data. Although many PMVC algorithms have been presented with appropriate performance, to the best of our knowledge, almost all of them need to be fed with the number of clusters as hyperparameters. Moreover, these existing algorithms only create hard partitions for objects with multiple views, which are often in highly overlapping areas. This ignores ambiguity and uncertainty in object assignment, which likely leads to performance degradation. To address these issues, we propose a novel self-filling evidential clustering algorithm (SFPMEC) for the PMVC problem. Based on the shared coefficient matrix learned from a self-filling process, SFPMEC provides a human-readable chart through which the number of clusters can be easily detected. SFPMEC then derives a credal partition under the framework of evidence theory, improving the fault tolerance of clustering and gaining deeper insight into the data structure. Extensive experiments are conducted to demonstrate the benefits of adopting the self-filling process and evidence theory. Besides, our algorithm shows better performance than other state-of-the-art methods on benchmark datasets11The source code can be found in https://drive.google.com/file/d/1KMi0N8cX85YhwiNjetBwA6QeolQP5TnC/view?usp=sharing..

DTEC: Decision tree-based evidential clustering for interpretable partition of uncertain data

Recently, the evidential clustering has been developed as a promising clustering framework for uncertain data, which generalizes those hard, fuzzy, possibilistic and rough clustering. However, the resulting cluster assignments are less interpretable in terms of human cognition, which limits its applications in those security, privacy or ethic related fields. In this study, the unsupervised decision tree model is introduced into the evidential clustering framework to improve the interpretability of the evidential partition. A Decision Tree-based Evidential Clustering (DTEC) algorithm is developed to build an unsupervised evidential decision tree, which uses the paths from the root node to leaf nodes to achieve the interpretability of each cluster. The proposed algorithm is composed of three procedures, i.e., cutting-point selection, node evidential splitting, and cluster adjustment, in which the first two procedures are carried out iteratively to build a preliminary unsupervised decision tree and the last procedure is designed to adjust the preliminary decision tree if the number of clusters is available. Both synthetic and real datasets are used to evaluate the performance of the proposed algorithm, and the experimental results demonstrate the good performance of the proposal compared with some representative fuzzy, evidential or decision tree-based clustering algorithms.

BGC: Belief gravitational clustering approach and its application in the counter-deception of belief functions

Counter-deception information fusion is a significant issue in Dempster–Shafer evidence theory (DST). How to effectively counter the deception is the key problem in belief function. Limited work has been presented, of which the negation view, degree of falsity view, and the minimum conflict view are popular ones. However, previous work may suffer from combinatorial explosion or be limited to simple practical application. Based on our previous belief universal gravitation (BUG) model, a simple belief gravitational clustering (BGC) is proposed to model the evidential clustering process. Some goals, like no parameter adjustment, no initial condition, objective and unique cluster number, and robustness are achieved. Furthermore, BGC-based fusion strategy is ulteriorly raised to determine whether the evidence should be fused, which fully considers the abnormal feature of the body of evidence and the nature of the combination rule to perceive the deception. Both theoretical analysis and experimental results demonstrate the accuracy and flexibility of the proposed work.

BSC: Belief Shift Clustering

It is still a challenging problem to characterize uncertainty and imprecision between specific (singleton) clusters with arbitrary shapes and sizes. In order to solve such a problem, we propose a belief shift clustering (BSC) method for dealing with object data. The BSC method is considered as the evidential version of mean shift or mode seeking under the theory of belief functions. First, a new notion, called belief shift, is provided to preliminarily assign each query object as the noise, precise, or imprecise one. Second, a new evidential clustering rule is designed to partial credal redistribution for each imprecise object. To avoid the “uniform effect” and useless calculations, a specific dynamic framework with simulated cluster centers is established to reassign each imprecise object to a singleton cluster or related meta-cluster. Once an object is assigned to a meta-cluster, this object may be in the overlapping or intermediate areas of different singleton clusters. Consequently, the BSC can reasonably characterize the uncertainty and imprecision between singleton clusters. The effectiveness has been verified on several artificial, natural, and image segmentation/classification datasets by comparison with other related methods.

Evidential prototype-based clustering based on transfer learning

In some real clustering tasks, the data may be sparse and uncertain. Although there is usually some useful knowledge in related scenes, the data among different domains is often of great inconsistency. A new unsupervised transfer learning method is proposed in the framework of belief functions to handle the insufficiency and uncertain problems in clustering simultaneously. Firstly, under the assumption that the source and target domains have the same number of clusters, the Transfer Evidential C Means (TECM) is developed by incorporating the idea of transfer learning and evidential clustering. A novel objective function is designed to employ the cluster prototypes of the source data as references to guide the clustering process on the target. Furthermore, ETECM, as an extended version of TECM, is also introduced for the situation that the two domains have different numbers of clusters. Some experiments conducted on synthetic and real-world data sets demonstrate the advantages of TECM and ETECM.

TECM: Transfer learning-based evidential c-means clustering

As a representative evidential clustering algorithm, evidential c-means (ECM) provides a deeper insight into the data by allowing an object to belong not only to a single class, but also to any subset of a collection of classes, which generalizes the hard, fuzzy, possibilistic, and rough partitions. However, compared with other partition-based algorithms, ECM must estimate numerous additional parameters, and thus insufficient or contaminated data will have a greater influence on its clustering performance. To solve this problem, in this study, a transfer learning-based ECM (TECM) algorithm is proposed by introducing the strategy of transfer learning into the process of evidential clustering. The TECM objective function is constructed by integrating the knowledge learned from the source domain with the data in the target domain to cluster the target data. Subsequently, an alternate optimization scheme is developed to solve the constraint objective function of the TECM algorithm. The proposed TECM algorithm is applicable to cases where the source and target domains have the same or different numbers of clusters. A series of experiments were conducted on both synthetic and real datasets, and the experimental results demonstrated the effectiveness of the proposed TECM algorithm compared to ECM and other representative multitask or transfer-clustering algorithms.

Distributed evidential clustering toward time series with big data issue

To analyze time series data with large volume, most of the existing clustering algorithms focus on data reduction techniques or multi-level strategies. However, the destruction of raw data structure is inevitable, leading to the information loss, even an abnormal and unaccountable clustering result. To tackle above issues, we propose a distributed evidential clustering algorithm that can be directly adopted on the raw big data, and parallelize it under the Apache Spark, which is a processing engine built for sophisticated data analysis. Concretely, in a parallel way, the possibility of becoming a cluster center is first calculated for each data object under the framework of evidence theory. After drawing a decision graph, the cluster centers are determined and then a credal partition of the time series data is derived. Without simplifying the data structure, the proposed algorithm can not only detect the number of clusters, but also describe the ambiguity and uncertainty in memberships of every data object. Experiments on several benchmark datasets and one real-world problem show the strong scalability and well clustering performance of the introduced algorithm.

NN-EVCLUS: Neural network-based evidential clustering

Evidential clustering is an approach to clustering based on the use of Dempster-Shafer mass functions to represent cluster-membership uncertainty. In this paper, we introduce a neural-network based evidential clustering algorithm, called NN-EVCLUS, which learns a mapping from attribute vectors to mass functions, in such a way that more similar inputs are mapped to output mass functions with a lower degree of conflict. The neural network can be paired with a one-class support vector machine to make it robust to outliers and allow for novelty detection. The network is trained to minimize the discrepancy between dissimilarities and degrees of conflict for all or some object pairs. Additional terms can be added to the loss function to account for pairwise constraints or labeled data, which can also be used to adapt the metric. Comparative experiments show the superiority of N-EVCLUS over state-of-the-art evidential clustering algorithms for a range of unsupervised and constrained clustering tasks involving both attribute and dissimilarity data.

Fast semi-supervised evidential clustering

Semi-supervised clustering is a constrained clustering technique that organizes a collection of unlabeled data into homogeneous subgroups with the help of domain knowledge expressed as constraints. These methods are, most of the time, variants of the popular k-means clustering algorithm. As such, they are based on a criterion to minimize. Amongst existing semi-supervised clusterings, Semi-supervised Evidential Clustering (SECM) deals with the problem of uncertain/imprecise labels and creates a credal partition. In this work, a new heuristic algorithm, called SECM-h, is presented. The proposed algorithm relaxes the constraints of SECM in such a way that the optimization problem is solved using the Lagrangian method. Experimental results show that the proposed algorithm largely improves execution time while accuracy is maintained.

Dynamic evidential clustering algorithm

In this paper, a dynamic evidential clustering algorithm (DEC) is introduced to address the computational burden of existing methods. To derive such a solution, an FCM-like objective function is first employed and minimized to obtain the support levels of the real singletons (specific) clusters to which the query objects belong, and then the query objects is initially adaptively assigned to outlier, precise or imprecise one via a new rule-based on the conflicts between the different support levels. For each imprecise object, it is finally reassigned to the singleton clusters or related meta-cluster by partial credal redistribution with the corresponding dynamic edited framework to reduce the computational burden. The proposed method can reduce the complexity to the level similar to that of the fuzzy and possibilistic clustering, which can effectively extend the application of evidential clustering, especially in big data. The effectiveness of the DEC method is tested by four experiments with artificial and real datasets.

An evidential clustering algorithm by finding belief-peaks and disjoint neighborhoods

In this paper, we introduce a new evidential clustering algorithm based on finding the belief-peaks and disjoint neighborhoods, called BPDNEC. The basic idea of BPDNEC is that each cluster center has the highest possibility of becoming a cluster center among its neighborhood and neighborhoods of those cluster centers are disjoint in vector space. Such possibility is measured by the belief notion in framework of evidence theory. By solving an equation related to neighborhood size, the size of such disjoint neighborhoods is determined and those objects having highest belief among their neighborhoods are automatically detected as cluster centers. Finally, a credal partition is created by minimizing an objective function concerning dissimilarity matrix of data objects. Experimental results show that BPDNEC can automatically detect cluster centers and derive an appropriate credal partition for both object data and proximity data. Simulations on synthetic and real-world datasets validate the conclusions.

Cumulative belief peaks evidential K-nearest neighbor clustering

This paper introduces a new evidential clustering algorithm based on finding the “cumulative belief peaks” and evidential K-nearest neighbor rule. The basic assumption of this algorithm is that a cluster center has the highest cumulative possibility of becoming a cluster center among its neighborhood and size of its neighborhood is relatively large. To measure such cumulative possibility, a new notion of cumulative belief is proposed in the framework of belief functions. By maximizing an objective function, an appropriate size of the relatively large neighborhood is determined. Then, the objects with highest cumulative belief among their own neighborhood of this size are automatically detected as cluster centers. Finally, a credal partition is derived by evidential K-nearest neighbor rule with the fixed cluster center. Experimental results show that the proposed evidential clustering algorithm can automatically detect cluster centers and well reveal the data structure in form of a credal partition in tolerable time, when tackling datasets with small number of data objects and dimensions. As the sizes of datasets increase, running time of such new clustering algorithm increases sharply and this reduces the practicability of it. Simulations on synthetic and real-world datasets validate our conclusions.

Calibrated model-based evidential clustering using bootstrapping

Evidential clustering is an approach to clustering in which cluster-membership uncertainty is represented by a collection of Dempster-Shafer mass functions forming an evidential partition. In this paper, we propose to construct these mass functions by bootstrapping finite mixture models. In the first step, we compute bootstrap percentile confidence intervals for all pairwise probabilities (the probabilities for any two objects to belong to the same class). We then construct an evidential partition such that the pairwise belief and plausibility degrees approximate the bounds of the confidence intervals. This evidential partition is calibrated, in the sense that the pairwise belief-plausibility intervals contain the true probabilities “most of the time”, i.e., with a probability close to the defined confidence level. This frequentist property is verified by simulation, and the practical applicability of the method is demonstrated using several real datasets.

Combining clusterings in the belief function framework

Abstract In this paper, we propose a clustering ensemble method based on Dempster-Shafer Theory. In the first step, base partitions are generated by evidential clustering algorithms such as the evidential c-means or EVCLUS. Base credal partitions are then converted to their relational representations, which are combined by averaging. The combined relational representation is then made transitive using the theory of intuitionistic fuzzy relations. Finally, the consensus solution is obtained by minimizing an error function. Experiments with simulated and real datasets show the good performances of this method.

Spatial Regularization and Adaptive Distance Metric Methods through DST for Tumor Segmentation

The process of segmentation in MRI pictures is turning into a significant assignment to be considered in clinical oncology applications, in view of the noise and blur that is available normally in MRI images. To minimize this natural disadvantages in the MRI images an imaging tool called belief theory is taken as the base alongside the proposed evidential clustering algorithm (ECM-MS) .This proposed technique joins the adaptive distance metric in so as to limit the clustering distortions and the comparability that happen between the voxels. The local homogeneity is measured by the spatial regularization dependent on the belief theory called Dempster Shafer Theory (DST). To get definite division the surface highlights are extricated from the data picture and is incorporated with the force of the voxels in the proposed strategy, thusly giving a decent presentation contrasted with different strategies.

Compact Belief Rule Base Learning for Classification with Evidential Clustering.

The belief rule-based classification system (BRBCS) is a promising technique for addressing different types of uncertainty in complex classification problems, by introducing the belief function theory into the classical fuzzy rule-based classification system. However, in the BRBCS, high numbers of instances and features generally induce a belief rule base (BRB) with large size, which degrades the interpretability of the classification model for big data sets. In this paper, a BRB learning method based on the evidential C-means clustering (ECM) algorithm is proposed to efficiently design a compact belief rule-based classification system (CBRBCS). First, a supervised version of the ECM algorithm is designed by means of weighted product-space clustering to partition the training set with the goals of obtaining both good inter-cluster separability and inner-cluster pureness. Then, a systematic method is developed to construct belief rules based on the obtained credal partitions. Finally, an evidential partition entropy-based optimization procedure is designed to get a compact BRB with a better trade-off between accuracy and interpretability. The key benefit of the proposed CBRBCS is that it can provide a more interpretable classification model on the premise of comparative accuracy. Experiments based on synthetic and real data sets have been conducted to evaluate the classification accuracy and interpretability of the proposal.

BPEC: Belief-Peaks Evidential Clustering

This paper introduces a new evidential clustering method based on the notion of “belief peaks” in the framework of belief functions. The basic idea is that all data objects in the neighborhood of each sample provide pieces of evidence that induce belief on the possibility of such sample to become a cluster center. A sample having higher belief than its neighbors and located far away from the other local maxima is then characterized as cluster center. Finally, a credal partition is created by minimizing an objective function with the fixed cluster centers. An adaptive distance metric is used to fit for unknown shapes of data structures. We show that the proposed evidential clustering procedure has very good performance with an ability to reveal the data structure in the form of a credal partition, from which hard, fuzzy, possibilistic, and rough partitions can be derived. Simulations on synthetic and real-world datasets validate our conclusions.