Proposed Uncertainty Measure Research Articles

Feature selection based on neighborhood complementary entropy for heterogeneous data

In practical applications, there exists a large amount of heterogeneous data consisting of a mixture of numerical and categorical data, which are more complex and difficult to handle than a single class. In recent years, feature selection for heterogeneous data has received more and more attention from scholars. However, the current research on feature selection for heterogeneous data seldom takes into account the fineness of the granularity structure and the complementary information contained in the complements of each granularity. Addressing the shortcomings of existing efforts, this paper introduces complementary entropy into the neighborhood rough set model to define an uncertainty measurement system in heterogeneous data, and constructs the corresponding feature selection algorithm. First, the neighborhood complementary entropy and its associated uncertainty measures based on neighborhood relation are defined, and the relevant theoretical properties of these uncertainty measures are investigated. Then, based on the proposed uncertainty measures, a significance function is defined to assess the importance of candidate features. Next, a feature selection method for heterogeneous data is designed using neighborhood complementary entropy. Finally, the feasibility of the method proposed in this paper is demonstrated through an experimental comparative analysis with various existing feature selection methods.

Multi-fuzzy [formula omitted]-covering fusion based accuracy and self-information for feature subset selection

Granular structures are mathematical representations of knowledge used in granular computing. As a new type of granular structure, fuzzy β-covering has attracted widespread attention in recent years. One of the most critical related studies is measuring the uncertainty of fuzzy β-coverings. It is the foundation for studying fuzzy β-covering applications, such as classification and clustering. In this paper, we investigate the uncertainty measures of fuzzy β-coverings from the viewpoints of algebra and information theory, and propose the corresponding methods of fuzzy β-covering reduction for feature selection. Firstly, a generalized fuzzy β-neighborhood and the associated fuzzy β-covering rough sets are presented. Secondly, three types of accuracy measures of fuzzy β-coverings are constructed via fuzzy β-covering rough sets. On this basis, the self-information of fuzzy β-coverings is defined by fusing accuracy and roughness measures. Based on the proposed uncertainty measures, two heuristic methods of fuzzy β-covering reduction are put forward and the corresponding algorithms are designed for monotonic feature selection. Finally, the performance of the proposed methods is compared with several mainstream feature selection methods and experimental results demonstrate the rationality and superiority of our proposed methods.

Sup-extropy: A measure of uncertainty

. In a recent development, a new measure of uncertainty, termed extropy, was introduced as the dual complement of Shannon entropy. This study proposes a modified extropy measure called sup-extropy. The proposed uncertainty measure is shown to be minimal value when the data follow a uniform distribution. Various fundamental properties of this measure are thoroughly examined. The joint and conditional versions of this measure are also presented. To illustrate its practical implications, highlights into its application in letter-catching experiments and information fusion analysis are provided.

Predictive uncertainty in deep learning-based MR image reconstruction using deep ensembles: Evaluation on the fastMRI data set.

To estimate pixel-wise predictive uncertainty for deep learning-based MR image reconstruction and to examine the impact of domain shifts and architecture robustness. Uncertainty prediction could provide a measure for robustness of deep learning (DL)-based MR image reconstruction from undersampled data. DL methods bear the risk of inducing reconstruction errors like in-painting of unrealistic structures or missing pathologies. These errors may be obscured by visual realism of DL reconstruction and thus remain undiscovered. Furthermore, most methods are task-agnostic and not well calibrated to domain shifts. We propose a strategy that estimates aleatoric (data) and epistemic (model) uncertainty, which entails training a deep ensemble (epistemic) with nonnegative log-likelihood (aleatoric) loss in addition to the conventional applied losses terms. The proposed procedure can be paired with any DL reconstruction, enabling investigations of their predictive uncertainties on a pixel level. Five different architectures were investigated on the fastMRI database. The impact on the examined uncertainty of in-distributional and out-of-distributional data with changes to undersampling pattern, imaging contrast, imaging orientation, anatomy, and pathology were explored. Predictive uncertainty could be captured and showed good correlation to normalized mean squared error. Uncertainty was primarily focused along the aliased anatomies and on hyperintense and hypointense regions. The proposed uncertainty measure was able to detect disease prevalence shifts. Distinct predictive uncertainty patterns were observed for changing network architectures. The proposed approach enables aleatoric and epistemic uncertainty prediction for DL-based MR reconstruction with an interpretable examination on a pixel level.

New links between invariant dynamical structures and uncertainty quantification

This paper proposes a new uncertainty measure, appropriate for quantifying the performance of transport models in assessing the origin or source of a given observation. It is found that in a neighborhood of the observation the proposed uncertainty measure is related to the invariant dynamical structures of the model. The paper illustrates the implementation of the proposed definition to quantify the performance of ocean data sets in the context of a real oil spill event in the Eastern Mediterranean in 2021.

Fuzzy rough feature selection using a robust non-linear vague quantifier for ordinal classification

Ordinal classification is a common classification problem, which widely exists in multi-attribute decision making problems. The dominance-based rough set approach (DRSA) is a knowledge acquisition tool for ordinal classification tasks. Nevertheless, the existence of noise information in the collected data greatly reduces the accuracy of the DRSA and some extended model. An effective way to solve this issue is to improve the robustness of these models. In addition, most existing feature selection strategies based on dominance rough sets mainly focus on the monotonic classification consistency between features and decision, but ignoring the classification information provided by features combination. Motivated by these two issues, this paper proposes a robust fuzzy dominance rough set model to combat noise interference and develop a feature selection approach based on the proposed model for ordinal classification tasks. First, a non-linear vague quantifier is adopted to construct the robust model, and related dependency function is introduced. Second, the rank entropy based uncertainty measures are explored to characterize the contribution of features combination for ordinal classification. Based on the proposed uncertainty measures, a new feature evaluation index is presented. Meanwhile, the corresponding feature selection algorithm is designed. Finally, numerical comparative experiments are performed to indicate that the proposed model and feature selection algorithm have good capability.

A New Reliability Coefficient Using Betting Commitment Evidence Distance in Dempster-Shafer Evidence Theory for Uncertain Information Fusion.

Dempster-Shafer evidence theory is widely used to deal with uncertain information by evidence modeling and evidence reasoning. However, if there is a high contradiction between different pieces of evidence, the Dempster combination rule may give a fusion result that violates the intuitive result. Many methods have been proposed to solve conflict evidence fusion, and it is still an open issue. This paper proposes a new reliability coefficient using betting commitment evidence distance in Dempster-Shafer evidence theory for conflict and uncertain information fusion. The single belief function for belief assignment in the initial frame of discernment is defined. After evidence preprocessing with the proposed reliability coefficient and single belief function, the evidence fusion result can be calculated with the Dempster combination rule. To evaluate the effectiveness of the proposed uncertainty measure, a new method of uncertain information fusion based on the new evidence reliability coefficient is proposed. The experimental results on UCI machine learning data sets show the availability and effectiveness of the new reliability coefficient for uncertain information processing.

Robust fuzzy rough approximations with kNN granules for semi-supervised feature selection

Fuzzy rough set theory has attracted much attention because of its successful application in uncertainty measurement. To improve the efficiency and robustness of uncertainty measure using the theory, robust fuzzy rough approximations have become popular research. In this work, it first designs a novel uncertainty measure based on kNN granules. By combining the relative distance, the method is effectively applicable to the datasets with large density variation. Using the measure a robust FRS model is studied and simply denoted KNN-FRS model. The new model takes a compromise between the maximum lower approximation degree and the minimum classification loss in computing fuzzy rough approximations, which reduces the sensitivity of the classical FRS model to noise. Besides, some properties of KNN-FRS model are summarized. Next, KNN-FRS model is used to design a feature selection algorithm which is tested to validate the feasibility of the KNN-FRS model. And then the KNN-FRS model is applied to design a semi-supervised feature selection algorithm which selects features while labeling samples and the labeling method is designed based on the proposed uncertainty measure. Some experiments are conducted to validate the KNN-FRS model and the designed algorithms. The results illustrate the proposed KNN-FRS model is feasible, effective and robust. And the designed semi-supervised feature selection algorithm performs excellent performance.

Some uncertainty measures for probabilistic hesitant fuzzy information

The probabilistic hesitant fuzzy sets (PHFSs), which have received a lot of attention and have been widely used in many domains, are effective in describing hesitant evaluations. Despite significant progress, entropy and cross-entropy, as the famous uncertainty measures of probabilistic hesitant fuzzy information, have not yet been thoroughly investigated. However, these uncertainty measures are quite important and useful in the context of multi-attribute decision-making (MADM). In this contribution, we revisit the concept of uncertainty measures for probabilistic hesitant fuzzy information by comprehensively considering their fuzziness and hesitancy, and proposed some novel entropy and cross-entropy measures for them. Specifically, the hybrid entropy measures are designed as convex combinations of the fuzzy entropy factor and the hesitant entropy factor, and in the same line of thought, the hybrid cross-entropy measures are also designed as convex combinations of the fuzzy cross-entropy factor and the hesitant cross-entropy factor. It is shown that the hybrid entropy measures are simple in structure, clear in physical meaning, and strong in discriminating capability, while the hybrid cross-entropy measures can avoid the design flaws of existing cross-entropy measures and employ natural symmetry. It is further shown that the hybrid entropy and cross-entropy measures can be designed in pairs and are mutually supportive, which suggests that they are internally unified as the uncertainty measures of probabilistic hesitant fuzzy information. In the end, based on the proposed uncertainty measures, an MADM model is developed within the TOPSIS framework to evaluate a company’s strategic initiatives under probabilistic hesitant fuzzy environment. Numerical results show that these uncertainty measures can support group decision-making effectively and efficiently with probabilistic hesitant fuzzy information.

A Modified Uncertainty Measure of Z-numbers

The Z-number is a more adequate construct for describing real-life information. While considering the uncertainty of the information, it also models the partial reliability of the information. It is a combination of probabilistric restriction and possibilistric restriction. In this paper, we modified the uncertainty measurement of the discrete Z-number and proposed the uncertainty measurement of the continuous Z-number. Some numerical examples are used to illustrate the calculation processes and advantages of the proposed method. An application of journey vehicle selection shows the effectiveness of the proposed uncertainty measurement in determining the weights of criteria.

Exploring interactive attribute reduction via fuzzy complementary entropy for unlabeled mixed data

Attribute reduction is one of the important applications in fuzzy rough set theory. However, most attribute reduction methods in fuzzy rough theory mainly focus on removing irrelevant or redundant attributes. There are few reports about the method of considering attribute interaction. For this reason, this paper proposes an interactive attribute reduction method for unlabeled mixed data. First, some uncertainty measures based on fuzzy complementary entropy are further defined. Then, based on the proposed uncertainty measure, the attribute evaluation criteria of maximal information, minimal redundancy, and maximal interactivity are developed respectively. As a result, the evaluation index of the attribute importance is established by using the idea of unsupervised maximal information-minimal redundancy-maximal interactivity. Finally, a corresponding algorithm is designed to select attributes. The experimental results show that the proposed algorithm has better performance.

A data-level fusion model for unsupervised attribute selection in multi-source homogeneous data

Information fusion refers to derive an overall precise description of data by using certain fusion technique for utilizing the complementary information from multiple sources of data, which can facilitate effective decision-making, prediction and classification, etc. Multi-source homogeneous data, characterizing the data type of variables in the sample in form of one type (i.e., numerical or categorical) in different information sources, which widely exists in many practical applications. This paper concentrates on efficient fusion of multi-source homogeneous data with a data-level fusion model which involves the consolidation of multiple information sources and unsupervised attribute selection of the fused data. A unified description and modeling method of a multi-source homogeneous information system is introduced. The neighborhood rough sets model is used to construct the neighborhood granular structure, which uses the idea of granular computing to build methods of uncertainty measures. Given the uncertainty of fusing multiple information sources, Sup–Inf fusion functions are developed based on the proposed uncertainty measures, which can fuse the multi-source homogeneous information system into a single-source information system. Finally, an unsupervised attribute selection approach is employed to eliminate redundant attribute of the single-source information system. Theoretical analysis and comprehensive experiments on several datasets demonstrate the feasibility and superiority of our method.

Fuzzy complementary entropy using hybrid-kernel function and its unsupervised attribute reduction

Fuzzy rough set theory has been proved to be an effective tool to deal with uncertainty data. Some different forms of fuzzy uncertainty measures have been introduced in fuzzy rough set theory, such as fuzzy information entropy, fuzzy conditional entropy, and fuzzy mutual information. However, as far as we know, most of the above fuzzy conditional entropy and fuzzy mutual information are non-monotonic, which may lead to a non-convergent learning algorithm. For this reason, this paper proposes a novel fuzzy complementary entropy based on the hybrid-kernel function. Then, based on the proposed fuzzy complementary entropy, some corresponding uncertainty measures are also proposed. Furthermore, fuzzy complementary conditional entropy and fuzzy complementary mutual information are proved to change monotonously with attributes. Finally, based on the proposed uncertainty measures, three kinds of evaluation criteria for unsupervised hybrid attribute reduction are defined and a generalized attribute reduction algorithm is designed. The experimental results show that the proposed method is an effective scheme for reducing hybrid attributes.

Two kinds of orthopair soft sets as novel approaches to granular computing based on parametrization

This paper aims at introducing two types of orthopair soft sets, which might serve as novel approaches to granular computing based on parametrization. These generalized soft sets emerge naturally when linguistic parameters are employed to convey uncertainty attached to elements of certain sets. Concepts of uncertainty measures attached to the parameters and the whole orthopair soft sets are presented as well. The proposed uncertainty measures are useful for classifying elements of the set of parameters. Different types of granularity measures associated with parameters are presented and are extended to orthopair soft sets. Collective wisdom is helpful in decision making based on consensus. A numerical example is given to demonstrate how orthopair soft sets can be employed in this regard.

Information-theoretic measures of uncertainty for interval-set decision tables

Uncertainty measurement is considered as a vital quantitative way for analyzing and mining potential characteristic features in different types of decision tables. However, considering the equivalent relation is not suitable for evaluating the relationships of objects, few studies focused on the interval-set decision tables. In this paper, we address the uncertainty measurement problem in interval-set decision tables. Firstly, a similarity relation is induced by the similarity degree. Based on the similarity relation, a notion of granular structure is defined and the corresponding properties are investigated in interval-set decision tables. Secondly, we extend the accuracy and the roughness, called the interval approximation accuracy and the interval approximation roughness, to measure the uncertainty under the granular structures. By the analysis of the two extended measures, they can effectively evaluate the uncertainty caused by the approximations in the rough set model. Considering that the size of similarity classes can also affect the uncertainty, an alternative uncertainty measure based on the conditional information entropy, called the interval-decision entropy, is proposed. Moreover, a definition of reduct based on our proposed measure is provided and a heuristic attribute reduction algorithm is designed. Finally, numerical experiments demonstrate that the proposed uncertainty measures are effective and suitable for interval-set decision tables.

The uncertainty measures for covering rough set models

Uncertainty measures are important tools for analyzing various data. However, there are relatively few studies on the uncertainty measures for covering rough set models. In this paper, from the viewpoint of the lower and upper approximations, we propose new uncertainty measures, the lower rough entropy and the upper rough entropy, for covering rough set models. Then, we define the concepts of the joint entropy and the conditional entropy in the covering rough set models. Some important properties of these measures are obtained, and their relationships are investigated. Furthermore, we provide a certain characterization of reducible element of a covering by means of the proposed measures, and apply the proposed rough entropy to evaluate the significance of covering granules of a covering. Finally, we apply these rough entropies to measure a dual degree between covering lower and upper approximations. The theoretical analysis and examples show that the proposed uncertainty measures for covering rough set models are reasonable and useful.

An Improved Total Uncertainty Measure in the Evidence Theory and Its Application in Decision Making.

Dempster–Shafer evidence theory (DS theory) has some superiorities in uncertain information processing for a large variety of applications. However, the problem of how to quantify the uncertainty of basic probability assignment (BPA) in DS theory framework remain unresolved. The goal of this paper is to define a new belief entropy for measuring uncertainty of BPA with desirable properties. The new entropy can be helpful for uncertainty management in practical applications such as decision making. The proposed uncertainty measure has two components. The first component is an improved version of Dubois–Prade entropy, which aims to capture the non-specificity portion of uncertainty with a consideration of the element number in frame of discernment (FOD). The second component is adopted from Nguyen entropy, which captures conflict in BPA. We prove that the proposed entropy satisfies some desired properties proposed in the literature. In addition, the proposed entropy can be reduced to Shannon entropy if the BPA is a probability distribution. Numerical examples are presented to show the efficiency and superiority of the proposed measure as well as an application in decision making.

Maximum of Entropy for Belief Intervals Under Evidence Theory

The Dempster-Shafer Theory (DST) or Evidence Theory has been commonly used to deal with uncertainty. It is based on the basic probability assignment concept (BPA). The upper entropy on the credal set associated with a BPA is the only uncertainty measure in DST that verifies all the necessary mathematical properties and behaviors. Nonetheless, its computation is notably complex. For this reason, many alternatives to this measure have been recently proposed, but they do not satisfy most of the mathematical requirements and present some undesirable behaviors. Belief intervals have been frequently employed to quantify uncertainty in DST in the last years, and they can represent the uncertainty-based-information better than a BPA. In this research, we develop a new uncertainty measure that consists of the maximum of entropy on the credal set corresponding to belief intervals for singletons. It verifies all the crucial mathematical requirements and presents good behavior, solving most of the shortcomings found in uncertainty measures proposed recently. Moreover, its calculation is notably easier than the upper entropy on the credal set associated with the BPA. Therefore, our proposed uncertainty measure is more suitable to be used in practical applications.

Uncertainty Measurement for a Grey Information System

A grey information system (GIS) is a new kind of IS. Uncertainty measurement (UM) can provide a new perspective for data analysis and help to reveal the essential features of data. This article studies UM for a GIS. The main work of this paper includes: first, the similarity degree between two information values of each attribute in a GIS is constructed. And then, the tolerance relation induced by a given subsystem is acquired by the similarity degree. After that, the information structure of this subsystem is brought forward. Additionally, measures of uncertainty for a GIS are explored. Moreover, the optimal selection of information structures based on the proposed uncertainty measures is studied and the application of these measures is displayed. Finally, to verify the validity of these measures, statistical effectiveness analysis is carried out. These results will help us understand the intrinsic properties of uncertainty in a GIS.

Granular maximum decision entropy-based monotonic uncertainty measure for attribute reduction

Attribute reduction is considered an important preprocessing step in machine learning, pattern recognition, and data mining, and several attribute reduction measures based on rough set theory have been introduced to deal with vague, imprecise, or uncertain data. However, some of the measures inherently suffer from nonmonotonicity and redundancy. In this paper, a monotonic uncertainty measure, called granular maximum decision entropy (GMDE), is proposed. Specifically, we first develop a notion of maximum decision entropy. By integrating the uncertainty of the maximum decision entropy with the granulation of knowledge, a novel uncertainty measure is then presented, and its monotonicity is theoretically proved. We also provide a forward heuristic attribute reduction algorithm based on the proposed uncertainty measure, which could simultaneously select the informative attributes and remove the unnecessary attributes in the procedure of attribute reduction, thus resulting in high efficiency. The experiments conducted on several UCI data sets demonstrate that the proposed measure and algorithm are effective and computationally inexpensive and are superior to the representatives in terms of classification performance and efficiency.