Conditional Information Entropy Research Articles

Adaptive fuzzy neighborhood decision tree

Decision tree algorithms have gained widespread acceptance in machine learning, with the central challenge lying in devising an optimal splitting strategy for node sample subspaces. In the context of continuous data, conventional approaches typically involve fuzzifying data or adopting a dichotomous scheme akin to the CART tree. Nevertheless, fuzzifying continuous features often entails information loss, whereas the dichotomous approach can generate an excessive number of classification rules, potentially leading to overfitting. To address these limitations, this study introduces an adaptive growth decision tree framework, termed the fuzzy neighborhood decision tree (FNDT). Initially, we establish a fuzzy neighborhood decision model by leveraging the concept of fuzzy inclusion degree. Furthermore, we delve into the topological structure of misclassified samples under the proposed decision model, providing a theoretical foundation for the construction of FNDT. Subsequently, we utilize conditional information entropy to sift through original features, prioritizing those that offer the maximum information gain for decision tree nodes. By leveraging the conditional decision partitions derived from the fuzzy neighborhood decision model, we achieve an adaptive splitting method for optimal features, culminating in an adaptive growth decision tree algorithm that relies solely on the inherent structure of real-valued data. Experimental evaluations reveal that, compared with advanced decision tree algorithms, FNDT exhibits a simple tree structure, stronger generalization capabilities, and superior performance in classifying continuous data.

Neighborhood entropy guided by a decision attribute and its applications in multi-source information fusion and attribute selection

The existing methods for information fusion and attribute selection typically require a computationally intensive grid search to determine the optimal parameter. This grid search can significantly impact the efficiency of these processes. To tackle this issue, this paper introduces the concept of neighborhood entropy, which incorporates both the neighborhood and decision classes, and delves into its applications in multi-source information fusion and attribute selection. First, the concept of a δ-neighborhood based on a predefined distance is introduced. Then the neighborhood entropy based on the δ-neighborhood is defined. The optimal parameters for information fusion and attribute selection are determined by minimizing the neighborhood entropy. Furthermore, an information fusion method and an attribute selection algorithm based on the neighborhood entropy, conditional information entropy (CIE), and belief functions are developed. Finally, we demonstrate the effectiveness of our approach through experiments and Wilcoxon tests on 12 datasets, including a large-scale gene dataset. The experimental results show that the optimal parameter determined by the neighborhood entropy aligns with that obtained through grid search and the neighborhood entropy reduces the computational burden of information fusion and attribute selection compared to grid search. When compared to other state-of-the-art algorithms, our approach based on the neighborhood entropy and CIE demonstrates superior classification performance and time efficiency. These findings demonstrate the efficiency and effectiveness of neighborhood entropy and pave the way for applying granular-computing-based information fusion methods to large-scale datasets.

Comprehensive Evaluation of the Development Level of China’s Characteristic Towns under the Perspective of an Urban–Rural Integration Development Strategy

With the advancement of urbanization and the continuous deepening of reforms in urban–rural systems, China’s urbanization process has entered a new era of integrated urban–rural integration. Currently, as a global “new green revolution” gains momentum, numerous countries are deeply integrating the concept of sustainable development into new urban planning. Against this backdrop, urban planners worldwide are committed to building green, livable, and smart cities that can meet the needs of the present generation without compromising the ability of future generations to meet their needs, thus achieving the vision of harmonious coexistence between humanity and nature. Characteristic towns, leveraging their resource advantages, play a significant role in achieving sustainable regional economic development. They serve as valuable references for China’s urban transformation and upgrading, as well as for promoting rural urbanization, and are crucial avenues for advancing China’s urban–rural integration development strategy. The evaluation of the development level of characteristic towns is a necessary step in their progress and a strong guarantee for promoting their construction and development. Therefore, effectively evaluating the social benefits of characteristic towns is paramount. This study constructs an evaluation model based on the grey rough set theory and Technique for Order Preference by Similarity to Ideal Solution of TOPSIS. Firstly, an evaluation index system for the development level of characteristic towns is established. Then, the grey relational analysis method and rough set theory are used to reduce the index attributes, while the conditional information entropy theory is introduced to determine the weights of the reduced indicators. Finally, the TOPSIS model is applied to evaluate the development level of characteristic towns. Through empirical research, eight characteristic towns in Zhejiang Province, China, were assessed and ranked, verifying the effectiveness and feasibility of the proposed model.

Intuitionistic Connection Cloud Model Based on Rough Set for Evaluation of the Shrinkage–Swelling of Untreated and Lime-Treated Expansive Clays

The evaluation of the shrinkage–swelling characteristic of expansive clay is of great significance, but it is a complex problem since the evaluation process involves numerous uncertain factors, such as randomness, non-subordination, and hesitation uncertainties. Here, an intuitionistic connection cloud model has been proposed to address this issue. First, an evaluation index system is established. According to the reliability of interval-valued evaluation indexes, the corresponding cloud numerical characteristic parameters are specified based on the membership interval generated by the intuitionistic fuzzy principle. Moreover, the improved conditional information entropy based on rough set theory is utilized to assign the index weight. Subsequently, combined with the weight, the intuitionistic connection degree of the sample to the classification standard is determined to identify the shrinkage–swelling grade. Finally, a case study on the shrinkage–swelling grade of untreated and lime-treated expansive clays in Hefei Xinqiao International Airport was performed to illustrate the validity and reliability of the model. The results show that the proposed model is reasonable and feasible for the evaluation of the shrinkage–swelling grade of untreated and lime-treated expansive clays.

Attribute reduction for heterogeneous data based on monotonic relative neighborhood granularity

The neighborhood rough set model serves as an important tool for handling attribute reduction tasks involving heterogeneous attributes. However, measuring the relationship between conditional attributes and decision in the neighborhood rough set model is a crucial issue. Most studies have utilized neighborhood information entropy to measure the relationship between attributes. When using neighborhood conditional information entropy to measure the relationships between the decision and conditional attributes, it lacks monotonicity, consequently affecting the rationality of the final attribute reduction subset. In this paper, we introduce the concept of neighborhood granularity and propose a new form of relative neighborhood granularity to measure the relationship between the decision and conditional attributes, which exhibits monotonicity. Moreover, our approach for measuring neighborhood granularity avoids the logarithmic function computation involved in neighborhood information entropy. Finally, we conduct comparative experiments on 12 datasets using two classifiers to compare the results of attribute reduction with six other attribute reduction algorithms. The comparison demonstrates the advantages of our measurement approach.

The degree of importance for partially labeled incomplete real-valued data with application to semi-supervised attribute reduction

Incomplete real-valued data often misses some labels due to the high cost of labeling data. This paper investigates for partially labeled incomplete real-valued data and considers its application in semi-supervised attribute reduction. There are two decision information systems (DISs) in a partially labeled incomplete real-valued data DIS (p-IRVDIS): a labeled incomplete real-valued data DIS (l-IRVDIS) and a unlabeled incomplete real-valued data DIS (u-IRVDIS). The degree of importance on an attribute subset in a p-IRVDIS are defined using an indistinguishable relation and conditional information entropy. It is the weighted sum of l-IRVDIS and u-IRVDIS using the missing rate of label to measure p-IRVDIS uncertainty. Based on the degree of importance, an adaptive semi-supervised attribute reduction algorithm in a p-IRVDIS is proposed. This algorithm can automatically adapt to various missing rates of label. The experimental results on 8 datasets reveal that the proposed algorithm performs statistically better than some state-of-the-art algorithms.

Exploring measure of uncertainty via a discernibility relation for partially labeled real-valued data

In practical applications of machine learning, only part of data is labeled because the cost of assessing class label is relatively high. Measure of uncertainty is abbreviated as MU. This paper explores MU for partially labeled real-valued data via a discernibility relation. First, a decision information system with partially labeled real-valued data (p-RVDIS) is separated into two decision information systems: one is the decision information system with labeled real-valued data (l-RVDIS) and the other is the decision information system with unlabeled real-valued data (u-RVDIS). Then, based on a discernibility relation, dependence function, conditional information entropy and conditional information amount, four degrees of importance on an attribute subset in a p-RVDIS are defined. They are calculated by taking the weighted sum of l-RVDIS and u-RVDIS based on the missing rate, which can be considered as four MUs for a p-RVDIS. Combining l-RVDIS and u-RVDIS provides a more accurate assessment of the importance and classification ability of attribute subsets in a p-RVDIS. This is precisely the novelty of this paper. Finally, experimental analysis on several datasets verify the effectiveness of these MUs. These findings will contribute to the comprehension of the essence of the uncertainty in a p-RVDIS.

Outlier detection using conditional information entropy and rough set theory

Outlier detection is critically important in the field of data mining. Real-world data have the impreciseness and ambiguity which can be handled by means of rough set theory. Information entropy is an effective way to measure the uncertainty in an information system. Most outlier detection methods may be called unsupervised outlier detection because they are only dealt with unlabeled data. When sufficient labeled data are available, these methods are used in a decision information system, which means that the decision attribute is discarded. Thus, these methods maybe not right for outlier detection in a a decision information system. This paper proposes supervised outlier detection using conditional information entropy and rough set theory. Firstly, conditional information entropy in a decision information system based on rough set theory is calculated, which provides a more comprehensive measure of uncertainty. Then, the relative entropy and relative cardinality are put forward. Next, the degree of outlierness and weight function are presented to find outlier factors. Finally, a conditional information entropy-based outlier detection algorithm is given. The performance of the given algorithm is evaluated and compared with the existing outlier detection algorithms such as LOF, KNN, Forest, SVM, IE, and ECOD. Twelve data sets have been taken from UCI to prove its efficiency and performance. For example, the AUC value of CIE algorithm in the Hayes data set is 0.949, and the AUC values of LOF, KNN, SVM, Forest, IE and ECOD algorithms in the Hayes data set are 0.647, 0.572, 0.680, 0.676, 0.928 and 0.667, respectively. The advantage of the proposed outlier detection method is that it fully utilizes the decision information.

An Analysis of the Values of Project-Based Learning in English Language Teaching and the Cultivating Functions it Displays

Abstract In this paper, an English teaching cultivation evaluation model based on RBF neural network is designed to solve the problems of poor generalization ability, low evaluation accuracy, and correlation coefficient of the current teaching evaluation model. Before establishing the evaluation model, conditional information entropy is used to screen the evaluation indexes of the English teaching cultivation function, and six dimensions of core literacy and five dimensions of thinking ability are selected as evaluation indexes. A new relative merit matrix is constructed under the constraint of relative merit by using the screened conditional attributes. Selection, crossover, and mutation operations are used in the RBF neural network to select the optimal parameters through the introduction of the genetic algorithm. Finally, the assessment data of students after project-based English teaching are analyzed for values and nurturing functions using the evaluation model. According to the findings, project-based English teaching led to the highest average score of 29.15 for students’ values compared to the dimensions of interest, ability, and personality. In terms of the six dimensions of core literacy and the five dimensions of thinking ability, the score difference between the experimental class and the control class is more than 1.2 points, and the Sig value of each dimension is less than 0.05, which indicates that English teaching based on Project-Based Learning promotes students’ values, core literacy and thinking ability.

Attribute Reduction of Hybrid Decision Information Systems Based on Fuzzy Conditional Information Entropy

Attribute Reduction of Hybrid Decision Information Systems Based on Fuzzy Conditional Information Entropy

A novel method to information fusion in multi-source incomplete interval-valued data via conditional information entropy: Application to mutual information entropy based attribute reduction

In the era of explosive data growth, data sources and volumes are rapidly increasing. A multi-source data refers to information from multi-sources. However, not every source of information is equally important; some sources are more important and some are essentially worthless. Therefore, it is very meaningful to study how to select the most valuable sources and to efficiently fuse information. Multi-source incomplete interval-valued data (MSIIV-data) is an important kind of multi-source data. This paper proposes a novel method to information fusion in MSIIV-data via conditional information entropy (CIE) and considers its application to attribute reduction based on mutual information entropy. First, the distance between two information values for each incomplete interval-valued data is defined, the neighborhood classes with a tunable parameter are obtained, and the neighborhood granularity structure is established. Then, a source selection method is given via CIE, which is used to fuse MSIIV-data into single-source incomplete interval-valued data (SSIIV-data). Based on the minimization of CIE, this method allows worthy and reliable information sources to be chosen. Moreover, an attribute reduction algorithm (denoted as MMQPSO) for the fused SSIIV-data is proposed by means of combining mutual information entropy and QPSO-algorithm. Finally, experiments are done to validate the effectiveness of the proposed algorithms. The results of experiment and statistical test on 12 datasets show that the proposed algorithms have certain feasibility and advancement than 6 other advanced algorithms.

Information fusion and attribute reduction for multi-source incomplete mixed data via conditional information entropy and D-S evidence theory

Multi-source incomplete mixed data abound in real life, like medical data, biological data, remote sensing data, military data, etc. However, some of these sources are of less importance than others, and some are essentially worthless. Therefore, their information fusion and attribute reduction face many challenges. This paper studies information fusion for multi-source incomplete mixed data via conditional information entropy (CIE) and considers its application to attribute reduction based on D-S evidence theory. First of all, a new distance function is defined to measure the difference between nominal attribute values with missing information, and the neighborhood rough set model is used to establish the granularity structure of multi-source incomplete mixed data. Then, a source selection method is given via CIE, which is used to fuse multi-source incomplete mixed data into single-source incomplete mixed data. Based on the maximization of CIE, this method allows worthy and reliable information sources to be chosen. Next, the connection between neighborhood rough set model and D-S evidence theory is established. Moreover, two attribute reduction algorithms for the fused incomplete mixed data are proposed based on the belief and plausibility. Finally, experiments are done to verify the effectiveness of the proposed fusion and reduction algorithms. The results of experiment and statistical test on 12 datasets show that the proposed algorithms exceed other advanced algorithms in classification performance.

Outlier detection for partially labeled categorical data based on conditional information entropy

Labeling a large amount of data is exceptionally costly and practically infeasible, and thus available data may have missing labels. In this article, we investigate outlier detection for partially labeled categorical data based on conditional information entropy. Firstly, the equivalence class in a partially labeled categorical decision information system (p-CDIS) is introduced, so that the missing labels can be predicted by use of conditional probability. Then, conditional information entropy in a p-CDIS is calculated, which provides a more comprehensive measure of uncertainty. Additionally, the relative information entropy and relative cardinality in a p-CDIS are proposed. Next, the degree of outlierness and the weight function are presented to find outlier factors. Finally, an outlier detection method in a p-CDIS based on conditional information entropy is proposed, and a corresponding conditional information entropy algorithm (CEOF) is designed. To evaluate the stability of the CEOF algorithm, experiments are performed on ten UCI Machine Learning Repository datasets. Compared with five other algorithms, the proposed method is shown to have good effectiveness and adaptability for categorical data.

Variable Gap MAG Welding Penetration Control Using Rough Set Knowledge Acquisition

The establishment of the correspondence between the important features of the topside weld pool and the penetration state is a key link in penetration control. This article presents a scheme of variable gap welding penetration control based on rough-fuzzy modeling. By utilizing the basic concepts of variable precision rough sets and conditional information entropy, relevant algorithms for attribute reduction and rule extraction in decision information systems are addressed. Raw data for modeling are obtained through current–gap combination experiments, and then rough-set-based knowledge acquisition algorithms are used to generate classification rules for the penetration state. Based on the correspondence between the important features of the topside pool and the penetration state, a fuzzy control model is established. Welding experiments under variable gaps are conducted to examine the proposed control scheme, and the results indicate that the weld has good penetration and meets the requirements of welding specifications.

A novel evaluation method of renewable energy development based on improved rough set theory and grey cloud model

Renewable energy is an effective solution to alleviate the energy crisis and achieve sustainable development, and is gradually growing into an important energy source to support economic and social development. Different renewable energy technical plans have different characteristics under multiple criteria. Therefore, before the further exploitation of renewable energy sources, it is of great significance to evaluate the comprehensive performance of different plans and then determine the best renewable energy sources. Given this background, this paper proposes a novel evaluation method for renewable energy technical plans based on improved rough set theory and grey cloud model. Firstly, based on the normalized evaluation data, self-organizing map (SOM) clustering algorithm and system clustering analysis are adopted to discretize the evaluation attributes indexes and decision attributes indexes, respectively. Next, the improved rough set conditional information entropy method is used to calculate the index weights. Finally, the grey cloud model is employed to determine the grade classification of each evaluated plan. At the same time, each grade is quantified by assigning a corresponding score, and the comprehensive performances of renewable energy plans can be calculated, and then the ranking of different plans is achieved. Based on the actual renewable energy development data from a province in China, experiments were carried out to evaluate the performance of the proposed method. The results show that the proposed method performs better than some evaluation methods for renewable energy technical plans.

Exploration and practice of combining theoretical teaching and practical teaching to implement course Civics in the context of big data--Take “Microbial Pharmaceutical Technology” as an example

Abstract To better cultivate talents in the direction of “Microbiological Pharmaceutical Technology”. In this paper, we first mine the Civic Elements from the Microbial Pharmaceuticals course with the help of a clustering algorithm and calculate the similarity between data objects by Euclidean distance. Secondly, the Apriori algorithm is applied to find the largest set of frequent items in the data set of Civic Science elements and analyze them. Then the ID3 decision tree is used to calculate the information gain involving conditional entropy and information entropy, and then the data are classified. Based on the above algorithm, a Civics teaching platform combined with theoretical and practical teaching is constructed. Finally, to verify that the platform constructed in this paper achieves the combination of theoretical teaching and practical teaching, a set of simulation experiments is designed in this paper, taking “Microbial Pharmaceutical Technology” as an example, and the results show that applying the Civics Teaching Platform to the microbial pharmaceutical course, the number of teachers Civics teaching elements mining increases from 14 to 22, and the ability of Civics elements mining increases significantly. Thus, it can be seen that the implementation of the Civic Government teaching platform based on the big data algorithm of the course has improved the teaching ability of teachers and is conducive to the realization of the talent cultivation goal of the course “Microbial Pharmaceutical Technology”.

Feature selection for multiset-valued data based on fuzzy conditional information entropy using iterative model and matrix operation

Multiset-valued data is a powerful tool to deal with missing feature values. Classical rough set theory is sensitive to noise in classification learning due to the stringent condition of equivalence relation. Thus, a class of fuzzy relations can be introduced to describe the similarity between samples in multiset-valued data. However, these fuzzy relations have deficiencies when they are used in the computation of fuzzy conditional information entropy. This paper proposes a new model for multiset-valued data by introducing two variable parameters: one controls the similarity between samples, the other dominates the distance between feature values. This model employs the iterative computation strategy to define fuzzy conditional information entropy that is computing by matrix. The notions of three fuzzy information entropies in a multiset-valued decision information system (MVDIS) are put forward. For measuring any added feature, the significance based on fuzzy conditional information entropy is introduced. Moreover, fuzzy conditional information entropy iterative model (FCIEI-model) is presented. Finally, feature selection in an MVDIS based on FCIEI-model and matrix operation is researched, and the relevant algorithm (denoted as FCIEIM-MO algorithm) is proposed. The experimental results demonstrate that FCIEIM-MO algorithm possesses good robustness and is superior to some feature selection algorithms. The experimental results demonstrate that FCIEIM-MO algorithm possesses good robustness and exhibit better performance compared with some feature selection algorithms.

Feature selection in a discrete feature space based on fuzzy conditional information entropy iterative model and matrix operation

A discrete feature space consists of the set of samples, the set of categorical attributes (features) describing the samples and a decision attribute. Generally, many feature selection algorithms for a discrete feature space are based on the lower approximation or information entropy. However, the calculation of the lower approximation is more cumbersome, and information entropy may result based on equivalence class in a poor selection of features. This paper proposes feature selection algorithm for a discrete feature space by using fuzzy conditional information entropy iterative strategy and matrix operation. Firstly, the fuzzy symmetry relation induced by a discrete feature space is defined. Then, fuzzy conditional and joint information entropy for a discrete feature space are presented, and some properties are obtained. Subsequently, fuzzy conditional information entropy iterative model (FCIEI-model) is proposed. Moreover, difference, block diagonal, and decision block diagonal matrices are introduced. Next, a feature selection algorithm (denoted as FDM-algorithm) on account of FCIEI-model and matrix operations is designed and its time complexity is analyzed. Finally, the performance of the algorithm is evaluated through a series of experiments. The results show that the given algorithm is better than the existing algorithms.

Uncertainty assessment of 3D geological models based on spatial diffusion and merging model

AbstractThe geological model plays an important role in geophysics and engineering geology. The data source of geological modeling comes from interpretation data, borehole data, and outcrop data. Due to economic and technical limitations, it is impossible to obtain highly accurate and high-density data sources. The sparsity and inaccuracy of data sources lead to the uncertainty in geological models. Unlike the problem of probability, there is not enough samples for a geological model. Spatial diffusion model and merging model are introduced, which are more satisfied with the cognition of uncertainty than the existing methods. And then, using conditional information entropy, a quantification method of geological uncertainty, is proposed. Compared with the approaches of information entropy, this method took full account of the constraints of geological laws. Based on the uncertainty models and conditional information entropy, a framework of uncertainty assessment in geological models is established. It is not necessary in our framework to create multiple geological models, which is a time-consuming and laborious task. The application of Hashan survey located at north of China shows that the method and framework of this study are reasonable and effective.

Semi-supervised attribute reduction for partially labeled categorical data based on predicted label

In many practical applications of machine learning, there are a large number of partially labeled categorical data due to the high cost of labeling data. The semi-supervised learning algorithm is needed to deal with such data. This paper studies the label prediction of partially labeled categorical data and considers semi-supervised attribute reduction in a partially labeled categorical decision information system (p-CDIS) with predicted labels. The labels of unlabeled data are first predicted by means of the conditional probability. Then, uncertainty measurement for a p-CDIS with predicted labels is studied, and the dependence and conditional information entropy (CIE) are defined. Next, based on the dependence and CIE, two attribute reduction algorithms are designed. In addition, the effect of label deletion rate (LDR) on the dependence, CIE and reduction results are also studied. Finally, the results of experiments and statistical tests on 16 categorical UCI datasets show that the designed algorithms are statistically better than some state-of-the-art algorithms in classification accuracy.