Heuristic Attribute Reduction Algorithm Research Articles

Consistent Matrix: A Feature Selection Framework for Large-Scale Datasets

Large-scale data processing based on limited computing resources has always been a difficult problem in data mining, where feature selection is often used as an effective data compressing mechanism. For granular computing of big data, discernibility matrix and dependency degree are the most representative methods for matrix-based and feature importance degree-based feature selection, respectively. However, their temporal and space complexities are high and often lead to poor performance. In this paper, a novel feature selection framework for large scale data processing with linear complexities was proposed for the first time. Firstly, a much more concise fuzzy granule set, called fuzzy arithmetic covering, was introduced to reduce computational costs. Then a new matrix-based feature selection framework, namely consistent matrix, was proposed for general rough set models. As a result, a heuristic attribute reduction algorithm, i.e., HARCM, was designed accordingly. Compared with six state-of-the-art algorithms for feature selection, the average running time of the newly proposed algorithm was reduced up to 2913 times, with a comparable or even better classification performance.

Three-Way Co-Training with Pseudo Labels for Semi-Supervised Learning

The theory of three-way decision has been widely utilized across various disciplines and fields as an efficient method for both knowledge reasoning and decision making. However, the application of the three-way decision theory to partially labeled data has received relatively less attention. In this study, we propose a semi-supervised co-training model based on the three-way decision and pseudo labels. We first present a simple yet effective method for producing two views by assigning pseudo labels to unlabeled data, based on which a heuristic attribute reduction algorithm is developed. The three-way decision is then combined with the concept of entropy to form co-decision rules for classifying unlabeled data into useful, uncertain, or useless samples. Finally, some useful samples are iteratively selected to improve the performance of the co-decision model. The experimental results on UCI datasets demonstrate that the proposed model outperforms other semi-supervised models, exhibiting its potential for partially labeled data.

Extended rough sets model based on fuzzy granular ball and its attribute reduction

Attribute reduction is one of the core steps of data analysis. The attribute reduction method based on neighborhood rough sets (NRS) is widely used. However, the time complexity of this method is excessively high because its radius selection relies on grid search. Granular ball neighborhood rough sets (GBNRS) can generate different neighborhoods adaptively and has more generality and flexibility than the NRS method. Nevertheless, due to the purity of granular ball is required to be 1, the GBNRS algorithm will generate many granular balls with the sample number of 1 at the boundaries of classes. These granular balls will be regarded as outliers and deleted, resulting in the loss of class boundary information. Moreover, due to the strict reduction conditions of GBNRS, the reduction effect cannot be achieved on some datasets. To solve the above problems, the fuzzy granular ball is defined by relaxing the generation conditions of the granular ball, and an extended rough sets model is proposed on the basis of the fuzzy granular ball (FGBERS). In this model, we retain as much class boundary information as possible by preserving the fuzzy granular ball of class boundary. A new forward heuristic attribute reduction algorithm is designed on the basis of this model. To demonstrate the performance of FGBERS, we conducted sufficient experiments on 18 real datasets from different fields. The experimental results show that FGBERS not only compensates for the shortcomings of GBNRS, but also demonstrates higher classification accuracy, especially on the KNN classifier, the classification accuracy of the FGBERS algorithm is 5% higher on average, and the highest improvement is 20% on high-dimensional dataset ALLAML. In addition, compared with the comparison algorithms, FGBERS has excellent reduction performance in both large-scale datasets and high-dimensional datasets.

A Novel Attribute Reduction Algorithm for Incomplete Information Systems Based on a Binary Similarity Matrix

With databases growing at an unrelenting rate, it may be difficult and complex to extract statistics by accessing all of the data in many practical problems. Attribute reduction, as an effective method to remove redundant attributes from massive data, has demonstrated its remarkable capability in simplifying information systems. In this paper, we concentrate on reducing attributes in incomplete information systems. We introduce a novel definition of a binary similarity matrix and present a method to calculate the significance of attributes in correspondence. Secondly, We develop a heuristic attribute reduction algorithm using a binary similarity matrix and attribute significance as heuristic knowledge. In addition, we use a numerical example to showcase the practicality and accuracy of the algorithm. In conclusion, we demonstrate through comparative analysis that our algorithm outperforms some existing attribute reduction methods.

A novel incremental attribute reduction by using quantitative dominance-based neighborhood self-information

Incremental attribute reduction aims to improve the efficiency of obtaining reduct from the dynamic data, which has attracted more and more attention. Nevertheless, the existing incremental attribute reduction methods for the dynamic ordered data with time-evolving objects ignore the updating strategies caused by selecting necessary (or removing redundant) attributes. To tackle this issue, we investigate a novel incremental attribute reduction based on quantitative dominance-based neighborhood self-information (QD-NSI) for dynamic hybrid ordered decision system. First, the measure QD-NSI is proposed by combining the quantitative dominance-based neighborhood rough set and the self-information. Different from the dependency commonly used in attribute reduction, the QD-NSI considers both deterministic and possible classification information. Based on the proposed measure, the corresponding matrix calculation method and heuristic attribute reduction algorithm are designed. Subsequently, the incremental attribute reduction mechanisms and algorithms for dynamic hybrid ordered decision system with time-evolving objects are explored. It is worth mentioning that the proposed incremental algorithms consider the update strategies caused by selecting necessary (or removing redundant) attributes. Finally, comparison experiments are performed on nine public datasets to illustrate the performance of the proposed incremental algorithms. Experimental results show that the QD-NSI based attribute reduction algorithm can effectively delete irrelevant or redundant attributes and the efficiency of the proposed incremental algorithms is better than that of the existing related incremental algorithms.

Neural Network Based on Health Monitoring Electrical Equipment Fault and Biomedical Diagnosis.

In order to improve the accuracy of electrical equipment failure diagnosis and keep electrical equipment operating safely and efficiently, this paper proposes to design an electrical equipment failure diagnosis system based on a neural network, analyze the faults of electrical equipment and their causes, and establish knowledge base according to relevant data and expert judgment. The fault knowledge base was introduced into the neural network operation structure, and the fault diagnosis results were classified step by step through multiple subnetworks. In data preprocessing, in order to avoid the redundancy of primary fault information features, the principal component heuristic attribute reduction algorithm was used to select the fault data samples optimally. The neural network learning algorithm is used to calculate the forward direction and error rate of the initial error data, and the reliability function is used to optimize the initial weight threshold of the neural network, propagating the error backwards and high. Experimental results show that adding attribute reduction improves error classification performance, avoids the problem of local minima through neural network operation, and has fewer iteration steps, lower average error, and higher accuracy of fault diagnosis, reaching 95.6%.

MapReduce accelerated attribute reduction based on neighborhood entropy with Apache Spark

Attribute reduction is nowadays an extremely important data preprocessing technique in the field of data mining, which has gained much attention due to its ability to provide better generalization performance and learning speed for analysis model. Rough set theory offers a systematic powerful framework for attribute reduction in terms of classificatory and decision abilities under uncertainty. In this paper, we present a parallel neighborhood entropy-based attribute reduction method with neighborhood rough sets that uses the Apache Spark cluster computing model to realize the parallelization of algorithm in a distributed computing environment. In leveraging the horizontal partitioning strategy to alleviate the task of data parallelism, three quantitative measures of attribute sets, i.e., neighborhood approximation accuracy, neighborhood credibility and coverage degrees are parallelized to accelerate the computation of decision neighborhood entropy during the heuristic search iterative process. A novel parallel heuristic attribute reduction algorithm is then developed by employing several operations from Spark API to ease the code parallelization. Extensive experimental results indicate the superiority and notable strengths of the proposed algorithm in terms of the criteria for evaluating parallel performance, i.e., scalability and extensibility.

Novel fuzzy [formula omitted]-covering rough set models and their applications

In recent years, fuzzy β-covering, as a natural extension of fuzzy coverings, has attracted considerable attention. However, existing fuzzy β-neighborhood operators cannot accurately describe the relationship between objects, which greatly restricts the application of fuzzy β-covering. For this reason, we first construct four new fuzzy β-neighborhood operators by using the existing fuzzy β-neighborhood operator and generalized fuzzy logic operators, and investigate their properties. To better portray the similarity between samples, inspired by the definition of fuzzy similarity relation, we define the concept of fuzzy β-covering relation. On this basis, we develop a new framework of fuzzy β-covering rough set models. We further propose an attribute reduction method by employing the new fuzzy β-covering relation, and design a heuristic attribute reduction algorithm with reference to an uncertainty measure called attribute significance. Finally, experimental results show the superiority of our proposed method through a series of experimental analyses.

Information-theoretic measures of uncertainty for interval-set decision tables

Uncertainty measurement is considered as a vital quantitative way for analyzing and mining potential characteristic features in different types of decision tables. However, considering the equivalent relation is not suitable for evaluating the relationships of objects, few studies focused on the interval-set decision tables. In this paper, we address the uncertainty measurement problem in interval-set decision tables. Firstly, a similarity relation is induced by the similarity degree. Based on the similarity relation, a notion of granular structure is defined and the corresponding properties are investigated in interval-set decision tables. Secondly, we extend the accuracy and the roughness, called the interval approximation accuracy and the interval approximation roughness, to measure the uncertainty under the granular structures. By the analysis of the two extended measures, they can effectively evaluate the uncertainty caused by the approximations in the rough set model. Considering that the size of similarity classes can also affect the uncertainty, an alternative uncertainty measure based on the conditional information entropy, called the interval-decision entropy, is proposed. Moreover, a definition of reduct based on our proposed measure is provided and a heuristic attribute reduction algorithm is designed. Finally, numerical experiments demonstrate that the proposed uncertainty measures are effective and suitable for interval-set decision tables.

Reduction of fuzzy-crisp concept lattice based on order-class matrix

Knowledge reduction is one of critical problems in data mining and information processing. It can simplify the structure of the lattice during the construction of fuzzy-crisp concept lattice. In terms of fuzzy-crisp concept, we develop an order-class matrix to represent extents and intents of concepts, respectively. In order to improve the computing efficiency, it is necessary to reduce the size of lattices as much as possible. Therefore the judgement theorem of meet-irreducible elements is proposed. To deal with attribute reductions, we develop a discernibility Boolean matrix in formal fuzzy contexts by preserving extents of meet-irreducible elements via order-class matrix. A heuristic attribute-reduction algorithm is proposed. Then we extend the proposed model to consistent formal fuzzy decision contexts. Our methods present a new framework for knowledge reduction in formal fuzzy contexts.

A fitting model for attribute reduction with fuzzy β-covering

As an intuitive extension of covering-based rough set models, fuzzy β covering provides an effective means to deal with uncertain information. However, classical fuzzy β covering can not fit a given data set well, which limits its popularization and application. On the one hand, this model can not guarantee that the lower approximation is contained in the upper approximation. It does not ideally describe the differences in sample classification. On the other hand, fuzzy β covering is sensitive to noise hidden in data, which leads to unstable classification performance. To address these issues, we set forth a new fitting model for attribute reduction based on fuzzy β covering. To this end, parameterized fuzzy β neighborhoods in terms of a family of fuzzy coverings are introduced to characterize the similarity between samples, using which the granular structures of fuzzy lower and upper approximations of a decision are presented. Fuzzy β covering decision tables are then formalized to deal with knowledge reduction from the viewpoint of maintaining the discriminative ability of a fuzzy covering family. Furthermore, a heuristic attribute reduction algorithm is developed to reduce redundant fuzzy coverings. Extensive numerical experiments are further conducted to examine the effectiveness and feasibility of the proposed model.

Neighborhood multi-granulation rough sets-based attribute reduction using Lebesgue and entropy measures in incomplete neighborhood decision systems

For incomplete data with mixed numerical and symbolic attributes, attribute reduction based on neighborhood multi-granulation rough sets (NMRS) is an important method to improve the classification performance. However, most classical attribute reduction methods can only handle finite sets as to produce more attributes and lower classification accuracy. This paper proposes a novel NMRS-based attribute reduction method using Lebesgue and entropy measures in incomplete neighborhood decision systems. First, some concepts of optimistic and pessimistic NMRS models in incomplete neighborhood decision systems are given, respectively. Then, a Lebesgue measure is combined with NMRS to study neighborhood tolerance class-based uncertainty measures. To analyze the uncertainty, noise and redundancy of incomplete neighborhood decision systems in detail, some neighborhood multi-granulation entropy-based uncertainty measures are developed by integrating Lebesgue and entropy measures. Inspired by both algebraic view with information view in NMRS, the pessimistic neighborhood multi-granulation dependency joint entropy is proposed. What is more, the corresponding properties are further deduced and the relationships among these measures are discussed, which can help to investigate the uncertainty of incomplete neighborhood decision systems. Finally, the Fisher linear discriminant method is used to eliminate irrelevant attributes to significantly reduce computational complexity for high-dimensional datasets, and a heuristic attribute reduction algorithm with complexity analysis is designed to improve classification performance of incomplete and mixed datasets. Experimental results under seven UCI datasets and eight gene expression datasets illustrate that the proposed method is effective to select most relevant attributes with higher classification accuracy, as compared with representative algorithms.

A novel measure of attribute significance with complexity weight

Attribute reduction is one of the most important problems in rough set theory. Conventional attribute reduction algorithms are based on minimal errors in seen objects, namely empirical risk minimization. Classification ability in unseen objects, namely generalization ability is more important in actual applications. Therefore, a good reduct should have good generalization ability. Structural risk minimization (SRM) inductive principle is an effective tool to control the generalization ability of learning machines, which considers complexity and errors in seen objects simultaneously. Therefore, this paper introduces the SRM principle into the definition of attribute significance, proposes that the number of rules can characterize the actual complexity of the rough set-based classifier effectively and defines a novel measure of attribute significance with complexity weight. Based on the new attribute significance, a new heuristic attribute reduction algorithm called HSRM-R algorithm is developed. The 10-fold cross-validation experiments in 21 UCI datasets show that HSRM-R algorithm obtains better generalization ability than conventional attribute reduction algorithms based on dependency degree, information entropy, Fisher score and Laplacian score. Further experiments show that HSRM-R algorithm obtains fewer rules and larger support coefficient. This means HSRM-R algorithm can extract stronger rules, which explains why it has better generalization ability to some extent. Although HSRM-R algorithm consumes more time than conventional algorithms, it obtains optimal classification accuracy in almost all datasets used in the experiments. Thus, the proposed HSRM-R algorithm provides an approach to guaranteeing the generalization ability theoretically in the case where users require high classification accuracy.

An Improved Attribute Reeducation Algorithm Based on Mutual Information for Non-Care Information System

For an information system the attribute reduction is one key to choose some important indexes, but the core attribute is the foundation for the present heuristic attribute reduction algorithms of rough set theory based on discernibility matrix and mutual information theory. When we apply these algorithms on the non-core information system, there will be the following problems, such as too much calculation problem, excessive reduction, or insufficient reduction. According to the characteristics of non-core information system, we propose one new heuristic attribute reduction algorithm based on mutual information, in which the evaluation of attribute important degree depends on the increment of mutual information and information entropy. And the attribute with the largest attribute important degree is selected for the initial core attribute, by which the problem that the random selection attribute cause the computational complexity is solved. By the proposed algorithm we cannot only improve the efficiency of attribute reduction, but decrease the number of attribute reduction. Both the theoretic analysis and the simulation experiments verify that the proposed algorithm is validity

An Attribute Reduction Method Using Neighborhood Entropy Measures in Neighborhood Rough Sets.

Attribute reduction as an important preprocessing step for data mining, and has become a hot research topic in rough set theory. Neighborhood rough set theory can overcome the shortcoming that classical rough set theory may lose some useful information in the process of discretization for continuous-valued data sets. In this paper, to improve the classification performance of complex data, a novel attribute reduction method using neighborhood entropy measures, combining algebra view with information view, in neighborhood rough sets is proposed, which has the ability of dealing with continuous data whilst maintaining the classification information of original attributes. First, to efficiently analyze the uncertainty of knowledge in neighborhood rough sets, by combining neighborhood approximate precision with neighborhood entropy, a new average neighborhood entropy, based on the strong complementarity between the algebra definition of attribute significance and the definition of information view, is presented. Then, a concept of decision neighborhood entropy is investigated for handling the uncertainty and noisiness of neighborhood decision systems, which integrates the credibility degree with the coverage degree of neighborhood decision systems to fully reflect the decision ability of attributes. Moreover, some of their properties are derived and the relationships among these measures are established, which helps to understand the essence of knowledge content and the uncertainty of neighborhood decision systems. Finally, a heuristic attribute reduction algorithm is proposed to improve the classification performance of complex data sets. The experimental results under an instance and several public data sets demonstrate that the proposed method is very effective for selecting the most relevant attributes with great classification performance.

A Neighborhood Rough Sets-Based Attribute Reduction Method Using Lebesgue and Entropy Measures.

For continuous numerical data sets, neighborhood rough sets-based attribute reduction is an important step for improving classification performance. However, most of the traditional reduction algorithms can only handle finite sets, and yield low accuracy and high cardinality. In this paper, a novel attribute reduction method using Lebesgue and entropy measures in neighborhood rough sets is proposed, which has the ability of dealing with continuous numerical data whilst maintaining the original classification information. First, Fisher score method is employed to eliminate irrelevant attributes to significantly reduce computation complexity for high-dimensional data sets. Then, Lebesgue measure is introduced into neighborhood rough sets to investigate uncertainty measure. In order to analyze the uncertainty and noisy of neighborhood decision systems well, based on Lebesgue and entropy measures, some neighborhood entropy-based uncertainty measures are presented, and by combining algebra view with information view in neighborhood rough sets, a neighborhood roughness joint entropy is developed in neighborhood decision systems. Moreover, some of their properties are derived and the relationships are established, which help to understand the essence of knowledge and the uncertainty of neighborhood decision systems. Finally, a heuristic attribute reduction algorithm is designed to improve the classification performance of large-scale complex data. The experimental results under an instance and several public data sets show that the proposed method is very effective for selecting the most relevant attributes with high classification accuracy.

Heuristic attribute reduction and resource-saving algorithm for energy data of data centers

Energy data, which consist of energy consumption statistics and other related data in green data centers, grow dramatically. The energy data have great value, but many attributes within them are redundant and unnecessary, and they have a serious impact on the performance of the data center’s decision-making system. Thus, attribute reduction for the energy data has been conceived as a critical step. However, many existing attribute reduction algorithms are often computationally time-consuming. To address these issues, firstly, we extend the methodology of rough sets to construct data center energy consumption knowledge representation system. Energy data will occur some degree of exceptions caused by power failure, energy instability or other factors; hence, we design an integrated data preprocessing method using Spark for energy data, which mainly includes sampling analysis, data classification, missing data filling, outlier data prediction and data discretization. By taking good advantage of in-memory computing, a fast heuristic attribute reduction algorithm (FHARA-S) for energy data using Spark is proposed. In this algorithm, we use an efficient algorithm for transforming energy consumption decision table, a heuristic formula for measuring the significance of attribute to reduce the search space, and introduce the correlation between condition attribute and decision attribute, which further improve the computational efficiency. We also design an adaptive decision management architecture for the green data center based on FHARA-S, which can improve decision-making efficiency and strengthen energy management. The experimental results show the speed of our algorithm gains up to 2.2X performance improvement over the traditional attribute reduction algorithm using MapReduce and 0.61X performance improvement over the algorithm using Spark. Besides, our algorithm also saves more computational resources.

Granular maximum decision entropy-based monotonic uncertainty measure for attribute reduction

Attribute reduction is considered an important preprocessing step in machine learning, pattern recognition, and data mining, and several attribute reduction measures based on rough set theory have been introduced to deal with vague, imprecise, or uncertain data. However, some of the measures inherently suffer from nonmonotonicity and redundancy. In this paper, a monotonic uncertainty measure, called granular maximum decision entropy (GMDE), is proposed. Specifically, we first develop a notion of maximum decision entropy. By integrating the uncertainty of the maximum decision entropy with the granulation of knowledge, a novel uncertainty measure is then presented, and its monotonicity is theoretically proved. We also provide a forward heuristic attribute reduction algorithm based on the proposed uncertainty measure, which could simultaneously select the informative attributes and remove the unnecessary attributes in the procedure of attribute reduction, thus resulting in high efficiency. The experiments conducted on several UCI data sets demonstrate that the proposed measure and algorithm are effective and computationally inexpensive and are superior to the representatives in terms of classification performance and efficiency.

Neighbor Inconsistent Pair Selection for Attribute Reduction by Rough Set Approach

Rough set theory, as one of the most useful soft computing methods dealing with vague and uncertain information, has been successfully applied to many fields, and one of its main applications is to perform attribute reduction. Although many heuristic attribute reduction algorithms have been proposed within the framework of the rough set theory, these methods are still computationally time consuming. In order to overcome this deficit, we propose, in this paper, two quick feature selection algorithms based on the neighbor inconsistent pair, which can reduce the time consumed in finding a reduct. At first, we propose several concepts regarding simplified decision table( $U^{^{\prime }}$ ) and neighbor inconsistent pairs. Based on neighbor inconsistent pairs, we constructed two new attribute significance measures. Furthermore, we put forward two new attribute reduction algorithms based on quick neighbor inconsistent pairs. The key characteristic of the presented algorithms is that they only need to calculate $U^{^{\prime }}$ /R once under the process of selecting the best attribute from attribute sets: $C-R$ , while most existing algorithms need to calculate partition of $U^{^{\prime }}$ for $|C-R|$ times. In addition, the proposed algorithms need only to deal with the equivalent classes in $U^{^{\prime }}$ /R that contain at least one neighbor inconsistent pair, while most existing algorithms need to consider all objects in $U^{^{\prime }}$ . The experimental results show that the proposed algorithms are feasible and efficient.

Maximum decision entropy-based attribute reduction in decision-theoretic rough set model

Decision-theoretic rough set model, as a probabilistic generalization of the Pawlak rough set model, is an effective method for decision making from vague, uncertain or imprecise data. Attribute reduction is one of the most important problems in the decision-theoretic rough set model and several uncertainty measures for attribute reduction have been presented. However, the monotonicity of the uncertainty measures does not always hold. In this paper, a novel monotonic uncertainty measure is introduced for attribute reduction in the decision-theoretic rough set model. More specifically, based on the concepts of the maximum inclusion degree and maximum decision, a new uncertainty measure, named maximum decision entropy, is first proposed, and the definitions of the positive, boundary and negative region preservation reducts are then provided by using the proposed uncertainty measure. Theoretically, it is proved that the proposed uncertainty measure is monotonic when adding or deleting the condition attributes. Additionally, a heuristic attribute reduction algorithm based on the maximum decision entropy is developed, which maximizes the relevance of the reduct to the class attribute and also minimizes the redundancy of the condition attributes within the reduct. The experimental results on artificial as well as real data sets demonstrate the competitive performance of our proposal in comparison with the state-of-the-art algorithms.