Minimal Reduct Research Articles

Breadth search strategies for finding minimal reducts: towards hardware implementation

Attribute reduction, being a complex problem in data mining, has attracted many researchers. The importance of this issue rises due to ever-growing data to be mined. Together with data growth, a need for speeding up computations increases. The contribution of this paper is twofold: (1) investigation of breadth search strategies for finding minimal reducts in order to emerge the most promising method for processing large data sets; (2) development and implementation of the first hardware approach to finding minimal reducts in order to speed up time-consuming computations. Experimental research showed that for software implementation blind breadth search strategy is in general faster than frequency-based breadth search strategy not only in finding all minimal reducts but also in finding one of them. An inverse situation was observed for hardware implementation. In the future work, the implemented tool is to be used as a fundamental module in a system to be built for processing large data sets.

A new approach for reduction of attributes based on stripped quotient sets

Attribute reduction is a key problem in many areas such as data mining, pattern recognition, machine learning. The problems of finding all reducts as well as finding a minimal reduct in a given data table have been proved to be NP-hard. Therefore, to overcome this difficulty, many heuristic attribute reduction methods have been developed in recent years. In the process of heuristic attribute reduction, accelerating calculation of attribute significance is very important, especially for big data cases. In this paper, we firstly propose attribute significance measures based on stripped quotient sets. Then, by using these measures, we design efficient algorithms for calculating core and reduct, in which the time complexity will be considered in detail. Additionally, we will also give properties directly related to efficiently computing the attribute significance and significantly reducing the data size in the process of calculation. By theoretical and experimental views, we will show that our method can perform efficiently for large-scale data sets.

Fast feature selection algorithm for neighborhood rough set model based on Bucket and Trie structures

Feature selection is viewed as the problem of finding the minimal number of features from an original set with the minimum information loss. Due to its high importance in the fields of pattern recognition and data mining, it is necessary to investigate fast and effective search algorithms. In this paper, we introduce a novel fast feature selection algorithm for neighborhood rough set model based on Bucket and Trie structures. This proposed algorithm can guarantee to find the optimal minimal reduct by adopting a global search strategy. In addition, the dependence degree is subsequently used to evaluate the relevance of the attribute subset. Consequently, the proposed algorithm is tested on several standard data sets from UCI repository and compared with the most recent related approaches. The obtained theoretical and experimental results reveal that the present algorithm is very effective and convenient for the problem of feature selection, indicating that it could be useful for many pattern recognition and data mining systems.

Minimal Element Selection in the Discernibility Matrix for Attribute Reduction

Discernibility matrix is a beautiful theoretical result to get reducts in the rough set, but the existing algorithms based on discernibility matrix share the same problem of heavy computing load and large store space, since there are numerous redundancy elements in discernibility matrix and these algorithms employ all elements to find reducts. We introduce a new method to compute attribute significance. A novel approach is proposed, called minimal element selection tree, which utilizes many strategies to eliminate redundancy elements in discernibility matrix. This paper presents two methods to find out a minimal reduct for a given decision table based this tree structure. The experimental results with UCI data show that the proposed approaches are effective and efficient than the benchmark methods.

Prediction of cancer using customised fuzzy rough machine learning approaches.

This Letter proposes a customised approach for attribute selection applied to the fuzzy rough quick reduct algorithm. The unbalanced data is balanced using synthetic minority oversampling technique. The huge dimensionality of the cancer data is reduced using a correlation-based filter. The dimensionality reduced balanced attribute gene subset is used to compute the final minimal reduct set using a customised fuzzy triangular norm operator on the fuzzy rough quick reduct algorithm. The customised fuzzy triangular norm operator is used with a Lukasiewicz fuzzy implicator to compute the fuzzy approximation. The customised operator selects the least number of informative feature genes from the dimensionality reduced datasets. Classification accuracy using leave-one-out cross validation of 94.85, 76.54, 98.11, and 99.13% is obtained using a customised function for Lukasiewicz triangular norm operator on leukemia, central nervous system, lung, and ovarian datasets, respectively. Performance analysis of the conventional fuzzy rough quick reduct and the proposed method are performed using parameters such as classification accuracy, precision, recall, F-measure, scatter plots, receiver operating characteristic area, McNemar test, chi-squared test, Matthew's correlation coefficient and false discovery rate that are used to prove that the proposed approach performs better than available methods in the literature.

An Arabic text categorization approach using term weighting and multiple reducts

Text categorization is the process of assigning a predefined category label to an unlabeled document based on its content. One of the challenges of automatic text categorization is the high dimensionality of data that may affect the performance of the categorization model. This paper proposed an approach for the categorization of Arabic text based on term weighting and the reduct concept of the rough set theory to reduce the number of terms used to generate the classification rules that form the classifier. The paper proposed a multiple minimal reduct extraction algorithm by improving the Quick reduct algorithm. The multiple reducts are used to generate the set of classification rules which represent the rough set classifier. To evaluate the proposed approach, an Arabic corpus of 2700 documents nine categories is used. In the experiment, we compared the results of the proposed approach when using multiple and single minimal reducts. The results showed that the proposed approach had achieved an accuracy of 94% when using multiple reducts, which outperformed the single reduct method which achieved an accuracy of 86%. The results of the experiments also showed that the proposed approach outperforms both the K-NN and J48 algorithms regarding classification accuracy using the dataset on hand.

Granular Partition and Concept Lattice Division Based on Quotient Space

In this paper, we investigate the relationship between the concept lattice and quotient space by granularity. A new framework of knowledge representation - granular quotient space - is constructed and it demonstrates that concept lattice classing is linked to quotient space. The covering of the formal context is firstly given based on this granule, then the granular concept lattice model and its construction are discussed on the sub-context which is formed by the granular classification set. We analyze knowledge reduction and give the description of granular entropy techniques, including some novel formulas. Lastly, a concept lattice constructing algorithm is proposed based on multi-granular feature selection in quotient space. Examples and experiments show that the algorithm can obtain a minimal reduct and is much more efficient than classical incremental concept formation methods.

Minimal decision cost reduct in fuzzy decision-theoretic rough set model

Decision-theoretic rough set model interprets the parameters of existing probabilistic rough sets by introducing Bayesian theory to minimize the risks of classification. It is a representative probabilistic rough set model. As an important extension of decision-theoretic rough set model, fuzzy decision-theoretic rough set model can deal with numerical data directly. To investigate the attribute reduction in fuzzy decision-theoretic rough set model, we propose two attribute reduction approaches: global reduction and local reduction. Global reduction can keep the cost of all the decision classes unchanged or decreased. Moreover, local reduction can make the cost of single decision class unchanged or decreased. In this paper, we use two different heuristic functions to implement heuristic algorithms. Heuristic algorithms are used to find the minimal subset of condition attributes which minimizes the (P) cost, (B) cost and (N) cost in fuzzy decision theoretic-rough set model. Six UCI datasets are employed to test our heuristic algorithms. The experimental results indicate that global reduction and local reduction can minimize the decision cost in fuzzy decision-theoretic rough set model effectively. After reduction, the numbers of condition attributes are reduced greatly. On the other hand, global reduction and local reduction both can decrease the numbers of (P) rule and (B) rule on the whole. The results show that when we reduce the decision cost, we will also reduce the rule numbers as well.

A Hybrid Scheme for Breast Cancer Detection using Intuitionistic Fuzzy Rough Set Technique

Diagnosis of cancer is of prime concern in recent years. Medical imaging is used to analyze these diseases. But, these images contain uncertainties due to various factors and thus intelligent techniques are essential to process these uncertainties. This paper hybridizes intuitionistic fuzzy set and rough set in combination with statistical feature extraction techniques. The hybrid scheme starts with image segmentation using intuitionistic fuzzy set to extract the zone of interest and then to enhance the edges surrounding it. Further feature extraction using gray-level co-occurrence matrix is presented. Additionally, rough set is used to engender all minimal reducts and rules. These rules then fed into a classifier to identify different zones of interest and to check whether these points contain decision class value as either cancer or not. The experimental analysis shows the overall accuracy of 98.3% and it is higher than the accuracy achieved by hybridizing fuzzy rough set model.

Finding rough set reducts with fish swarm algorithm

Rough set theory is one of the effective methods to feature selection which can preserve the characteristics of the original features by deleting redundant information. The main idea of rough set approach to feature selection is to find a globally minimal reduct, the smallest set of features keeping important information of the original set of features. Rough set theory has been used as a dataset preprocessor with much success, but current approaches to feature selection are inadequate for finding a globally minimal reduct. In this paper, we propose a novel rough set based method to feature selection using fish swarm algorithm. The fish swarm algorithm is a new intelligent swarm modeling approach that consists primarily of searching, swarming, and following behaviors. It is attractive for feature selection since fish swarms can discover the best combination of features as they swim within the subset space. In our proposed algorithm, a minimal subset can be located and verified. To show the efficiency of our algorithm, we carry out numerical experiments based on some standard UCI datasets. The results demonstrate that our algorithm can provide an efficient tool for finding a minimal subset of the features without information loss.

An exact feature selection algorithm based on rough set theory

Feature reduction based on rough set theory is an effective feature selection method in pattern recognition applications. Finding a minimal subset of the original features is inherent in rough set approach to feature selection. As feature reduction is a Nondeterministic Polynomial‐time‐hard problem, it is necessary to develop fast optimal or near‐optimal feature selection algorithms. This article aims to propose an exact feature selection algorithm in rough set that is efficient in terms of computation time. The proposed algorithm begins the examination of a solution tree by a breadth‐first strategy. The pruned nodes are held in a version of the trie data structure. Based on the monotonic property of dependency degree, all subsets of the pruned nodes cannot be optimal solutions. Thus, by detecting these subsets in trie, it is not necessary to calculate their dependency degree. The search on the tree continues until the optimal solution is found. This algorithm is improved by selecting an initial search level determined by the hill‐climbing method instead of searching the tree from the level below the root. The length of the minimal reduct and the size of data set can influence which starting search level is more efficient. The experimental results using some of the standard UCI data sets, demonstrate that the proposed algorithm is effective and efficient for data sets with more than 30 features. © 2014 Wiley Periodicals, Inc. Complexity 20: 50–62, 2015

A set covering based approach to find the reduct of variable precision rough set

Attribute reduction is one of the core problems in Rough Set (RS) theory. In the Variable Precision Rough Set (VPRS) model, attribute reduction faces two difficulties: firstly, in the VPRS model, a reduct anomaly problem may arise and it may cause an inconsistency of positive regions and decision rules after attribute reduction. Secondly, the attribution reduction problem has been proved an NP-hard problem; accordingly, we would need to find a tradeoff between calculating the minimal reduct and reducing computing complexity to avoid the combinatorial explosion problem. We propose a new approach to calculate the reduct in VPRS model. This new method focuses on calculating a β-distribution reduct while avoiding the anomaly problem in the VPRS model. The basic idea of the proposed approach is to convert the reduct problem into a Set Covering Problem (SCP) according to the positive regions in the VPRS model; and consequently, a Set-Covering Heuristic Function (SCHF) algorithm is applied to calculate the reduct after this conversion. This approach keeps the positive regions consistent after the attribute reduction and moreover, based on the SCP, the performance ratio of the proposed method to calculate the minimal reduct ranges between ln(U′)-lnln(U′)+o(1) and (1-o(1))ln(U′) with a computational complexity having an upper bound as o(MN(M+N)2). Finally, we demonstrate the practical application of the VPRS model using real case scenario from China’s electricity power yield to verify the validity of our proposed approach. We then apply statistical evaluation to explain the economic significance of the attributes.

Knowledge reduction for decision tables with attribute value taxonomies

Attribute reduction and attribute generalization are two basic methods for simple representations of knowledge. Attribute reduction can only reduce the number of attributes and is thus unsuitable for attributes with hierarchical domains. Attribute generalization can transform raw attribute domains into a coarser granularity by exploiting attribute value taxonomies (AVTs). As the control of how high an attribute should be generalized is typically quite subjective, it can easily result in over-generalization or under-generalization. This paper investigates knowledge reduction for decision tables with AVTs, which can objectively control the generalization process, and construct a reduced data set with fewer attributes and smaller attribute domains. Specifically, we make use of Shannon’s conditional entropy for measuring classification capability for generalization and propose a novel concept for knowledge reduction, designated attribute-generalization reduct, which can objectively generalize attributes to maximize high levels while keep the same classification capability as the raw data. We analyze major relationships between attribute reduct and attribute-generalization reduct and prove that finding a minimal attribute-generalization reduct is an NP-hard problem and develop a heuristic algorithm for attribute-generalization reduction, namely, AGR-SCE. Empirical studies demonstrate that our algorithm accomplishes better classification performance and assists in computing smaller rule sets with better generalized knowledge compared with the attribute reduction method.

A novel method for attribute reduction of covering decision systems

Attribute reduction has become an important step in pattern recognition and machine learning tasks. Covering rough sets, as a generalization of classical rough sets, have attracted wide attention in both theory and application. This paper provides a novel method for attribute reduction based on covering rough sets. We review the concepts of consistent and inconsistent covering decision systems and their reducts and we develop a judgment theorem and a discernibility matrix for each type of covering decision system. Furthermore, we present some basic structural properties of attribute reduction with covering rough sets. Based on a discernibility matrix, we develop a heuristic algorithm to find a subset of attributes that approximate a minimal reduct. Finally, the experimental results for UCI data sets show that the proposed reduction approach is an effective technique for addressing numerical and categorical data and is more efficient than the method presented in the paper [D.G. Chen, C.Z. Wang, Q.H. Hu, A new approach to attribute reduction of consistent and inconsistent covering decision systems with covering rough sets, Information Sciences 177(17) (2007) 3500–3518].

Equivalent Characterizations of Some Graph Problems by Covering-Based Rough Sets

Covering is a widely used form of data structures. Covering-based rough set theory provides a systematic approach to this data. In this paper, graphs are connected with covering-based rough sets. Specifically, we convert some important concepts in graph theory including vertex covers, independent sets, edge covers, and matchings to ones in covering-based rough sets. At the same time, corresponding problems in graphs are also transformed into ones in covering-based rough sets. For example, finding a minimal edge cover of a graph is translated into finding a minimal general reduct of a covering. The main contributions of this paper are threefold. First, any graph is converted to a covering. Two graphs induce the same covering if and only if they are isomorphic. Second, some new concepts are defined in covering-based rough sets to correspond with ones in graph theory. The upper approximation number is essential to describe these concepts. Finally, from a new viewpoint of covering-based rough sets, the general reduct is defined, and its equivalent characterization for the edge cover is presented. These results show the potential for the connection between covering-based rough sets and graphs.

A Novel Algorithm for Finding Reducts With Fuzzy Rough Sets

Attribute reduction is one of the most meaningful research topics in the existing fuzzy rough sets, and the approach of discernibility matrix is the mathematical foundation of computing reducts. When computing reducts with discernibility matrix, we find that only the minimal elements in a discernibility matrix are sufficient and necessary. This fact motivates our idea in this paper to develop a novel algorithm to find reducts that are based on the minimal elements in the discernibility matrix. Relative discernibility relations of conditional attributes are defined and minimal elements in the fuzzy discernibility matrix are characterized by the relative discernibility relations. Then, the algorithms to compute minimal elements and reducts are developed in the framework of fuzzy rough sets. Experimental comparison shows that the proposed algorithms are effective.

High Confidence Heuristic Strategy for Minimal Reduction

In order to improve the confidence of minimal reducts calculated by heuristic methods, some important characters of attributes, such as absorption, repulsion, and mutex etc., are presented based on the discernibility matrix. Then the related heuristic strategies are proposed by analyzing the relation between these characters and the minimal reducts. Some confidence models of these strategies are established to order these strategies. On the basis, an integrated strategy and a related reduction algorithm are proposed to calculate a minimal redcut. Theoretic and experimental analyses show that the proposed strategies are of high confidence and effectiveness.

Test-cost-sensitive attribute reduction

In many data mining and machine learning applications, there are two objectives in the task of classification; one is decreasing the test cost, the other is improving the classification accuracy. Most existing research work focuses on the latter, with attribute reduction serving as an optional pre-processing stage to remove redundant attributes. In this paper, we point out that when tests must be undertaken in parallel, attribute reduction is mandatory in dealing with the former objective. With this in mind, we posit the minimal test cost reduct problem which constitutes a new, but more general, difficulty than the classical reduct problem. We also define three metrics to evaluate the performance of reduction algorithms from a statistical viewpoint. A framework for a heuristic algorithm is proposed to deal with the new problem; specifically, an information gain-based λ-weighted reduction algorithm is designed, where weights are decided by test costs and a non-positive exponent λ, which is the only parameter set by the user. The algorithm is tested with three representative test cost distributions on four UCI (University of California – Irvine) datasets. Experimental results show that there is a trade-off while setting λ, and a competition approach can improve the quality of the result significantly. This study suggests potential application areas and new research trends concerning attribute reduction.

Analysis of alternative objective functions for attribute reduction in complete decision tables

Attribute reduction and reducts are important notions in rough set theory that can preserve discriminatory properties to the highest possible extent similar to the entire set of attributes. In this paper, the relationships among 13 types of alternative objective functions for attribute reduction are systematically analyzed in complete decision tables. For inconsistent and consistent decision tables, it is demonstrated that there are only six and two intrinsically different objective functions for attribute reduction, respectively. Some algorithms have been put forward for minimal attribute reduction according to different objective functions. Through a counterexample, it is shown that heuristic methods cannot always guarantee to produce a minimal reduct. Based on the general definition of discernibility function, a complete algorithm for finding a minimal reduct is proposed. Since it only depends on reasoning mechanisms, it can be applied under any objective function for attribute reduction as long as the corresponding discernibility matrix has been well established.

Reductos mínimos con Grasp

La detección adecuada del nivel de demencia de un paciente es importante para brindar el tratamiento adecuado. El diagnóstico se basa en ciertos criterios, reflejados en los exámenes clínicos. De estos exámenes surgen las limitaciones y el grado en que se encuentra cada una de ellas en los pacientes. Para reducir el total de limitaciones a ser evaluadas, utilizamos la teoría de Conjuntos Imprecisos, esta teoría ha sido aplicada en áreas de la inteligencia artificial tales como análisis de decisiones, sistemas expertos, descubrimiento de conocimiento, clasificación con múltiples atributos. En nuestro caso se aplica para encontrar el conjunto mínimo de limitaciones o reducto que genera la misma clasificación que considerando todas las limitaciones, para ello se desarrollo un algoritmo GRASP (Greedy Randomized Adaptive Search Procedure).