Neighborhood Decision Systems Research Articles

A distributed attribute reduction based on neighborhood evidential conflict with Apache Spark

Attribute reduction is widely employed to improve the efficiency and accuracy of data analysis by eliminating redundant and irrelevant attributes from datasets. However, with the emergence of growing big data, the sequential execution of such algorithms becomes time-consuming and requires distributed computing capabilities to achieve scalable parallelization. This study proposes a novel attribute reduction algorithm for neighborhood decision systems. We introduce two novel metrics—the neighborhood evidential conflict degree (NECD) and neighborhood evidential conflict rate (NECR)—to compute heterogeneity between samples in the neighborhood and assess the significance of attributes in the feature space, respectively. These metrics assess the quality and selection of attribute subsets in attribute reduction, improving classification accuracy and computational efficiency. We also develop a sequentially forward selection attribute reduction method to select a feature subset through the defined NECR. Finally, we develop a distributed attribute reduction algorithm implemented in Apache Spark. Our approach involves a two-phase Map-Reduce process for K-Nearest Neighbors search, evidence combination, and NECR computation. NECR, as a measure of feature subset quality, enhances the feature subset's decision approximation capability of the data. Experimental results on small and large datasets demonstrate that the proposed algorithm outperforms benchmarking algorithms regarding classification accuracy and computational efficiency.

Three-way fusion measures and three-level feature selections based on neighborhood decision systems

Uncertainty measures exhibit algebraic and informational perspectives, and the two-view measure integration facilitates feature selections in classification learning. According to neighborhood decision systems (NDSs), two basic algorithms of feature selections (called JE-FS and DE-FS) already exist by using joint and decisional entropies, respectively, but they have advancement space for informationally fusing algebraic measures. In this paper on NDSs, three-way fusion measures are systematically constructed by combining three-way algebraic and informational measures, and thus three-level feature selections are hierarchically investigated by using corresponding monotonic and nonmonotonic measures and strategies. At first, the accuracy, granularity, and composite granularity-accuracy constitute three-way algebraic measures, while the joint, conditional, and decisional entropies (JE, CE, DE) formulate three-way informational measures. Then, three-way algebraic and informational measures are combined via normalization and multiplication, so three-way fusion measures based on JE, CE, DE are established. These new measures acquire granulation monotonicity and nonmonotonicity. Furthermore by relevant measures and monotonicity/nonmonotonicity, three-level feature selections (with null, single, and double fusion levels) related to JE, CE, DE are proposed, and corresponding heuristic algorithms are designed by monotonic and nonmonotonic principles. 4×3=12 selection algorithms comprehensively emerge, and they extend and improve current JE-FS and DE-FS. Finally by data experiments, related uncertainty measures and granulation properties are validated, and all 12 selection algorithms are compared in classification learning. As a result, new algorithms outperform JE-FS and DE-FS for classification performance, and the algorithmic improvements accord with the fusion-hierarchical deepening and entropy-systematic development of uncertainty measures.

Feature selection based on self-information and entropy measures for incomplete neighborhood decision systems

For incomplete datasets with mixed numerical and symbolic features, feature selection based on neighborhood multi-granulation rough sets (NMRS) is developing rapidly. However, its evaluation function only considers the information contained in the lower approximation of the neighborhood decision, which easily leads to the loss of some information. To solve this problem, we construct a novel NMRS-based uncertain measure for feature selection, named neighborhood multi-granulation self-information-based pessimistic neighborhood multi-granulation tolerance joint entropy (PTSIJE), which can be used to incomplete neighborhood decision systems. First, from the algebra view, four kinds of neighborhood multi-granulation self-information measures of decision variables are proposed by using the upper and lower approximations of NMRS. We discuss the related properties, and find the fourth measure-lenient neighborhood multi-granulation self-information measure (NMSI) has better classification performance. Then, inspired by the algebra and information views simultaneously, a feature selection method based on PTSIJE is proposed. Finally, the Fisher score method is used to delete uncorrelated features to reduce the computational complexity for high-dimensional gene datasets, and a heuristic feature selection algorithm is raised to improve classification performance for mixed and incomplete datasets. Experimental results on 11 datasets show that our method selects fewer features and has higher classification accuracy than related methods.

AFNFS: Adaptive fuzzy neighborhood-based feature selection with adaptive synthetic over-sampling for imbalanced data

The classification efficiency of majority classes for imbalanced data is so concerned in real-world applications. Almost fuzzy neighborhood radius still needs to be manually set and many entropy measures may ignore the boundary region of data, these limitations will result in the poor classification effect. To address these limitations, this paper designs a novel adaptive fuzzy neighborhood-based feature selection method for imbalanced data with adaptive synthetic over-sampling. First, the closeness is defined according to the variance distance between the samples of the minority class, the pair set of neighboring samples is designed, and then an improved adaptive synthetic over-sampling model is presented for constructing balanced decision systems consisting of the synthetic samples and original samples. Second, an adaptive fuzzy neighborhood radius is developed when using the data margins of all homogeneous and heterogeneous samples. Then the adaptive fuzzy neighborhood granule and upper and lower approximations are defined to construct a new FNRS model. Thus, approximate accuracy and roughness are presented to measure the uncertainty from the fuzzy and rough perspectives for imbalanced data. Third, by combining the roughness with adaptive fuzzy neighborhood entropy, adaptive fuzzy neighborhood joint entropy is constructed to evaluate the uncertainty in fuzzy neighborhood decision systems from two viewpoints of algebra and information. Then the reduced set and the significance of the feature are further developed. Finally, this improved adaptive synthetic over-sampling algorithm is designed to aim to build this balanced decision system, and an adaptive fuzzy neighborhood-based feature selection algorithm with the tolerance parameter is developed to achieve an optimal feature subset. Experiments on 26 imbalanced datasets demonstrate that the constructed algorithms compared to the other related algorithms are effective.

Two‐stage‐neighborhood‐based multilabel classification for incomplete data with missing labels

In recent years, it has been difficult for multilabel classification to obtain complete multilabel data in real-world applications, and even a large number of labels for training samples are randomly missed. As a result, the classification task of incomplete multilabel data with missing labels faces formidable challenges. This paper presents a two-stage-neighborhood-based multilabel classification method for incomplete data with missing labels in neighborhood decision systems. First, to solve the problem of selecting the neighborhood radius manually, as well as balancing the samples in the neighborhood, the neighborhood radius based on the feature distribution function is defined, and the differences and similarities between samples through the identifiable and indiscernible matrices are, respectively, computed. Then, a restoration method for missing feature values is proposed for use in the first stage. Second, to consider the nonlinear relationship among features, a neighborhood-based fuzzy similarity relationship between samples is investigated based on the Gaussian kernel function. By integrating the fuzzy similarity relationship matrix, label-specific feature matrix, and label correlation matrix, an objective function based on the regression model is presented, the optimal solutions to the label-specific feature and label correlation matrices based on the gradient descent strategy are provided, and a new multilabel classification method with missing labels is developed during the second stage. Finally, two-stage multilabel classification algorithms are designed. Experiments on 18 multilabel data sets demonstrate that our designed algorithms are effective not only for recovering missing feature values, but also for improving the classification performance of data with missing labels.

Multi-label feature selection based on fuzzy neighborhood rough sets

Multi-label feature selection, a crucial preprocessing step for multi-label classification, has been widely applied to data mining, artificial intelligence and other fields. However, most of the existing multi-label feature selection methods for dealing with mixed data have the following problems: (1) These methods rarely consider the importance of features from multiple perspectives, which analyzes features not comprehensive enough. (2) These methods select feature subsets according to the positive region, while ignoring the uncertainty implied by the upper approximation. To address these problems, a multi-label feature selection method based on fuzzy neighborhood rough set is developed in this article. First, the fuzzy neighborhood approximation accuracy and fuzzy decision are defined in the fuzzy neighborhood rough set model, and a new multi-label fuzzy neighborhood conditional entropy is designed. Second, a mixed measure is proposed by combining the fuzzy neighborhood conditional entropy from information view with the approximate accuracy of fuzzy neighborhood from algebra view, to evaluate the importance of features from different views. Finally, a forward multi-label feature selection algorithm is proposed for removing redundant features and decrease the complexity of multi-label classification. The experimental results illustrate the validity and stability of the proposed algorithm in multi-label fuzzy neighborhood decision systems, when compared with related methods on ten multi-label datasets.

Classification-level and Class-level Complement Information Measures Based on Neighborhood Decision Systems

Classification-level and Class-level Complement Information Measures Based on Neighborhood Decision Systems

Feature selection using Fisher score and multilabel neighborhood rough sets for multilabel classification

In recent years, feature selection for multilabel classification has attracted attention in machine learning and data mining. However, some feature selection methods ignore the correlations among labels, resulting in low performance, and most of them face challenges in determining an appropriate neighborhood radius for neighborhood systems and suffer from expensive time cost. To overcome the issues, we propose a novel feature selection method using Fisher score and multilabel neighborhood rough sets (MNRS) in multilabel neighborhood decision systems. First, to identify the correlations between labels under a binary distribution, two types of new mutual information between labels are considered, and their balance coefficients are defined. By enhancing strong correlations and weakening weak correlations between labels, a mutual information-based Fisher score model with a second-order correlation between labels is designed to fit multilabel data. Second, to address the problem of automatically choosing a neighborhood radius, a subset of heterogeneous and homogeneous samples is employed to develop a new classification margin as a neighborhood radius, and some concepts of neighborhood, neighborhood class, and upper and lower approximations are formulated for multilabel neighborhood decision systems. The weight and dependency degree are presented to effectively measure the uncertainty of samples in multilabel neighborhood decision systems. Thus, we further present a new classification margin-based MNRS model. Finally, a filter-wrapper preprocessing algorithm for feature selection using the improved Fisher score model is proposed to decrease the spatiotemporal complexity of multilabel data, and a heuristic feature selection algorithm is designed for improve classification performance on multilabel datasets. Experimental results on thirteen multilabel datasets show that the proposed algorithm is effective in selecting significant features, demonstrating its excellent classification ability in multilabel datasets.

Multilabel feature selection using ML-ReliefF and neighborhood mutual information for multilabel neighborhood decision systems

Feature selection as an essential preprocessing step in multilabel classification has been widely researched. Due to the diversity and complexity of multilabel datasets, some feature selection methods are unstable and yield low predictive accuracy. To address these issues, this paper presents a novel multilabel feature selection method using multilabel ReliefF (ML-ReliefF) and neighborhood mutual information in multilabel neighborhood decision systems. First, to solve the problem of the few available randomly selected samples when searching the nearest samples in ReliefF, the coefficient of difference and the average distance among the nearest similar and heterogeneous samples are introduced to evaluate the differences among the samples, and then the average differences among the similar or heterogeneous samples are defined. Using the Jaccard correlation coefficient, a new formula for updating feature weights is presented. Second, the margin of the sample is studied to granulate all samples under each label, and the concept of the neighborhood is given. By combining algebra with information views, some neighborhood entropy-based uncertainty measures for multilabel classification are investigated, and new neighborhood mutual information is proposed. Furthermore, an optimization objective function is constructed to evaluate the candidate features in multilabel neighborhood decision systems, all the properties are discussed, and the relationships of these measures are established. Finally, an improved ML-ReliefF algorithm is designed for preliminarily eliminating unrelated features to decrease the computational complexity for multilabel classification, and a heuristic forward multilabel feature selection algorithm is developed to remove redundant features and improve classification performance. Experimental results conducted on thirteen multilabel datasets to verify the effectiveness of the proposed algorithms in multilabel neighborhood decision systems are compared with representative methods.

Feature Selection Using Fuzzy Neighborhood Entropy-Based Uncertainty Measures for Fuzzy Neighborhood Multigranulation Rough Sets

For heterogeneous data sets containing numerical and symbolic feature values, feature selection based on fuzzy neighborhood multigranulation rough sets (FNMRS) is a very significant step to preprocess data and improve its classification performance. This article presents an FNMRS-based feature selection approach in neighborhood decision systems. First, some concepts of fuzzy neighborhood rough sets and neighborhood multigranulation rough sets are given, and then the FNMRS model is investigated to construct uncertainty measures. Second, the optimistic and pessimistic FNMRS models are built by using fuzzy neighborhood multigranulation lower and upper approximations from algebra view, and some fuzzy neighborhood entropy-based uncertainty measures are developed in information view. Inspired by both algebra and information views based on the FNMRS model, the fuzzy neighborhood pessimistic multigranulation entropy is proposed. Third, the Fisher score model is utilized to delete irrelevant features to decrease the complexity of high-dimensional data sets, and then, a forward feature selection algorithm is provided to promote the performance of heterogeneous data classification. Experimental results on 12 data sets show that the presented model is effective for selecting important features with the higher stability of classification in neighborhood decision systems.

Incremental updating probabilistic neighborhood three-way regions with time-evolving attributes

Neighborhood decision systems (NDSs) are commonly-used systems in real-world applications such as credit scoring. Probabilistic neighborhood rough sets (PNRSs) model, which is a generalized rough set model by combining neighborhood rough sets (NRSs) with probabilistic theory, can efficiently handle numerical data with noise values and allow tolerance of errors. Approximations of a target concept are fundamental and vital notions of the PNRSs, which can induce the certain and uncertain decision rules. Nevertheless, data evolves over time due to dynamic characteristics, especially, the addition of new attributes and the deletion of redundant or irrelevant attributes. Therefore, the potential meaningful knowledge may alter over time accordingly. To improve the efficiency of acquiring decision rules, the three-way regions (i.e., the positive, negative, and boundary regions) of the PNRSs need to be updated in an incremental fashion. To address this issue, two incremental algorithms for updating the three-way regions of each decision class of the PNRSs are investigated from the matrix perspective with time-evolving attributes (namely, adding attributes and deleting attributes). First, a matrix-based characterization of the three-way regions of each decision class of the PNRSs is constructed. Subsequently, considering the variation of the attributes, matrix-based updating mechanisms for relevant matrices are developed by means of the previously calculated information. Furthermore, the incremental algorithms are developed by taking advantage of the matrix-based updating mechanisms. Comprehensive experiments are performed on benchmark UCI data sets, and comparative results demonstrate our proposed algorithms consistently outperform the non-incremental algorithm in terms of computational efficiency.

Neighborhood multi-granulation rough sets-based attribute reduction using Lebesgue and entropy measures in incomplete neighborhood decision systems

For incomplete data with mixed numerical and symbolic attributes, attribute reduction based on neighborhood multi-granulation rough sets (NMRS) is an important method to improve the classification performance. However, most classical attribute reduction methods can only handle finite sets as to produce more attributes and lower classification accuracy. This paper proposes a novel NMRS-based attribute reduction method using Lebesgue and entropy measures in incomplete neighborhood decision systems. First, some concepts of optimistic and pessimistic NMRS models in incomplete neighborhood decision systems are given, respectively. Then, a Lebesgue measure is combined with NMRS to study neighborhood tolerance class-based uncertainty measures. To analyze the uncertainty, noise and redundancy of incomplete neighborhood decision systems in detail, some neighborhood multi-granulation entropy-based uncertainty measures are developed by integrating Lebesgue and entropy measures. Inspired by both algebraic view with information view in NMRS, the pessimistic neighborhood multi-granulation dependency joint entropy is proposed. What is more, the corresponding properties are further deduced and the relationships among these measures are discussed, which can help to investigate the uncertainty of incomplete neighborhood decision systems. Finally, the Fisher linear discriminant method is used to eliminate irrelevant attributes to significantly reduce computational complexity for high-dimensional datasets, and a heuristic attribute reduction algorithm with complexity analysis is designed to improve classification performance of incomplete and mixed datasets. Experimental results under seven UCI datasets and eight gene expression datasets illustrate that the proposed method is effective to select most relevant attributes with higher classification accuracy, as compared with representative algorithms.

Feature selection using Lebesgue and entropy measures for incomplete neighborhood decision systems

Feature selection for mixed and incomplete data in terms of numerical and categorical features with missing values has currently gained considerable attention. The development of the neighborhood rough sets-based feature selection method is an important step in improving classification performance, especially in incomplete data with mixed continuous numerical and categorical features. In this paper, a novel feature selection method based on the neighborhood rough sets using Lebesgue and entropy measures in incomplete neighborhood decision systems is proposed, and the method has the capacity to handle mixed and incomplete datasets; further, it can simultaneously maintain the original classification information. First, a Lebesgue measure based on the neighborhood tolerance class is developed to study the positive region and dependency degree. To thoroughly analyze the uncertainty, noise and incompleteness of incomplete neighborhood decision systems, some neighborhood tolerance entropy-based uncertainty measures are presented based on Lebesgue and entropy measures. Then, by combining an algebraic view with an information view in neighborhood rough sets, the neighborhood tolerance dependency joint entropy is defined in incomplete neighborhood decision systems. Moreover, all the corresponding properties are discussed, and the relationships among these measures are established to meaningfully convey the knowledge essence and investigate the uncertainty of incomplete neighborhood decision systems. Finally, for all high-dimensional datasets, the Fisher score method is used to preliminarily eliminate irrelevant features to significantly reduce the computational complexity, and a heuristic feature selection algorithm is designed to improve the classification performance of mixed and incomplete datasets. Experiments under an instance and fifteen public datasets demonstrate that the proposed feature selection method is effective in selecting the most relevant features, achieving great classification ability for incomplete neighborhood decision systems.

Feature selection using neighborhood entropy-based uncertainty measures for gene expression data classification

Gene expression data classification is an important technology for cancer diagnosis in bioinformatics and has been widely researched. Due to the large number of genes and the small sample size in gene expression data, feature selection based on neighborhood rough sets is a key step for improving the performance of gene expression data classification. However, some quantitative measures of feature sets may be nonmonotonic in neighborhood rough sets, and many feature selection methods based on evaluation functions yield high cardinality and low predictive accuracy. Therefore, investigating effective and efficient heuristic reduction algorithms is necessary. In this paper, a novel feature selection method based on neighborhood rough sets using neighborhood entropy-based uncertainty measures for cancer classification from gene expression data is proposed. First, some neighborhood entropy-based uncertainty measures are investigated for handling the uncertainty and noise of neighborhood decision systems. Then, to fully reflect the decision-making ability of attributes, the neighborhood credibility and neighborhood coverage degrees are defined and introduced into decision neighborhood entropy and mutual information, which are proven to be nonmonotonic. Moreover, some of the properties and relationships among these measures are derived, which is helpful for understanding the essence of the knowledge content and the uncertainty of neighborhood decision systems. Finally, the Fisher score method is employed to preliminarily eliminate irrelevant genes to significantly reduce complexity, and a heuristic feature selection algorithm with low computational complexity is presented to improve the performance of cancer classification using gene expression data. Experiments on ten gene expression datasets show that our proposed algorithm is indeed efficient and outperforms other related methods in terms of the number of selected genes and the classification accuracy, especially as the size of the genes increases.

An Attribute Reduction Method Using Neighborhood Entropy Measures in Neighborhood Rough Sets.

Attribute reduction as an important preprocessing step for data mining, and has become a hot research topic in rough set theory. Neighborhood rough set theory can overcome the shortcoming that classical rough set theory may lose some useful information in the process of discretization for continuous-valued data sets. In this paper, to improve the classification performance of complex data, a novel attribute reduction method using neighborhood entropy measures, combining algebra view with information view, in neighborhood rough sets is proposed, which has the ability of dealing with continuous data whilst maintaining the classification information of original attributes. First, to efficiently analyze the uncertainty of knowledge in neighborhood rough sets, by combining neighborhood approximate precision with neighborhood entropy, a new average neighborhood entropy, based on the strong complementarity between the algebra definition of attribute significance and the definition of information view, is presented. Then, a concept of decision neighborhood entropy is investigated for handling the uncertainty and noisiness of neighborhood decision systems, which integrates the credibility degree with the coverage degree of neighborhood decision systems to fully reflect the decision ability of attributes. Moreover, some of their properties are derived and the relationships among these measures are established, which helps to understand the essence of knowledge content and the uncertainty of neighborhood decision systems. Finally, a heuristic attribute reduction algorithm is proposed to improve the classification performance of complex data sets. The experimental results under an instance and several public data sets demonstrate that the proposed method is very effective for selecting the most relevant attributes with great classification performance.

A Neighborhood Rough Sets-Based Attribute Reduction Method Using Lebesgue and Entropy Measures.

For continuous numerical data sets, neighborhood rough sets-based attribute reduction is an important step for improving classification performance. However, most of the traditional reduction algorithms can only handle finite sets, and yield low accuracy and high cardinality. In this paper, a novel attribute reduction method using Lebesgue and entropy measures in neighborhood rough sets is proposed, which has the ability of dealing with continuous numerical data whilst maintaining the original classification information. First, Fisher score method is employed to eliminate irrelevant attributes to significantly reduce computation complexity for high-dimensional data sets. Then, Lebesgue measure is introduced into neighborhood rough sets to investigate uncertainty measure. In order to analyze the uncertainty and noisy of neighborhood decision systems well, based on Lebesgue and entropy measures, some neighborhood entropy-based uncertainty measures are presented, and by combining algebra view with information view in neighborhood rough sets, a neighborhood roughness joint entropy is developed in neighborhood decision systems. Moreover, some of their properties are derived and the relationships are established, which help to understand the essence of knowledge and the uncertainty of neighborhood decision systems. Finally, a heuristic attribute reduction algorithm is designed to improve the classification performance of large-scale complex data. The experimental results under an instance and several public data sets show that the proposed method is very effective for selecting the most relevant attributes with high classification accuracy.

Hybrid Multilabel Feature Selection Using BPSO and Neighborhood Rough Sets for Multilabel Neighborhood Decision Systems

Recently, feature selection for multilabel classification has attracted substantial attention in many fields; however, some of the available methods ignore the correlations among labels and yield low classification performance. In addition, most feature selection algorithms that are based on multilabel neighborhood rough sets (MNRS) can only deal with finite sets for multilabel data. To address the issues, this paper presents a novel hybrid filter-wrapper multilabel feature selection method that is based on binary particle swarm optimization (BPSO) and MNRS with the Lebesgue measure for multilabel neighborhood decision systems. First, to overcome the problem that the traditional correlation-based feature selection (CFS) algorithm ignores the dependencies among labels, two types of average correlation between single labels and label sets and among labels are presented. Via combination with information-entropy-based uncertainty measures, a new average correlation among labels is studied. A novel comprehensive evaluation function of CFS (NCFS) is constructed. Then, NCFS is introduced as a fitness function into the original BPSO and improved BPSO algorithms to optimize multilabel classification in the early and later stages, respectively, and the optimization process is terminated when the maximum number of iterations is reached. Next, the Lebesgue measure of the neighborhood class is developed for investigating the neighborhood approximation accuracy and the dependency degree based on MNRS. Various properties are deduced, and the relationships among these measures are used to evaluate the uncertainty and correlations among labels of multilabel data. Finally, a hybrid filter-wrapper feature selection algorithm using NCFS-BPSO is designed for preliminarily eliminating redundant features to decrease the complexity, and a heuristic forward multilabel feature selection algorithm is proposed for improving the performance of multilabel classification. Experimental results on fifteen multilabel datasets demonstrate that our proposed algorithms are effective in selecting significant features and realizing great classification performance in multilabel neighborhood decision systems.

A Gene selection approach based on the fisher linear discriminant and the neighborhood rough set

ABSTRACTIn recent years, tumor classification based on gene expression profiles has drawn great attention, and related research results have been widely applied to the clinical diagnosis of major gene diseases. These studies are of tremendous importance for accurate cancer diagnosis and subtype recognition. However, the microarray data of gene expression profiles have small samples, high dimensionality, large noise and data redundancy. To further improve the classification performance of microarray data, a gene selection approach based on the Fisher linear discriminant (FLD) and the neighborhood rough set (NRS) is proposed. First, the FLD method is employed to reduce the preliminarily genetic data to obtain features with a strong classification ability, which can form a candidate gene subset. Then, neighborhood precision and neighborhood roughness are defined in a neighborhood decision system, and the calculation approaches for neighborhood dependency and the significance of an attribute are given. A reduction model of neighborhood decision systems is presented. Thus, a gene selection algorithm based on FLD and NRS is proposed. Finally, four public gene datasets are used in the simulation experiments. Experimental results under the SVM classifier demonstrate that the proposed algorithm is effective, and it can select a smaller and more well-classified gene subset, as well as obtain better classification performance.