Neighborhood Rough Set Model Research Articles

Local fuzzy rough attribute reduction for large-scale mixed data with limited missing labels based on local fuzzy self information

The advent of the era of big data is accompanied by the generation of large-scale data of various types. Extracting the potential value and rules from such data has always been a challenge. Due to various external and internal factors, it is commonplace for large-scale data to exhibit the phenomenon of missing limited labels. In addressing a large-scale mixed information system with limited label missing (LSMDISLML), local neighborhood rough set model (LNRS-model) is typically employed. However, the identical neighborhood radius is often used by such model when confronted with numerical attributes, which could potentially attenuate the classification capability of the data. Local fuzzy rough set model (LFRS-model) can overcomes this point. This paper studies local fuzzy rough attribute reduction for large-scale mixed data with limited missing labels based on LFRS-model via local fuzzy self information and overlap degree function. First, leveraging the statistical distribution of data as a foundation, fuzzy relations on the entire sample set are established, which has the advantage of being able to use different fuzzy similarity radii to calculate similarity, thereby adapting to different data distributions. Subsequently, the samples with missing labels are discarded as they constitute a small proportion of the entire sample set and have little impact on overall performance of dataset. The limited computing resources and storage space are focused on the sample set with complete labels (denoted as target set). Thereafter, based on the target set, local fuzzy λ-upper and lower approximations are defined, and LFRS-model is constructed. This model not only reduces processing time and sources of error in large-scale data but also improves data quality and enhances the reliability of the experimental results. Then, local fuzzy λ-self information is introduced and used to design a local fuzzy rough attribute reduction algorithm in a LSMDISLML. Furthermore, a overlap degree function is introduced to evaluate and reorder the attributes based on their importance, prioritizing the elimination of redundant attributes with high overlap and low importance from the preordered attribute set. This strategy effectively improves the efficiency of obtaining the optimal subset. Finally, a series of experiments are carried out. The experiment results demonstrate that the designed algorithm exhibits excellent performance in classification tasks and outlier detection tasks, surpassing existing four algorithms.

Neighborhood margin rough set: Self-tuning neighborhood threshold

The neighborhood threshold in the neighborhood rough set has a significant impact on the neighborhood relation. When the neighborhood threshold of an object exceeds the critical value, the labels of objects in the neighborhood are not completely consistent, and the critical value of each object often differs. Most existing neighborhood rough set models cannot adaptively regulate the neighborhood threshold. In this paper, we introduce a novel neighborhood rough set model that incorporates a self-tuning mechanism for the neighborhood threshold, taking into account the distribution of objects across different areas. The neighborhood margin is a measure proposed to assess the condition of neighborhoods, and it is calculated by subtracting the neighborhood threshold from the closest distance between heterogeneous elements. The neighborhood margin accurately represents the local state of the neighborhood, taking into account decision information. The margin neighborhood is proposed with a self-tuning the neighborhood threshold. Finally, we introduce the margin neighborhood rough set model and margin neighborhood-based attribute reduction algorithm, and explore the relationship between the proposed model and the neighborhood rough set model. The experiment examines the performance of reducts under various measures, and demonstrates that the neighborhood margin rough set reduces the uncertainty of neighborhood granules effectively, leading to excellent classification performance compared to other neighborhood-based SOTA models.

Feature selection based on neighborhood complementary entropy for heterogeneous data

In practical applications, there exists a large amount of heterogeneous data consisting of a mixture of numerical and categorical data, which are more complex and difficult to handle than a single class. In recent years, feature selection for heterogeneous data has received more and more attention from scholars. However, the current research on feature selection for heterogeneous data seldom takes into account the fineness of the granularity structure and the complementary information contained in the complements of each granularity. Addressing the shortcomings of existing efforts, this paper introduces complementary entropy into the neighborhood rough set model to define an uncertainty measurement system in heterogeneous data, and constructs the corresponding feature selection algorithm. First, the neighborhood complementary entropy and its associated uncertainty measures based on neighborhood relation are defined, and the relevant theoretical properties of these uncertainty measures are investigated. Then, based on the proposed uncertainty measures, a significance function is defined to assess the importance of candidate features. Next, a feature selection method for heterogeneous data is designed using neighborhood complementary entropy. Finally, the feasibility of the method proposed in this paper is demonstrated through an experimental comparative analysis with various existing feature selection methods.

A novel adaptive neighborhood rough sets based on sparrow search algorithm and feature selection

Neighborhood rough sets-based methods have been widely used for feature selection. However, the existing methods have some problems in neighborhood construction, such as the application of the same neighborhood radius for all samples. Thus, this paper proposed a novel adaptive neighborhood rough set model based on Sparrow Search Algorithm (SSA) to tackle the above problems, and applied the model to feature selection. First, we reconsidered the problems mentioned above from the viewpoint of optimization where the neighborhood radius of the target sample is considered as the solution to the optimization, and the maximum percentage of the label of the neighborhood formed is considered as the target to the optimization. Second, SSA is introduced to design the adaptive neighborhood construction to tackle the optimization problem where all candidate neighborhood radii of the target sample are considered as sparrows, the maximum and minimum distances between the target sample and other samples are considered as the search range, and the maximum label rate defined in this paper is considered as the search target. Then, a novel adaptive neighborhood rough set model is proposed by using the adaptive neighborhood construction. Third, we proposed a feature selection algorithm based on the adaptive neighborhood rough set model. Finally, the experimental results on seventeen datasets demonstrate the effectiveness of our algorithm. The running time of the proposed algorithm is at least one time less than classical algorithms under the condition that the classification performance is better, the accuracy increases 3% and the balanced accuracy increases 4%.

Multi-label feature selection using self-information in divergence-based fuzzy neighborhood rough sets

Multi-label feature selection corresponds to pattern recognition and knowledge mining, and its application has been expanded to different scenarios. As an excellent processing platform for uncertain and ambiguous information, divergence-based fuzzy rough sets (Div-FRSs) have been proposed and applied to feature selection. However, there are three critical problems to be solved when applying Div-FRSs to multi-label learning. The first is how to effectively dispose the noise produced by features in multi-label data. The second is how to synthetically consider the relevance among all labels. The last is how to thoroughly mine the uncertainty brought by upper approximations neglected in Div-FRSs existing researches. To address these issues, this study presents a new divergence-based fuzzy neighborhood rough set model (Div-FNRSs) for multi-label learning using self-information. First, the divergence-based fuzzy neighborhood relation and class are gradually raised to manage the noise in multi-label data, and fuzzy decision is introduced to dispose all labels as a whole. Combining them together, a new model Div-FNRSs is constructed. Then, divergence-based fuzzy neighborhood self-information containing upper approximations and lower approximations is designed to depict distinguishing ability of features through three-level uncertainty measure establishment and granulation property exploration. Furthermore, feature significance for choosing the optimal features is given and it motivates a heuristic feature-selection algorithm DivFNSI-FS. Finally, data experiments are completed to validate DivFNSI-FS effectiveness with six state-of-the-art multi-label feature selection approaches on fourteen multi-label datasets. A conclusion can be drawn that DivFNSI-FS outperforms existing algorithms to obtain better performance on eight commonly-used evaluation indexes.

Fusing multiple interval-valued fuzzy monotonic decision trees

As a powerful knowledge mining technique for ordinal classification tasks, dominance-based rough set theory has many advantages but also some issues. Sensitivity to noisy information means that even a single mislabeled sample can lead to substantial fluctuations in the approximation calculations. Another issue is that most monotonic classifiers can only handle real-valued data and cannot directly deal with interval-valued data, which is common in practical applications. To address these issues, a tree-based fusion learning method for monotonic classification of interval-valued attributes, named FM-IFMDT, has been proposed. Its functional structure comprises three components: (i) The proposed robust β-precision interval-valued fuzzy dominance neighborhood rough set model (β-IFDNRS) can adaptively identify and filter noise samples. (ii) The interval dominance discernibility matrix based on β-IFDNRS is developed for feature selection and can generate a set of complete and diverse feature subsets. (iii) The novel interval-valued fuzzy monotonic decision tree (IFMDT) based on the probability distribution is trained on each feature subset and is used as the base classifier of the weighted voting fusion model. Extensive experiments show the proposed fusion learning method has significant advantages.

Attribute reduction for heterogeneous data based on monotonic relative neighborhood granularity

The neighborhood rough set model serves as an important tool for handling attribute reduction tasks involving heterogeneous attributes. However, measuring the relationship between conditional attributes and decision in the neighborhood rough set model is a crucial issue. Most studies have utilized neighborhood information entropy to measure the relationship between attributes. When using neighborhood conditional information entropy to measure the relationships between the decision and conditional attributes, it lacks monotonicity, consequently affecting the rationality of the final attribute reduction subset. In this paper, we introduce the concept of neighborhood granularity and propose a new form of relative neighborhood granularity to measure the relationship between the decision and conditional attributes, which exhibits monotonicity. Moreover, our approach for measuring neighborhood granularity avoids the logarithmic function computation involved in neighborhood information entropy. Finally, we conduct comparative experiments on 12 datasets using two classifiers to compare the results of attribute reduction with six other attribute reduction algorithms. The comparison demonstrates the advantages of our measurement approach.

Distance metric learning-based multi-granularity neighborhood rough sets for attribute reduction

Attribute reduction is a hot research topic in data mining, in which rough set theory-based attribute reduction methods have been widely focused. The neighborhood rough set (NRS) model has good generalization performance and practicality in uncertainty reasoning, so it is often used for attribute reduction in recent years. Calculating the distance between samples in different attribute spaces is a key step in the NRS-based attribute reduction methods, which directly affects the performance of the reduction algorithm. However, the NRS model uses a fixed computational paradigm in calculating the distance between samples and does not consider the influence of labels in the attribute spaces on the distance calculation, which is not conducive to improving the performance of the reduction algorithm. Distance metric learning takes full account of the labels information in the multi-dimensional attribute space, and it can learn the distance between samples by taking into account the integrated principle that samples with the same label are closer and samples with different labels are further away, which helps to reduce the classification uncertainty. Inspired by this, this paper firstly incorporates distance metric learning into the NRS model from the perspective of multi-dimensional attribute space and proposes a distance metric learning-based multi-granularity neighborhood rough set (DmlMNRS) model. The related properties of the DmlMNRS model are also introduced and proved. Then, the DmlMNRS-based attribute reduction criterion and the significance of the attributes are defined. A DmlMNRS-based heuristic attribute reduction (DMNHAR) algorithm is designed based on this. Finally, experiments are performed on fifteen publicly datasets, and the experimental results show that the proposed algorithm has better robustness and classification performance.

Adaptive neighborhood rough set model for hybrid data processing: a case study on Parkinson’s disease behavioral analysis

Extracting knowledge from hybrid data, comprising both categorical and numerical data, poses significant challenges due to the inherent difficulty in preserving information and practical meanings during the conversion process. To address this challenge, hybrid data processing methods, combining complementary rough sets, have emerged as a promising approach for handling uncertainty. However, selecting an appropriate model and effectively utilizing it in data mining requires a thorough qualitative and quantitative comparison of existing hybrid data processing models. This research aims to contribute to the analysis of hybrid data processing models based on neighborhood rough sets by investigating the inherent relationships among these models. We propose a generic neighborhood rough set-based hybrid model specifically designed for processing hybrid data, thereby enhancing the efficacy of the data mining process without resorting to discretization and avoiding information loss or practical meaning degradation in datasets. The proposed scheme dynamically adapts the threshold value for the neighborhood approximation space according to the characteristics of the given datasets, ensuring optimal performance without sacrificing accuracy. To evaluate the effectiveness of the proposed scheme, we develop a testbed tailored for Parkinson’s patients, a domain where hybrid data processing is particularly relevant. The experimental results demonstrate that the proposed scheme consistently outperforms existing schemes in adaptively handling both numerical and categorical data, achieving an impressive accuracy of 95% on the Parkinson’s dataset. Overall, this research contributes to advancing hybrid data processing techniques by providing a robust and adaptive solution that addresses the challenges associated with handling hybrid data, particularly in the context of Parkinson’s disease analysis.

Dynamic multi-label feature selection algorithm based on label importance and label correlation

Multi-label distribution is a popular direction in current machine learning research and is relevant to many practical problems. In multi-label learning, samples are usually described by high-dimensional features, many of which are redundant or invalid. This paper proposes a multi-label static feature selection algorithm to solve the problems caused by high-dimensional features of multi-label learning samples. This algorithm is based on label importance and label relevance, and improves the neighborhood rough set model. One reason for using neighborhood rough sets is that feature selection using neighborhood rough sets does not require any prior knowledge of the feature space structure. Another reason is that it does not destroy the neighborhood and order structure of the data when processing multi-label data. The method of mutual information is used to achieve the extension from single labels to multiple labels in the multi-label neighborhood; through this method, the label importance and label relevance of multi-label data are connected. In addition, in the multi-label task scenario, features may be interdependent and interrelated, and features often arrive incrementally or can be extracted continuously; we call these flow features. Traditional static feature selection algorithms do not handle flow features well. Therefore, this paper proposes a dynamic feature selection algorithm for flow features, which is based on previous static feature selection algorithms. The proposed static and dynamic algorithms have been tested on a multi-label learning task set and the experimental results show the effectiveness of both algorithms.

A local rough set method for feature selection by variable precision composite measure

Feature selection using variable precision neighborhood rough sets (VPNRS) has garnered considerable attention in data mining and knowledge discovery. Nevertheless, the positive region of VPNRS may not be strictly divided due to the introduction of variable parameters, which could reduce the credibility of feature significance. Meanwhile, the calculation of approximate space is also complex and expensive. Hence, how to improve the computation efficiency is also an investigated issue. As for these issues, we propose a variable precision composite measure and design a novel local method for the feature selection of decision data. Firstly, we introduce the variable precision neighborhood rough set model to process uncertain information from global and local viewpoints. Furthermore, the variable precision composite measure is defined to evaluate the model’s accuracy and further used to select the essential features. Finally, a local forward algorithm is provided for feature selection to improve computing efficiency. All experiments on twelve datasets show that the local method is efficient, and the feature selection algorithm based on variable precision composite measure performs well in classification performance. Our work will provide a convenient tool for feature selection methods with uncertainty measures.

Heterogeneous Feature Selection Based on Neighborhood Combination Entropy.

Feature selection aims to remove irrelevant or redundant features and thereby remain relevant or informative features so that it is often preferred for alleviating the dimensionality curse, enhancing learning performance, providing better readability and interpretability, and so on. Data that contain numerical and categorical representations are called heterogeneous data, and they exist widely in many real-world applications. Neighborhood rough set (NRS) can effectively deal with heterogeneous data by using neighborhood binary relation, which has been successfully applied to heterogeneous feature selection. In this article, the NRS model as a unified framework is used to design a feature selection method to handle categorical, numerical, and heterogeneous data. First, the concept of neighborhood combination entropy (NCE) is presented. It can reflect the probability of pairs of the neighborhood granules that are probably distinguishable from each other. Then, the conditional neighborhood combination entropy (cNCE) based on NCE is proposed under the condition of considering decision attributes. Moreover, some properties and relationships between cNCE and NCE are derived. Finally, the functions of inner and outer significances are constructed to design a feature selection algorithm based on cNCE (FScNCE). The experimental results show the effectiveness and superiority of the proposed algorithm.

Feature Selection for Unbalanced Distribution Hybrid Data Based on ${k}$-Nearest Neighborhood Rough Set

Neighborhood rough sets are now widely used to process numerical data. Nevertheless, most of the existing neighborhood rough sets are not able to distinguish class mixture samples well when dealing with classification problems. That is, it cannot effectively classify categories when dealing with data with an unbalanced distribution. Because of this, in this aiticle, we propose a new feature selection method that takes into consideration both heterogeneous data and feature interaction. The proposed model well integrates the ascendancy of δ-neighborhood and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -nearest neighbor. Such heterogeneous data can be handled better than existing neighborhood models. We utilize information entropy theories such as mutual information and conditional mutual information, and employ an iterative strategy to define the importance of each feature in decision-making. Furthermore, we design a feature extraction algorithm based on above idea. Experimental results displays that the raised algorithm has superior effect than some existing algorithms, particularly the δ-neighborhood rough set model and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -nearest neighborhood rough set model.

Information fusion and attribute reduction for multi-source incomplete mixed data via conditional information entropy and D-S evidence theory

Multi-source incomplete mixed data abound in real life, like medical data, biological data, remote sensing data, military data, etc. However, some of these sources are of less importance than others, and some are essentially worthless. Therefore, their information fusion and attribute reduction face many challenges. This paper studies information fusion for multi-source incomplete mixed data via conditional information entropy (CIE) and considers its application to attribute reduction based on D-S evidence theory. First of all, a new distance function is defined to measure the difference between nominal attribute values with missing information, and the neighborhood rough set model is used to establish the granularity structure of multi-source incomplete mixed data. Then, a source selection method is given via CIE, which is used to fuse multi-source incomplete mixed data into single-source incomplete mixed data. Based on the maximization of CIE, this method allows worthy and reliable information sources to be chosen. Next, the connection between neighborhood rough set model and D-S evidence theory is established. Moreover, two attribute reduction algorithms for the fused incomplete mixed data are proposed based on the belief and plausibility. Finally, experiments are done to verify the effectiveness of the proposed fusion and reduction algorithms. The results of experiment and statistical test on 12 datasets show that the proposed algorithms exceed other advanced algorithms in classification performance.

A new method for feature selection based on weighted [formula omitted]-nearest neighborhood rough set

The neighborhood rough set theory is a helpful instrument for working with data that is numerical, and the performance of its uncertainty measures is generally stable. Even one noisy sample may result in several samples into the border domain because of the partitioning approach for upper approximation and lower approximation in neighborhood rough sets, which is particularly sensitive to incorrectly labeled samples. Based on the distributional information about the data in the sample area, the Weighted k-nearest Neighborhood Rough Set (WKNRS) model was proposed as a solution to this problem. Firstly, we weight the neighbor samples using the standard deviation of pertaining class labels before offering a sample quality evaluation. Next, using this sample quality evaluation, a WKNRS is proposed, which fixes the neighborhood rough set’s noise sensitivity issue. Then, a Weighted k-nearest Neighbor Features Selection(WKNFS) is generated using this model, along with a forward greedy search methodology. Finally, because we wanted to assess this method’s efficiency, we put it to the test using 16 UCI datasets. The results of the trials demonstrate that this algorithm performs better than the other 6 algorithms of feature selection currently in use.

GRRS: Accurate and Efficient Neighborhood Rough Set for Feature Selection

Feature selection is an important preprocessing step in data mining and pattern recognition. The neighborhood rough set (NRS) model is a widely-used rough set model for feature selection on continuous data. All currently known NRS models are defined on a distance metric — mostly the Euclidean distance metric — which invalidates the NRS models in scenarios wherein the Euclidean distance is ineffective, for example, while considering differing attribute weights. We first introduce the concept of space division of granular-rectangular, and then construct the neighborhood radius in our method by describing the relationship between child and parent spaces, which avoids the use of a distance metric and reduces the search space for the neighborhood radius. This greatly improves both the accuracy and efficiency of NRS. In addition, the upper and lower approximations of the granular-rectangular rough sets (GRRSs) comprise equivalence classes; this results in better performance of GRRS in knowledge representation compared with the traditional NRS. Experimental results on public benchmark datasets reveal that our method, GRRS, achieves higher accuracy than ten popular and state-of-the-art feature-selection methods, including two NRS algorithms. Moreover, GRRS outperforms the established NRS algorithms regarding efficiency, including the state-of-the-art NRS algorithm, GBNRS. All code has been released as an open libary called GRRS: https://github.com/syxiaa/GRRS.

An Emerging Fuzzy Feature Selection Method Using Composite Entropy-Based Uncertainty Measure and Data Distribution

Feature selection based on neighborhood rough set is a noteworthy step in dealing with numerical data. Information entropy, proven in many theoretical analysis and practical applications, is a compelling feature evaluation method for uncertainty information measures. Nonetheless, information entropy replaces probability with uncertainty measure to evaluate the average amount of information and ignores the decision distribution of data, especially in describing the uncertainty in imbalanced data. This paper discusses an emerging method for the feature selection in fuzzy data with imbalanced data by presenting a local composite entropy based on a neighborhood rough set. Based on the neighborhood rough set model, we discuss a similar relation to describe the relationship between different objects in unbalanced fuzzy data. In this process, to fully consider the distribution characteristics of unbalanced data, we construct a local composite entropy for handling the fuzzy decision systems with uncertainty and decision distribution, which is proven to be monotonic. Moreover, to improve the selection efficiency, a local heuristic forward greedy selection algorithm based on the local composite measure is designed to select the optimal feature subset. Finally, experimental results on twelve public datasets demonstrate that our method has better classification performance than some state-of-the-art feature selection methods in fuzzy data.

A soft neighborhood rough set model and its applications

Neighborhood rough set theory is widely used to measure the uncertainty of data in machine learning and data mining. However, the neighborhood radius has a significant influence on the effectiveness and robustness of the models and algorithms based on this theory. To address this problem, the soft-margin theory is introduced into neighborhood rough sets to define a soft neighborhood rough set model that can adaptively determine the appropriate neighborhood radius for each sample. The model effectively reduces the influence of the neighborhood radius on the uncertainty measure. Specific properties and theoretical analysis of the new model are presented. Based on soft neighborhood rough sets, this study presents a feature selection algorithm and classifier. The experimental results demonstrate that the designed algorithms exhibit acceptable performance, confirming that the soft neighborhood rough set model is feasible and robust.

Multi-label feature selection based on label distribution and neighborhood rough set

Multi-label feature selection is an indispensable technology in multi-semantic high-dimensional data preprocessing, which has been brought into focus in recent years. However, most existing methods explicitly assume that the significance of all relevant labels is the same for every instance, while ignoring the real scenarios that the significance of available labels to each instance is usually different. In this paper, we propose a novel multi-label feature selection based on label distribution and neighborhood rough set, known as LDRS. To be specific, we first construct a label enhancement method based on instance information distribution to convert the logical labels of multi-label data into label distribution, thereby capturing label significance to provide additional information for learning tasks. Then, we extend the neighborhood rough set model for label distribution learning, and discuss the related properties in detail. This extended model can effectively avoid the selection of neighborhood granularity and seamlessly apply to handle label distribution data. After that, two feature significance measures are established to realize the quality evaluation of features and the fusion of label-specific features. Finally, a novel feature selection framework is designed, which takes into account feature significance, label significance, and label-specific features, simultaneously. Experiments on both public and real-world datasets exhibit the advantages of the proposed method.

Noise-resistant multilabel fuzzy neighborhood rough sets for feature subset selection

Feature selection attempts to capture the more discriminative features and plays a significant role in multilabel learning. As an efficient mathematical tool to handle incomplete and uncertain information, fuzzy rough set theory has been widely used in feature selection. However, there are two pivotal issues to be addressed when adopting fuzzy rough sets for multilabel data analysis. One is how to effectively characterize the discriminative ability of the features to multilabel samples. Another one is how to alleviate the effect of data noise on feature selection. For these reasons, this study mainly studies the noise-tolerant fuzzy neighborhood rough set model for multilabel learning and its feature selection strategy. Firstly, a parameterized hybrid fuzzy similarity relation is introduced to granulate multilabel data, and the parameterized fuzzy decision is extended to multilabel learning. Then, a noise-resistant multilabel fuzzy neighborhood rough set model using inclusion relation is constructed for describing the discriminative ability of the features to multilabel samples. Furthermore, a noise-resistant heuristic feature selection algorithm titled NRFSFN is designed. Besides, motivated by local sampling and ensemble strategy, an efficient extended version of NRFSFN is designed from the perspective of data distribution, named ENFSFN. Finally, a series of comparison experiments on sixteen multilabel datasets demonstrate the effectiveness of the proposed two algorithms.