Incremental Feature Selection Algorithm Research Articles

Matrix-based incremental feature selection method using weight-partitioned multigranulation rough set

Incremental feature selection methods have gained increasing research attention as they improve the efficiency of feature selection for dynamic datasets. Multigranulation rough set, as an extension of rough set theory, allows for a comprehensive and rational analysis of problems from multiple hierarchical and granular perspectives. However, existing research on granularity partitioning relies on the decision maker's subjective experience, which lacks convincing power. In this paper, we propose a generalized multigranulation neighborhood rough set based on weight partition model, using a matrix form. We discuss several properties and define a new entropy measure to evaluate feature importance. A heuristic feature selection algorithm is developed based on this entropy to search for the optimal subset. Furthermore, we discuss dynamic updating mechanism and design two incremental feature selection algorithms. Finally, we conduct experiments on 12 public datasets to evaluate the performance of the proposed algorithms and validate their effectiveness and efficiency in feature selection for both static and dynamic datasets.

An incremental feature selection approach for dynamic feature variation

In numerous domains, there is ample evidence indicating that a significant portion of real-world data exhibits temporal variations, such as medical research and meteorological studies. Particularly in the era of big data, not only does the size of data change dynamically but also its dimensions at an unprecedented pace. Consequently, employing traditional methods to handle such dynamic data becomes highly impractical. To address this limitation, this paper proposes an incremental feature selection algorithm tailored for dynamic feature variations. For scenarios involving an increase in features, the novel algorithm efficiently identifies a target subset with effective features. In order to showcase the efficacy of the proposed algorithm, this paper conduct experiments using four commonly used classifiers and five UCI data sets. The experimental results further validate both the feasibility and efficiency of the new approach.

Incremental feature selection for dynamic incomplete data using sub-tolerance relations

Tolerance Rough Set (TRS) theory is commonly employed for feature selection with incomplete data. However, TRS has limitations such as ignoring uncertainty, which often leads to the inclusion of redundant features and diminished classification accuracy. To address these limitations, we propose an extension called Subrelation Tolerance Class (STC). STC decomposes the tolerance relation into two subrelations, enabling a two-stage certainty measurement. This approach progressively filters out certain regions, thereby reducing computational space requirements, and introduces a new significance measure that considers both certain and uncertain information. Leveraging STC and our proposed measure, we develop an incremental feature selection algorithm capable of handling incomplete streaming data. We conduct experiments on real-world datasets and compare the performance with existing algorithms to validate the superiority of our method. The experimental results show that our algorithm reduces the execution time by over 89.78% compared to the baselines while maintaining the classification accuracy.

PseU-ST: A new stacked ensemble-learning method for identifying RNA pseudouridine sites.

Background: Pseudouridine (Ψ) is one of the most abundant RNA modifications found in a variety of RNA types, and it plays a significant role in many biological processes. The key to studying the various biochemical functions and mechanisms of Ψ is to identify the Ψ sites. However, identifying Ψ sites using experimental methods is time-consuming and expensive. Therefore, it is necessary to develop computational methods that can accurately predict Ψ sites based on RNA sequence information. Methods: In this study, we proposed a new model called PseU-ST to identify Ψ sites in Homo sapiens (H. sapiens), Saccharomyces cerevisiae (S. cerevisiae), and Mus musculus (M. musculus). We selected the best six encoding schemes and four machine learning algorithms based on a comprehensive test of almost all of the RNA sequence encoding schemes available in the iLearnPlus software package, and selected the optimal features for each encoding scheme using chi-square and incremental feature selection algorithms. Then, we selected the optimal feature combination and the best base-classifier combination for each species through an extensive performance comparison and employed a stacking strategy to build the predictive model. Results: The results demonstrated that PseU-ST achieved better prediction performance compared with other existing models. The PseU-ST accuracy scores were 93.64%, 87.74%, and 89.64% on H_990, S_628, and M_944, respectively, representing increments of 13.94%, 6.05%, and 0.26%, respectively, higher than the best existing methods on the same benchmark training datasets. Conclusion: The data indicate that PseU-ST is a very competitive prediction model for identifying RNA Ψ sites in H. sapiens, M. musculus, and S. cerevisiae. In addition, we found that the Position-specific trinucleotide propensity based on single strand (PSTNPss) and Position-specific of three nucleotides (PS3) features play an important role in Ψ site identification. The source code for PseU-ST and the data are obtainable in our GitHub repository (https://github.com/jluzhangxinrubio/PseU-ST).

Incremental Feature Selection Using a Conditional Entropy Based on Fuzzy Dominance Neighborhood Rough Sets

Incremental feature selection approaches can improve the efficiency of feature selection used for dynamic datasets, which has attracted increasing research attention. Nevertheless, there is currently no work on incremental feature selection approaches for dynamic ordered data. Moreover, the monotonic classification effect of ordered data is easily affected by noise, so a robust feature evaluation metric is needed for feature selection algorithm. Motivated by these two issues, we investigate incremental feature selection approaches using a new conditional entropy with robustness for dynamic ordered data in this study. First, we propose a new rough set model, i.e., fuzzy dominance neighborhood rough sets (FDNRS). Second, a conditional entropy with robustness is defined based on FDNRS model, which is used as evaluation metric for features and combined with a heuristic feature selection algorithm. Finally, two incremental feature selection algorithms are designed on the basis of the above researches. Experiments are performed on ten public datasets to evaluate the robustness of the proposed metric and the performance of the incremental algorithms. Experimental results verify that the proposed metric is robust and our incremental algorithms are effective and efficient for updating reducts in dynamic ordered data.

Incremental feature selection by sample selection and feature-based accelerator

Incremental feature selection is an efficient paradigm that updates an optimal feature subset from added-in data without forgetting the previously learned knowledge. Most existing studies of rough set-based incremental feature selection require scanning all added-in samples and all possible candidate features when determining a best feature. However, such a classical search strategy has to perform some redundant calculations, which increase the computing and memory space resources. To avoid the redundant calculations, we propose a novel incremental feature selection method using sample selection and feature-based accelerator. First, a feature selection framework based on discernibility score is proposed as basis for our incremental method. Second, sample selection scheme is proposed to eliminate useless samples from added-in data. This scheme ensures that only useful samples are considered in the incremental process. Third, feature-based accelerator is designed to incrementally select a best feature and simultaneously remove redundant candidate features. It is theoretically guaranteed redundant features removed earlier remain redundant and will not be reexamined during the rest of the process. Finally, our incremental feature selection algorithm is designed by a two-stage procedure including sample selection scheme and feature-based accelerator. The results of experiments validate the time efficiency of the proposed incremental algorithm, especially on datasets with numerous instances or high dimensions.

Matrix representation of the conditional entropy for incremental feature selection on multi-source data

In many real applications, the data are always collected from different information sources and are subject to evolve over time. Such data are referred to as dynamic multi-source data. How to efficiently select the informative features from dynamic multi-source data is a challenging problem in data mining. Incremental feature selection with rough sets is an effective method to select features from dynamic data. However, existing methods focus on single-source data and are not suitable for dynamic multi-source data with variations in data sources. To deal with this issue, we present an incremental feature selection method based on the matrix representation of the conditional entropy. We first propose a novel conditional entropy for multi-source data and discuss its properties, including the monotonicity and boundedness. Then, matrix characterization of the conditional entropy is presented by employing the condition and decision relation matrices associated with some matrix operators. Finally, considering the addition and deletion of data sources in multi-source data, we employ the matrix approach to investigate the incremental mechanisms for the computation of the conditional entropy and develop the corresponding incremental feature selection algorithms. Extensive comparative experimental results are obtained to verify the effectiveness and efficiency of the proposed method.

Information granularity-based incremental feature selection for partially labeled hybrid data

Feature selection can reduce the dimensionality of data effectively. Most of the existing feature selection approaches using rough sets focus on the static single type data. However, in many real-world applications, data sets are the hybrid data including symbolic, numerical and missing features. Meanwhile, an object set in the hybrid data often changes dynamically with time. For the hybrid data, since acquiring all the decision labels of them is expensive and time-consuming, only small portion of the decision labels for the hybrid data is obtained. Therefore, in this paper, incremental feature selection algorithms based on information granularity are developed for dynamic partially labeled hybrid data with the variation of an object set. At first, the information granularity is given to measure the feature significance for partially labeled hybrid data. Then, incremental mechanisms of information granularity are proposed with the variation of an object set. On this basis, incremental feature selection algorithms with the variation of a single object and group of objects are proposed, respectively. Finally, extensive experimental results on different UCI data sets demonstrate that compared with the non-incremental feature selection algorithms, incremental feature selection algorithms can select a subset of features in shorter time without losing the classification accuracy, especially when the group of objects changes dynamically, the group incremental feature selection algorithm is more efficient.

Screening of Serum miRNAs as Diagnostic Biomarkers for Lung Cancer Using the Minimal-Redundancy-Maximal-Relevance Algorithm and Random Forest Classifier Based on a Public Database

Background: Lung cancer is one of the deadliest cancers, early diagnosis of which can efficiently enhance patient’s survival. We aimed to screening out the serum miRNAs as diagnostic biomarkers for patients with lung cancer. Methods: A total of 416 remarkably differentially expressed miRNAs were acquired using the limma package, and next feature ranking was derived by the minimal-redundancy-maximal-relevance method. An incremental feature selection algorithm of a random forest (RF) classifier was utilized to choose the top 5 miRNA combination with the optimum predictive performance. The performance of the RF classifier of top 5 miRNAs was analyzed using the receiver operator characteristic (ROC) curve. Afterward, the classification effect of the 5-miRNA combination was validated through principal component analysis and hierarchical clustering analysis. Analysis of top 5 miRNA expressions between lung cancer patients and normal people was performed based on GSE137140 dataset, and their expression was validated by qPCR. The hierarchical clustering analysis was used to analyze the similarity of 5 miRNAs expression profiles. ROC analysis was undertaken on each miRNA. Results: We acquired top 5 miRNAs finally, with the Matthews correlation coefficient value as 0.988 and the area under the curve (AUC) value as 0.996. The 5 feature miRNAs were capable of distinguishing most cancer patients and normal people. Furthermore, except for the lowly expressed miR-6875-5p in lung cancer tissue, the other 4 miRNAs all expressed highly in cancer patients. Performance analysis revealed that their AUC values were 0.92, 0.96, 0.94, 0.95, and 0.93, respectively. Conclusion: By and large, the 5 feature miRNAs screened here were anticipated to be effective biomarkers for lung cancer.

Feature selection for dynamic interval-valued ordered data based on fuzzy dominance neighborhood rough set

Incremental learning strategy based feature selection approaches can improve the efficiency of reduction algorithm used for datasets with dynamic characteristic, which has attracted increasing research attention. Nevertheless, there is currently no work on incremental feature selection approaches for dynamic interval-valued ordered data. Interval-valued ordered data is a generalized form of single-valued ordered data, which is more widely used in practice. However, the endpoints of the interval numbers are easily polluted by noise, thereby the knowledge granules are very sensitive. Motivated by these two issues, we study incremental feature selection approaches based on a fuzzy dominance neighborhood rough set (FDNRS) for dynamic interval-valued ordered data in this work. First, we propose the FDNRS model for an interval-valued ordered decision system (IvODS) and investigate its related properties. Second, a conditional entropy with robustness is proposed based on the proposed model. This conditional entropy can measure the degree of monotonic consistency of the IvODS, so it is used as a metric and combined with a heuristic feature selection algorithm. Finally, two incremental feature selection algorithms are proposed on the basis of the above researches. Experiments are performed on nine public datasets to evaluate the robustness of the proposed metric and the performance of the incremental algorithms. Experimental results verify that the proposed metric is robust and our incremental algorithms are effective and efficient for updating reducts in dynamic IvODS.

RIFS2D: A two-dimensional version of a randomly restarted incremental feature selection algorithm with an application for detecting low-ranked biomarkers

The era of big data introduces both opportunities and challenges for biomedical researchers. One of the inherent difficulties in the biomedical research field is to recruit large cohorts of samples, while high-throughput biotechnologies may produce thousands or even millions of features for each sample. Researchers tend to evaluate the individual correlation of each feature with the class label and use the incremental feature selection (IFS) strategy to select the top-ranked features with the best prediction performance. Recent experimental data showed that a subset of continuously ranked features randomly restarted from a low-ranked feature (an RIFS block) may outperform the subset of top-ranked features. This study proposed a feature selection Algorithm RIFS2D by integrating multiple RIFS blocks. A comprehensive comparative experiment was conducted with the IFS, RIFS and existing feature selection algorithms and demonstrated that a subset of low-ranked features may also achieve promising prediction performance. This study suggested that a prediction model with promising performance may be trained by low-ranked features, even when top-ranked features did not achieve satisfying prediction performance. Further comparative experiments were conducted between RIFS2D and t-tests for the detection of early-stage breast cancer. The data showed that the RIFS2D-recommended features achieved better prediction accuracy and were targeted by more drugs than the t-test top-ranked features.

Incremental approaches for heterogeneous feature selection in dynamic ordered data

Feature selection can identify essential features and reduce the dimensionality of features, improving the classification ability of a learning model. In this study, we consider data with a preference-order relation, i.e., ordered data. In the big data era, ordered data contain noise and exhibit heterogeneous features (including numerical and categorical features) and dynamic characteristics (i.e., new objects are added and obsolete objects are removed with evolving time). The dominance-based neighborhood rough set (DNRS) considers the preference order relation of heterogeneous features and demonstrates fault tolerance; thus, it can be applied well to heterogeneous feature selection in ordered data. At present, DNRS-based heterogeneous feature selection methods are only used for static ordered data. For dynamic ordered data, existing heterogeneous feature selection approaches are highly time-consuming because they are required to recalculate knowledge from scratch when multiple objects vary. Motivated by this issue, we utilize a matrix-based method in this work to study incremental heterogeneous feature selection based on DNRS in dynamic ordered data. First, we define neighborhood dominance conditional entropy (NDCE) as the uncertainty measure and introduce a non-monotonic feature selection strategy based on this measure. Second, the neighborhood dominance relation matrix and its diagonal matrix are defined to calculate NDCE in matrix form. Third, the updating mechanisms of the diagonal matrix are studied when objects vary and used to update NDCE. Lastly, two incremental feature selection algorithms are proposed when multiple objects are added to or deleted from heterogeneous ordered data. Experiments are performed on public data sets. Experimental results verify that the proposed incremental algorithms are effective and efficient for updating feature subsets in dynamic heterogeneous ordered data.

An effective disease prediction system using incremental feature selection and temporal convolutional neural network

Rapid growth of communication technologies and expert systems produces enormous volume of medical data. Deep learning technique is an advancement of machine learning technique for analysing the huge amount of various diseases related medical dataset. Even though, no healthcare systems are achieved better prediction accuracy with the various medical datasets in the past decades. For improving the accuracy level of disease prediction, we develop a disease prediction system to predict the serious death diseases including heart disease, diabetic disease and cancer diseases effectively in this paper. This disease prediction system consists of feature selection method that works as incremental in nature named as Incremental Feature Selection Algorithm (IFSA) which combines the concepts of Intelligent Conditional Random Field (CRF) on feature selection process and the Linear Correlation Coefficient based Feature Selection (ICRF-LCFS) method algorithm and an existing Convolutional Neural Network (CNN) with temporal features (T-CNN). The proposed disease prediction system is evaluated and achieved better prediction accuracy in less time with low false alarm rate.

Incremental feature selection for dynamic hybrid data using neighborhood rough set

Feature selection with rough sets aims to delete redundant conditional features from static data by considering single type features. However, traditional feature selection methods generally ignore real-world scenarios: hybrid conditional feature set including missing, categorical and numerical ones coexists in the data, and the object set may change dynamically in one by one over time. To deal with dynamic hybrid data with mixed-type features, we propose a neighborhood entropy-based incremental feature selection framework by neighborhood rough set model. In this paper, the dynamics of an object set involves the change of a single object and multiple objects. Therefore, two incremental feature selection algorithms are developed for hybrid data with the dynamic change of a single object and multiple objects, respectively. At first, an incremental manner is utilized to compute the neighborhood entropy as feature criterion. On this basis, the incremental computations of feature significance are used to select candidate features in a descending order. Meanwhile, a deletion strategy is employed to filter out redundant features from the selection results. Finally, experimental results on different real-life data sets demonstrate the proposed incremental algorithms can outperform the non-incremental algorithm for feature selection in speed within comparable classification accuracy. Especially for multiple objects adding into and deleting from the hybrid data, the incremental algorithm is more efficient to select a subset of features than the algorithm for handling the dynamic change of a single object.

A Feature Selection Algorithm Based on Equal Interval Division and Minimal-Redundancy–Maximal-Relevance

Minimal-redundancy–maximal-relevance (mRMR) algorithm is a typical feature selection algorithm. To select the feature which has minimal redundancy with the selected features and maximal relevance with the class label, the objective function of mRMR subtracts the average value of mutual information between features from mutual information between features and the class label, and selects the feature with the maximum difference. However, the problem is that the feature with the maximum difference is not always the feature with minimal redundancy maximal relevance. To solve the problem, the objective function of mRMR is first analyzed and a constraint condition that determines whether the objective function can guarantee the effectiveness of the selected features is achieved. Then, for the case where the objective function is not accurate, an idea of equal interval division is proposed and combined with ranking to process the interval of mutual information between features and the class label, and that of the average value of mutual information between features. Finally, a feature selection algorithm based on equal interval division and minimal-redundancy–maximal-relevance (EID–mRMR) is proposed. To validate the performance of EID–mRMR, we compare it with several incremental feature selection algorithms based on mutual information and other feature selection algorithms. Experimental results demonstrate that the EID–mRMR algorithm can achieve better feature selection performance.

Fuzzy neuro-genetic approach for feature selection and image classification in augmented reality systems

In this paper, a new approach for implementing an Augmented Reality system by applying fuzzy genetic neural networks is proposed. It consists of two components namely feature selection and classification modules. For feature detection, extraction and selection, the proposed model uses a fuzzy logic based incremental feature selection algorithm which has been proposed in this work in order to recognize the important features from 3D images. Moreover, this paper explains the implementation and results of the proposed algorithms for an Augmented Reality system using image recognition, feature extraction, feature selection and classification by considering the global and local features of the images. For this purpose, we propose a three layer fuzzy neural network that has been implemented based on weight adjustments using fuzzy rules in the convolutional neural networks with genetic algorithm for effective optimization of rules. The classification algorithm is also based on fuzzy neuro-genetic approach which consists of two phases namely Training phase and testing phase. During the training phase, rules are formed based on objects and these rules are applied during the testing phase for recognizing the objects which can be used in robotics for effective object recognition. From the experiments conducted in this work, it is proved that the proposed model is more accurate in 3D <br /> object recognition.

Discernible neighborhood counting based incremental feature selection for heterogeneous data

Incremental feature selection refreshes a subset of information-rich features from added-in samples without forgetting the previously learned knowledge. However, most existing algorithms for incremental feature selection have no explicit mechanisms to handle heterogeneous data with symbolic and real-valued features. Therefore, this paper presents an incremental feature selection method for heterogeneous data with the sequential arrival of samples in group. Discernible neighborhood counting that measures different types of features, is first introduced to establish a framework for feature selection from heterogeneous data. With the arrival of new samples, the discernible neighborhood counting of a feature subset is then updated to reveal the incremental feature selection scheme. This scheme determines the criterion for efficiently adding informative features and deleting redundant features. Based on the incremental scheme, our incremental feature selection algorithm is further formulated to select valuable features from heterogeneous data. Extensive experiments are finally conducted to demonstrate the effectiveness and the efficiency of the proposed incremental feature selection algorithm.

Incremental approaches for feature selection from dynamic data with the variation of multiple objects

Owing to the dynamic characteristics of data in the big data era, multiple objects of a decision system often vary with time when new information arrives in real-world applications. However, many feature selection algorithms are designed for static decision systems, and some dynamic feature selection algorithms treat the variation of multiple objects as the cumulative variation of a single object. In an environment where multiple objects vary with time, these algorithms are often time-consuming. Therefore, strategic behaviors need to be reinforced to improve the efficiency of feature selection. Incremental updating is an efficient technique, which can be applied to deal with dynamic learning tasks because it can make use of previous knowledge to obtain new knowledge. In this paper, we focus on the incremental updating to select a new feature subset with the variation of multiple objects. First, the dependency function is updated in an incremental manner to evaluate the quality of candidate features. Then two incremental feature selection algorithms are developed when multiple objects are added to or deleted from a decision system. Experiments on different UCI data sets show that the proposed algorithms can select new feature subset in much less computational time and do not lose the classification performance when compared with other algorithms.

Smaller feature subset selection for real-world datasets using a new mutual information with Gaussian gain

A new filter method is proposed for feature selection and ranking that incorporates a novel mutual information with Gaussian gain for evaluating the relationships between features and the class, and in-between features. The new mutual information is derived as per the axioms of classical information theory from the recently introduced non-extensive entropy with Gaussian gain. The characteristic of this entropy is its non-linearity when representing correlated information in natural texture images represented by sparse probability distributions. In this work, we trace this property in our new mutual information in the context of correlated random variables associated with real-world datasets. The non-linearity of the Gaussian function embedded in the mutual information formula is utilized for identifying the most important features in the correct order of rank, right at the outset of the incremental feature selection algorithm. This leads to formation of smaller groups of ranked feature subsets that give the highest classification accuracies. Extensive experimentation on twenty benchmark datasets from the UCI repository along with comparison to the state-of-the-art confirms the efficacy of our approach. An automated optimum feature subset selection is also proposed based on a simple statistical test on the new measure.

Prediction of lysine glutarylation sites by maximum relevance minimum redundancy feature selection

Lysine glutarylation is new type of protein acylation modification in both prokaryotes and eukaryotes. To better understand the molecular mechanism of glutarylation, it is important to identify glutarylated substrates and their corresponding glutarylation sites accurately. In this study, a novel bioinformatics tool named GlutPred is developed to predict glutarylation sites by using multiple feature extraction and maximum relevance minimum redundancy feature selection. On the one hand, amino acid factors, binary encoding, and the composition of k-spaced amino acid pairs features are incorporated to encode glutarylation sites. And the maximum relevance minimum redundancy method and the incremental feature selection algorithm are adopted to remove the redundant features. On the other hand, a biased support vector machine algorithm is used to handle the imbalanced problem in glutarylation sites training dataset. As illustrated by 10-fold cross-validation, the performance of GlutPred achieves a satisfactory performance with a Sensitivity of 64.80%, a Specificity of 76.60%, an Accuracy of 74.90% and a Matthew's correlation coefficient of 0.3194. Feature analysis shows that some k-spaced amino acid pair features play the most important roles in the prediction of glutarylation sites. The conclusions derived from this study might provide some clues for understanding the molecular mechanisms of glutarylation.