Proposed Feature Selection Algorithm Research Articles

SFE: A Simple, Fast, and Efficient Feature Selection Algorithm for High-Dimensional Data

In this paper, a new feature selection algorithm, called SFE (Simple, Fast, and Efficient), is proposed for high-dimensional datasets. The SFE algorithm performs its search process using a search agent and two operators: non-selection and selection. It comprises two phases: exploration and exploitation. In the exploration phase, the non-selection operator performs a global search in the entire problem search space for the irrelevant, redundant, trivial, and noisy features, and changes the status of the features from selected mode to non-selected mode. In the exploitation phase, the selection operator searches the problem search space for the features with a high impact on the classification results, and changes the status of the features from non-selected mode to selected mode. The proposed SFE is successful in feature selection from high-dimensional datasets. However, after reducing the dimensionality of a dataset, its performance cannot be increased significantly. In these situations, an evolutionary computational method could be used to find a more efficient subset of features in the new and reduced search space. To overcome this issue, this paper proposes a hybrid algorithm, SFE-PSO (particle swarm optimization) to find an optimal feature subset. The efficiency and effectiveness of the SFE and the SFE-PSO for feature selection are compared on 40 high-dimensional datasets. Their performances were compared with six recently proposed feature selection algorithms. The results obtained indicate that the two proposed algorithms significantly outperform the other algorithms, and can be used as efficient and effective algorithms in selecting features from high-dimensional datasets.

Identification of OSAHS patients based on ReliefF-mRMR feature selection.

Obstructive Sleep Apnea Hypopnea Syndrome (OSAHS) is a serious chronic sleep disorder. Snoring is a common and easily observable symptom of OSAHS patients. The purpose of this work is to identify OSAHS patients by analyzing the acoustic characteristics of snoring sounds throughout the entire night. Ten types of acoustic features, such as Mel-frequency cepstral coefficients (MFCC), linear prediction coefficients (LPC) and spectral entropy among others, were extracted from the snoring sounds. A fused feature selection algorithm based on ReliefF and Max-Relevance and Min-Redundancy (mRMR) was proposed for optimal feature set selection. Four types of machine learning models were then applied to validate the effectiveness of OSAHS patient identification. The results show that the proposed feature selection algorithm can effectively select features with high contribution, including MFCC and LPC. Based on the selected top-20 features and using a support vector machine model, the accuracies in identifying OSAHS patients under the thresholds of AHI = 5,15, and 30, were 100%, 100%, and 98.94%, respectively. This indicates that the proposed model can effectively identify OSAHS patients.

Modeling nanofluid viscosity: comparing models and optimizing feature selection—a novel approach

BackgroundThe accurate prediction of viscosity in nanofluids is essential for comprehending their flow behavior and enhancing their effectiveness in different industries. This research delves into modeling the viscosity of nanofluids and assessing various models through cross-validation techniques. The models are compared based on the root mean square error of the cross-validation sets, which served as the selection criteria.The main body of the abstractFour feature selection algorithms namely the minimum redundancy maximum relevance, F-test, RReliefF were evaluated to identify the most influential features for viscosity prediction. The feature selection based on physical meaning was the algorithm that yielded the best results, as outlined in this study. This methodology takes into account the physical relevance of most aspects of the nanofluid's viscosity. To assess the predictive performance of the models, a cross-validation process was conducted, which provided a robust evaluation. The root mean squared error of the validation sets was used to compare the models. This rigorous evaluation identified the most accurate and reliable model for predicting nanofluid viscosity.ResultsThe results showed that the novel feature selection algorithm outclassed the established approaches in predicting the viscosity of single material nanofluid. The proposed feature selection algorithm had a root mean squared error of 0.022 and an r squared value of 0.9941 for the validation set, while for the test set, the root mean squared error was 0.0146, the mean squared error was 0.0157, the r squared value was 0.9924.ConclusionsThis research provides valuable insights into nanofluid viscosity and offers guidance on choosing the most suitable features for viscosity modeling. The study also highlights the importance of using physical meaning to select features and cross-validation to assess model performance. The models developed in this study can be helpful in predicting nanofluid viscosity and optimizing their use in different industrial processes.

Predictive modelling of thermal conductivity in single-material nanofluids: a novel approach

BackgroundThis research introduces a novel approach for modelling single-material nanofluids, considering the constituents and characteristics of the fluids under investigation. The primary focus of this study was to develop models for predicting the thermal conductivity of nanofluids using a range of machine learning algorithms, including ensembles, trees, neural networks, linear regression, Gaussian process regressors, and support vector machines.The main body of the abstractTo identify the most relevant features for accurate thermal conductivity prediction, the study compared the performance of established feature selection algorithms, such as minimum redundancy maximum relevance, Ftest, and RReliefF, a newly proposed feature selection algorithm. The novel algorithm eliminated features lacking direct implications for fluid thermal conductivity. The selected features included temperature as a thermal property of the fluid itself, multiphase features such as volume fraction and particle size, and material features including nanoparticle material and base fluid material, which could be fixed based on any two intensive properties. Statistical methods were employed to select the features accordingly.ResultsThe results demonstrated that the novel feature selection algorithm outperformed the established approaches in predicting the thermal conductivity of nanofluids. The models were evaluated using fivefold cross-validation, and the best model was the model based on the proposed feature selection algorithm that exhibited a root-mean-squared error of validation of 1.83 and an R-squared value of 0.94 on validation set. The model achieved a root-mean-squared error of 1.46 and an R-squared value of 0.97 for the test set.ConclusionsThe developed predictive model holds practical significance by enabling nanofluids' numerical study and optimisation before their creation. This model facilitates the customisation of conventional fluids to attain desired fluid properties, particularly their thermal properties. Additionally, the model permits the exploration of numerous nanofluid variations based on permutations of their features. Consequently, this research contributes valuable insights to the design and optimisation of nanofluid systems, advancing our understanding and application of thermal conductivity in nanofluids and introducing a novel and methodological approach for feature selection in machine learning.

Nested ensemble selection: An effective hybrid feature selection method

It has been shown that while feature selection algorithms are able to distinguish between relevant and irrelevant features, they fail to differentiate between relevant and redundant and correlated features. To address this issue, we propose a highly effective approach, called Nested Ensemble Selection (NES), that is based on a combination of filter and wrapper methods. The proposed feature selection algorithm differs from the existing filter-wrapper hybrid methods in its simplicity and efficiency as well as precision. The new algorithm is able to separate the relevant variables from the irrelevant as well as the redundant and correlated features. Furthermore, we provide a robust heuristic for identifying the optimal number of selected features which remains one of the greatest challenges in feature selection. Numerical experiments on synthetic and real-life data demonstrate the effectiveness of the proposed method. The NES algorithm achieves perfect precision on the synthetic data and near optimal accuracy on the real-life data. The proposed method is compared against several popular algorithms including mRMR, Boruta, genetic, recursive feature elimination, Lasso, and Elastic Net. The results show that NES significantly outperforms the benchmarks algorithms especially on multi-class datasets.

An improved hybrid chameleon swarm algorithm for feature selection in medical diagnosis

Feature selection (FS) is generally associated with the process of using a probabilistic method to select optimal feature combinations during pre-processing steps in data mining. This technique can optimize the datasets’ features that need to be considered to heighten the performance of classification on the grounds of the selected optimal feature set. In this paper, a hybridization model is evolved and applied to select the optimal feature subset based on a binary version of the Hybrid Memory Improved Chameleon Swarm Algorithm (CSA) (HMICSA) and the k-Nearest Neighbor (k-NN) classifier. In this FS model, the following are proposed and applied: (1) Four kinds of transfer functions, (2) Amendments to the velocity of the CSA’s individuals, (3) Addition of internal memory to the CSA’s individuals, and (4) Hybridization of CSA with Ali baba and the Forty Thieves (AFT) algorithm. These actions are aimed to strike an adequate equipoise between global exploration and local exploitation conducts of the search space of the basic CSA. This is to mitigate the problem of early convergence, and to sidestep trapping into a local optima in CSA. The efficacy of the proposed FS algorithm was evaluated on 24 medical diagnosis benchmark datasets collected from different specialized repositories and compared with other k-NN-based FS methods. The all-inclusive outcomes using various evaluation methods disclose the competence of the proposed method in augmenting the classification performance compared to other methods, ensuring its potential in scouting the feature space and designating the most useful features for classification tasks.

Relative Fuzzy Rough Approximations for Feature Selection and Classification.

Fuzzy rough set (FRS) theory is generally used to measure the uncertainty of data. However, this theory cannot work well when the class density of a data distribution differs greatly. In this work, a relative distance measure is first proposed to fit the mentioned data distribution. Based on the measure, a relative FRS model is introduced to remedy the mentioned imperfection of classical FRSs. Then, the positive region, negative region, and boundary region are defined to measure the uncertainty of data with the relative FRSs. Besides, a relative fuzzy dependency is defined to evaluate the importance of features to decision. With the proposed feature evaluation, we propose a feature selection algorithm and design a classifier based on the maximal positive region. The classification principle is that an unlabeled sample will be classified into the class corresponding to the maximal degree of the positive region. Experimental results show the relative fuzzy dependency is an effective and efficient measure for evaluating features, and the proposed feature selection algorithm presents better performance than some classical algorithms. Besides, it also shows the proposed classifier can achieve slightly better performance than the KNN classifier, which demonstrates that the maximal positive region-based classifier is effective and feasible.

Multilevel Feature Selection Method for Improving Classification of Microarray Gene Expression Data

Microarray gene expression profiles provide valuable answers to a variety of problems, and contributes to advances in clinical medicine. Gene expression data typically has a high dimension and a small sample size. Gene selection from microarray gene expression data is a challenge due to high dimensionality of the data. The number of samples in the microarray dataset is much smaller compared to the number of genes as features. To extract useful gene information from cancer microarray data and reduce dimensionality, selection of significant genes is necessary. An effective method of gene feature selection helps in dimensionality reduction and improves the classification performance. Experimental results suggest that appropriate combination of filter gene selection methods is more effective than individual techniques for microarray data classification. In this paper, we propose a two-layered feature selection method. In the first layer, t-test statistical method is used to remove the features that have little correlation with the classification results. In the second layer, line segment approximation method is used to transform the feature subset into a less dimensional feature space. Four well known classifiers kNN, SVM, NBC, DT were used to verify the performance of the proposed feature selection algorithm on binary class microarray data. The experimental results show that the proposed method can effectively select relevant gene subsets, and achieves higher classification accuracy.

Feature Selection Considering Multiple Correlations Based on Soft Fuzzy Dominance Rough Sets for Monotonic Classification

Monotonic classification is a common task in the field of multicriteria decision-making, in which features and decision obey a monotonic constraint. The dominance-based rough set theory is an important mathematical tool for knowledge acquisition in monotonic classification tasks (MCTs). However, existing dominance-based rough set models are very sensitive to noise information, and only a misclassified sample will lead to large errors in acquiring knowledge. This unstable phenomenon does not meet the requirements of practical applications. On the other hand, feature selection is supposedly an effective dimensionality reduction approach for classification tasks. In the real world, feature combinations with multiple correlations can often provide important classification information, where the multiple correlations include redundancy, complementarity, and interaction between features. To the best of our knowledge, most of the existing feature selection methods for MCTs only consider the relevance between features and decision, while ignoring the multiple correlations. To overcome these two drawbacks, in this article, we propose a robust fuzzy dominance rough set model, and develop a feature selection method that considers multiple correlations based on the robust model for MCTs. First, a soft fuzzy dominance rough set (SFDRS) with robustness is proposed. Second, a feature evaluation index considering multiple correlations is presented. Finally, a feature selection algorithm based on SFDRS is designed to select an optimal feature subset. Extensive experiments are conducted on 12 public datasets, and the results show that the SFDRS model has good robustness and the proposed feature selection algorithm has excellent classification performance.

Unsupervised Feature Selection Using Iterative Shrinking and Expansion Algorithm

In this article, we propose an unsupervised feature selection algorithm based on the recently developed shrinking and expansion algorithm (SEA). The SEA is already an established, efficient algorithm to identify dense subgraphs within a weighted graph. To conduct SEA on a weighted graph, the dataset is first mapped onto an equivalent graph notation where each vertex corresponds to a feature, and the weight of each edge represents the normalized mutual information between the associated two features that form the edge. As a result, the feature selection problem is presented as the best way to obtain several dense sub-feature spaces by using the SEA on a weighted feature graph. The proposed feature selection algorithm consists of a two-stage structure, in which the first stage is utilized to find a number of dense feature subgraphs, whereas the second stage is used to identify a representative feature from each of these feature subgraphs. The main advantage of the proposed algorithm is that the users are not required to provide the number of features to be selected, which is a major flaw in most of the existing algorithms. The current findings from the researchers' method are better than the other state-of-the-art algorithms because the feature selection process identifies approximately one-fifth to one-third of the number of original features while providing a high accuracy score. The superiority of the proposed algorithm over the other conventional methods of feature selection is established for a number of real-life datasets. In addition, the performance of the proposed feature selection algorithm is also found to be better than other algorithms in one of the innovative application areas of computational intelligence such as ambient intelligence.

Semi-supervised feature selection for partially labeled mixed-type data based on multi-criteria measure approach

In many real applications, the data are always collected from different types and they are subjected to obtain partial labeling information of objects. Such data are referred to as partially labeled mixed-type data. There is currently few work on feature selection approaches for these data. Motivated by this issue, this paper aims at selecting the informative feature subset from partially labeled mixed-type data. At first, to improve the classification performance, an improved label propagation algorithm based on K-nearest neighbor is proposed, which marks the decision labels of unlabeled objects by making use of the information between unlabeled objects and labeled objects. On this basis, a feature multi-criteria measure based on the dependency, information entropy and information granulation is proposed for selecting candidate features. Finally, the corresponding semi-supervised feature selection algorithm is developed to select the feature subset for the partially labeled mixed-type data. Experimental results on UCI data sets demonstrate the effectiveness of the proposed feature selection algorithm and the superiority in terms of the classification accuracy compared with other algorithms.

Load2Load: Day-ahead load forecasting at aggregated level

A reliable and accurate short-term load forecasting (STLF) helps utilities and energy providers deal with the challenges posed by supply and demand balance, higher penetration of renewable energies and the development of electricity markets with increasingly complex pricing strategies in future smart grids. Recent advances in deep learning have been successively utilized to STLF. However, there is no certain study that evaluates the performances of different STLF methods at an aggregated level on different datasets with different numbers of daily measurements.In this study, a deep learning STLF architecture called Load2Load is proposed for day-ahead forecasting. Different forecasting methods have been evaluated and compared on two datasets with different temporal resolutions and features. An additive ensemble method as well as a selective ensemble method that selects the outputs of different forecasters in an hourly manner are proposed. Moreover; a modified sequential forward feature selection algorithm is proposed, resulting in better performance with a much smaller number of features.Numerical results show that the proposed Load2Load architecture has a competing performance compared to other advanced forecasters. When used together with the proposed ensemble methods, Load2Load can significantly improve the forecasting performance. The proposed feature selection algorithm results in better performance for the majority of the cases while reducing the dimensionality. According to the results with two different datasets; the proposed methods are shown to be robust to temporal resolutions, feature types and sequence lengths.

Occupancy prediction using deep learning approaches across multiple space types: A minimum sensing strategy

The proliferation of sensing technologies has allowed the collection of occupancy-related data to support various building applications, including adaptive HVAC and lighting controls, maintenance operations, and space utilisation. However, past occupancy prediction studies often considered different combinations of sensor data and investigated a limited number of space types. This study performs occupancy prediction based on a minimum sensing strategy by using a comprehensive set of sensor data (i.e., indoor environmental and outdoor weather conditions, Wi-Fi connected devices, energy consumption data, HVAC operations, and time-related information) to identify the most crucial features through a proposed feature selection algorithm. Occupancy predictions were subsequently performed using different deep learning architectures, including Deep Neural Network (DNN), Long Short-Term Memory (LSTM), Bi-directional LSTM (Bi-LSTM), Gated Recurrent Unit (GRU), and Bi-directional GRU (Bi-GRU) in an office, library, and lecture room. Our findings highlighted that the proposed feature selection algorithm outperformed a popular feature selection algorithm to achieve a higher model performance with lower sensing requirements. Furthermore, empirical results showed that indoor CO2 levels and Wi-Fi connected devices were crucial features for predicting occupancy across all space types. The best model performances were achieved using Bi-GRU for office, GRU for library, and Bi-GRU for lecture room.

AHA-AO: Artificial Hummingbird Algorithm with Aquila Optimization for Efficient Feature Selection in Medical Image Classification

This paper presents a system for medical image diagnosis that uses transfer learning (TL) and feature selection techniques. The main aim of TL on pre-trained models such as MobileNetV3 is to extract features from raw images. Here, a novel feature selection optimization algorithm called the Artificial Hummingbird Algorithm based on Aquila Optimization (AHA-AO) is proposed. The AHA-AO is used to select only the most relevant features and ensure the improvement of the overall model classification. Our methodology was evaluated using four datasets, namely, ISIC-2016, PH2, Chest-XRay, and Blood-Cell. We compared the proposed feature selection algorithm with five of the most popular feature selection optimization algorithms. We obtained an accuracy of 87.30% for the ISIC-2016 dataset, 97.50% for the PH2 dataset, 86.90% for the Chest-XRay dataset, and 88.60% for the Blood-cell dataset. The AHA-AO outperformed the other optimization techniques. Moreover, the developed AHA-AO was faster than the other feature selection models during the process of determining the relevant features. The proposed feature selection algorithm successfully improved the performance and the speed of the overall deep learning models.

Self-adaptive weighted interaction feature selection based on robust fuzzy dominance rough sets for monotonic classification

Feature selection is an effective dimensionality reduction technique for classification tasks. Monotonic classification task (MCT) is a special classification task in which features and decision obey a monotonic constraint, which is widely used in various application fields. Most of the existing feature selection methods for MCTs are based on some dominance rough set models. However, these methods only consider the correlation between features and decisions, and ignore the interaction between features. Moreover, these dominance rough set models are not robust to noisy information. Thus, to make up for these two deficiencies, we propose a novel feature selection approach based on an improved robust fuzzy dominance rough set for MCTs. First, a β−precision fuzzy dominance neighborhood rough set (β−PFDNRS) with robustness is proposed. Second, a feature evaluation index is presented, which not only considers the correlation between features and decision, but also considers the interaction between features. Finally, a β−PFDNRS based self-adaptive weighted interaction feature selection (β−SWIFS) algorithm is designed to select a feature sequence. Extensive experiments are conducted on fifteen public datasets, and the results show that the β−PFDNRS model has good robustness and the proposed feature selection algorithm has an excellent classification performance.

Optimization of the Intelligent Sports Health Management System Based on Big Data Technology and Internet of Things

People often suffer from unpredictable injuries during physical exercise. One of the important reasons is the absence of a scientific sports health management system. Therefore, the construction of such a scientific and effective system has gradually attracted the attention of scholars, which is of great significance to realizing people’s scientific and personalized physical fitness. An intelligent sports health management system based on big data analysis and the Internet of things (IoT) is constructed to solve this problem. The system consists of the user, IoT, cloud, system analysis, evaluation, and data layers. Firstly, a new multilabel feature selection algorithm is proposed in the system analysis layer. The suggested multilabel feature selection algorithm maps the sample space to the label space through the L 21 norm. Then, the consistency of various topologies is guaranteed by combining with feature popularity so that the factors affecting user health can be better selected. Secondly, the experiment is compared with SCLS, SSFS, and six other multilabel feature selection algorithms in 6 classic medical multilabel datasets. Experimental results under five indexes show the effectiveness and superiority of the proposed feature selection algorithm. Finally, the feasibility of the proposed intelligent sports management system is analyzed.

Novel optimized crow search algorithm for feature selection

Feature selection techniques have been presented to allow us to choose a small subset of the original components’ relevant features by removing irrelevant or redundant features. Feature selection is essential for many reasons such as simplification, performance, computational efficiency, and quality interpretability. Owing to the importance mentioned above, many researchers have proposed and developed many algorithms to solve the feature selection problem. Although these approaches produce useful results, they possess some shortcomings like inadequate feature reduction. In this paper, a novel feature selection algorithm based on the crow search algorithm is presented. The algorithm uses dynamic awareness probability to keep the balance between the local and global search processes. Moreover, a novel neighborhood assigning strategy has been introduced to optimize the local search. Considering the best-selected features in each iteration helps attain more benefits in global search. The main superiority of the proposed algorithm is the significant feature reduction along with retaining the accuracy. Compared to enhanced crow search algorithm, the proposed algorithm has improved the feature reduction metric and fitness metric by 27.12% and 5.16%, respectively, while losing the accuracy metric by only 0.53%. Several popular UCI datasets have been employed to evaluate the proposed feature selection algorithm. The experimental results show that the proposed algorithm outperformed other feature selection algorithms in state-of-the-art related works regarding feature reduction and accuracy.

Machine Learning Assisted Methodology for Multiclass Classification of Malignant Brain Tumors

Analysis of malignant and non-malignant brain tumors is done using a computer-aided diagnosis system by practitioners worldwide. Radiologists refer computer-assisted techniques to draw conclusions using image modalities and inferences. Pedagogically, various machine learning approaches have been used, which usually focus on the classification of imaging modality into two categories, either normal and abnormal images or differentiating between benign and malignant tumors. Still, the work requirement is to classify these multi-class malignant tumors into their specific class with better precision. The proposed work focuses on distinguishing between the types of high-grade malignant brain tumors. This study is performed on real-life malignant brain tumor datasets having five classes. The proposed methodology uses the vast feature set from six domains to capture most of the hidden information in the extracted region of interest. Later, relevant features are extracted from the feature set pool using a new proposed feature selection algorithm named the Cumulative Variance method (CVM). Next, the selected features are used for model training and testing using K-Nearest Neighbour (KNN), multi-class Support Vector Machine (mSVM), and Neural Network (NN) for predicting multi-class classification accuracy. The experiments are performed using the proposed feature selection algorithm with three classifiers. The mean average classification accuracy achieved by using the proposed approach is88.43% (KNN), 92.5% (mSVM), and 95.86% (NN), respectively. The comparative analysis of the proposed approach with other existing algorithms like ICA, and GA suggest that the proposed approach gains an increase of accuracy around 2% (KNN), 3% (SVM), and 4% (NN).The experimentation results concluded that the proposed approach is found better with NN classifier with an accuracy of 95.86% using diversified features.

Fuzzy information decomposition incorporated and weighted Relief-F feature selection: When imbalanced data meet incompletion

Data classification is an important computer task in data analysis, which suffers seriously unknown features, imbalanced class, and incomplete data. However, despite their vital yet practical significance, few results have been made on such three distinct issues. To address this problem, we propose a novel feature selection method for the data subject to incomplete data and imbalanced class, namely, improved fuzzy information decomposition (IFID) incorporated and weighted Relief-F (WRelief-F) feature selection. The main idea of the proposed feature selection method is threefold. (1) The proposed IFID algorithm can deal with the imbalanced class and incomplete data at the same time. (2) In IFID, a new membership function is provided to reflect the influence of the observed data on the missing values appropriately. Based on this establishment, a more delicate information decomposition is adopted to make a better recovery than the traditional FID. (3) After using IFID, WRelief-F is put forward to take the relationship of the target instance to inter-class instances and the intra-class instances into consideration in a proper manner. Finally, experiments on the seven public data sets are utilized to show the effectiveness and universal applicability of the proposed feature selection algorithm.

A relative uncertainty measure for fuzzy rough feature selection

Uncertainty measure is an important tool for data analysis. In practical applications, the collected data are subject to different probability distributions. This requires that the uncertainty measure has generalization performance. Fuzzy rough set (FRS) theory is a popular mathematical tool for uncertainty measure, but the theory does not work well for some data distributions. For example, when the class density difference of the data set is large, FRS theory cannot effectively evaluate the classification uncertainty of samples. In this study, we combine the relative measure with the lower approximation of FRSs to propose a relative uncertainty measure which can address the above-mentioned problem. Furthermore, a fuzzy rough feature selection algorithm is designed, and it is mainly used to test the effectiveness and efficiency of the proposed measure. Experimental results demonstrate that the proposed feature selection algorithm has good performance. It indirectly proves that the relative uncertainty measure is effective and efficient in classification tasks.