Proposed Feature Selection Method Research Articles

Feature selection using non-parametric correlations and important features on recursive feature elimination for stock price prediction

Stock price prediction using machine learning is a rapidly growing area of research. However, the large number of features that can be used can complicate the learning process. The feature selection method that can be used to overcome this problem is recursive feature elimination. Standard recursive feature elimination carries the risk of producing inaccurate algorithms because the top-ranked features are not necessarily the most important features. This research proposes a feature selection method that combines important features and nonparametric correlation in recursive feature elimination for stock price prediction. The data features used are technical indicators and stock price history. The recursive feature elimination method is modified with important features and nonparametric correlation features. The strategy for combining important features and non-parametric features is average weight, 25:75% weight, 75:25% weight, maximum weight, and minimum weight. The performance evaluation results show that the proposed feature selection method succeeded in obtaining small error values. The proposed method for predicting PT Bank Rakyat Indonesia Tbk (BBRI) stock prices obtains mean squared error, root mean square error, mean absolute error, and mean absolute percentage error evaluation values of 0.0000336, 0.00577, 0.00459, and 1.78%, respectively. This shows that recursive feature elimination with feature selection that combines important features and non-parametric correlation works better than the original recursive feature elimination at predicting stock prices.

Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients

During outbreaks of infectious diseases, such as COVID-19, it is critical to rapidly determine treatment priorities and identify patients requiring hospitalization based on clinical severity. Although various machine learning models have been developed to predict COVID-19 severity, most have limitations, such as small dataset sizes, the limited availability of clinical variables, or a constrained classification of severity levels by a single classifier. In this paper, we propose an adaptive stacking ensemble technique that identifies various COVID-19 patient severity levels and separates them into three formats: Type 1 (low or high severity), Type 2 (mild, severe, critical), and Type 3 (asymptomatic, mild, moderate, severe, fatal). To enhance the model’s generalizability, we utilized a nationwide dataset from the South Korean government, comprising data from 5644 patients across over 100 hospitals. To address the limited availability of clinical variables, our technique employs data-driven strategies and a proposed feature selection method. This ensures the availability of clinical variables across diverse hospital environments. To construct optimal stacking ensemble models, our technique adaptively selects candidate base classifiers by analyzing the correlation between their predicted outcomes and performance. It then automatically determines the optimal multi-layer combination of base and meta-classifiers using a greedy search algorithm. To further improve the performance, we applied various techniques, including imputation of missing values and oversampling. The experimental results demonstrate that our stacking ensemble models significantly outperform existing single classifiers and AutoML approaches, with improvements of 6.42% and 8.86% in F1 and AUC scores for Type 1, 9.59% and 6.68% for Type 2, and 11.94% and 9.24% for Type 3, respectively. Consequently, our approach improves the prediction of COVID-19 severity levels and potentially assists frontline healthcare providers in making informed decisions.

Providing a Feature Selection Method for Lung Cancer Prediction Using Neural Network

Introduction: The remarkable growth of lung cancer and its associated impacts and consequences, along with the substantial costs it imposes on society, has driven the medical community to pursue programs aimed at further examination, prevention, early detection, and diagnosis. In medicine science, timely discovery and diagnosis of diseases can prevent many life-threatening conditions and save people's lives.Material and Methods: This study aims to predict lung cancer using a novel feature selection method integrated with a classifier. Our approach entails a comprehensive four-stage method. Initially, we calculate feature similarities within a lung cancer dataset using the absolute value of the Pearson correlation coefficient, followed by the clustering of initial features using the community detection algorithm called Louvain. Next, we employ techniques to determine the optimal subset of features using the concept of node centrality. Ultimately, lung cancer diagnosis is executed using the selected features, leveraging a classifier.Results: Comparative analysis reveals that our proposed method outperforms existing techniques in terms of reduced execution time and improved prediction accuracy. When compared with established methods, our approach demonstrates superior outcomes in terms of the number of selected features and classification accuracy. Our method reduced 12600 features to 118 features and its accuracy was 95.28, 95.49, 95.23 and 95.32 for Support Vector Machine (SVM), Decision Tree (DT), Naive Bayes (NB) and K-Nearest Neighbor (KNN) classifier. The comparison of runtime shows that the proposed method is significantly improved with a runtime of 2.146 seconds compared to other methods.Conclusion: The proposed feature selection method successfully reduced the initial feature set and significantly decreased computational time. Moreover, the achieved prediction accuracies underscore the reliability of our approach. This significant reduction in feature space while maintaining consistently high prediction accuracies serves as a strong validation of the potency and practical applicability of our methodology in the domain of lung cancer prediction. These compelling results strongly advocate for the potential real-world impact of our approach.

Semi-Supervised Feature Selection of Educational Data Mining for Student Performance Analysis

In recent years, the informatization of the educational system has caused a substantial increase in educational data. Educational data mining can assist in identifying the factors influencing students’ performance. However, two challenges have arisen in the field of educational data mining: (1) How to handle the abundance of unlabeled data? (2) How to identify the most crucial characteristics that impact student performance? In this paper, a semi-supervised feature selection framework is proposed to analyze the factors influencing student performance. The proposed method is semi-supervised, enabling the processing of a considerable amount of unlabeled data with only a few labeled instances. Additionally, by solving a feature selection matrix, the weights of each feature can be determined, to rank their importance. Furthermore, various commonly used classifiers are employed to assess the performance of the proposed feature selection method. Extensive experiments demonstrate the superiority of the proposed semi-supervised feature selection approach. The experiments indicate that behavioral characteristics are significant for student performance, and the proposed method outperforms the state-of-the-art feature selection methods by approximately 3.9% when extracting the most important feature.

Genetic Clustering Algorithm-Based Feature Selection and Divergent Random Forest for Multiclass Cancer Classification Using Gene Expression Data

AbstractComputational identification and classification of clinical disorders gather major importance due to the effective improvement of machine learning methodologies. Cancer identification and classification are essential clinical areas to address, where accurate classification for multiple types of cancer is still in a progressive stage. In this article, we propose a multiclass cancer classification model that categorizes the five different types of cancers using gene expression data. To perform efficient analysis of the available clinical data, we propose feature selection and classification methods. We propose a genetic clustering algorithm (GCA) for optimal feature selection from the RNA-gene expression data, consisting of 801 samples belonging to the five major classes of cancer. The proposed feature selection method reduces the 1621 gene expressions into a cluster of 21 features. The optimum feature set acts as input data to the proposed divergent random forest. Based on the features computed, the proposed classifier categorizes the data samples into 5 different classes of cancers, including breast cancer, colon cancer, kidney cancer, lung cancer, and prostate cancer. The proposed divergent random forest provided performance improvisation in terms of accuracy with 95.21%, specificity with 93%, and sensitivity with 94.29% which outperformed all the other existing multiclass classification algorithms.

An intelligent fault detection and diagnosis model for refrigeration systems with a comprehensive feature selection method

Feature selection and model establishment are two essential steps for fault detection and diagnosis (FDD) of refrigeration systems. A robust and powerful FDD model combined with a suitable feature selection method can exhibit excellent performance in FDD tasks for refrigeration systems. In this study, a novel FDD method that integrates a comprehensive feature selection method and a deep learning-based intelligent FDD model is proposed. Including three steps, the comprehensive feature selection method combines filter methods and wrapper methods. It can optimize the features and the model jointly by using the multi-objective optimization algorithm to achieve a better performance. In addition, a novel FDD model that combines one-dimensional convolutional neural network (1D-CNN) and self-attention (SA) mechanism is proposed based on the deep learning technology. To evaluate the proposed method, experiments are performed on a miniature refrigeration system under 4 situations with multiple working conditions, forming a dataset for the FDD study. The proposed three-step feature selection method is utilized to obtain the best feature subset. The 1D-CNN and SA FDD model is constructed and the model is jointly optimized with the features. Several comparisons are carried out to demonstrate the effectiveness and superiority of the proposed feature selection method and the FDD model. The results demonstrate that the presented integrated optimization achieved a test accuracy of around 99.66 %, surpassing other popular FDD models including MLP, CNN, and LSTM.

High-Dimensional Feature Selection for Power System Congestion Event Prognosis With Enhanced Evolutionary Computation

Power system congestion event prognosis (CEP) based on multivariate time series (MTS) learning is an effective way to improve the warning abilities against risky situations. However, it is still challenging to decide which measurement sequences of system variables should be selected to train MTS learning-based CEP models, especially when facing high-dimensional candidate features from power systems. Focusing on this issue, this paper proposes a novel high-dimensional feature selection (FS) method to identify and select variables that contain beneficial data patterns for the prognosis of network congestion events. A wrapper framework embedded with multiple MTS learning models is first built to treat FS as a combinatorial optimization task and receive MTS as inputs directly without sequence compressions. Then, to improve the combinatorial optimization efficiency and FS results in the high-dimensional case, a hybrid structure called enhanced evolutionary computation (EEC) is developed by combining with information theory-based feature priority scores, which play the role of semi-guidance in the iterative search. Test results on both the real-world and synthetic datasets validate the effectiveness of the developed EEC structure and show that the selected features by the proposed FS method are more beneficial in identifying the early patterns of network congestion events, thus contributing to efficient and accurate CEP for power systems.

Low-voltage AC series arc fault detection based on Fisher-mutual information feature selection

The detection of multi-feature fusion is a crucial approach to address the issue of series arc fault detection. Effective feature selection plays a vital role in enhancing the accuracy of the classifier and reducing system complexity. In this study, a feature selection algorithm based on Fisher-mutual information is proposed to tackle the problem of feature selection in multi-feature fusion detection. This algorithm utilizes the characteristics of arc fault voltage source to construct a feature pool. The Fisher-score algorithm and mutual information algorithm are employed to construct an optimal feature subset. The feature subset undergoes rough selection by retaining key features of the classifier and fine selection by eliminating redundant features. Experimental results and comparisons with related methods demonstrate that the proposed feature selection method significantly enhances the classifier's recognition accuracy, reduces classification and recognition time, diminishes the feature dimension, and outperforms other existing methods.

Filter unsupervised spectral feature selection method for mixed data based on a new feature correlation measure

In recent years, Unsupervised Feature Selection (UFS) methods have attracted considerable interest in different research areas due to their wide application in problems where unlabeled data appears. Nevertheless, few UFS methods can process datasets described by both numerical and non-numerical features (mixed data). This paper introduces a new filter UFS method based on a new feature correlation measure for mixed data to select a relevant and non-redundant feature subset. The proposed method addresses the feature selection problem in two stages through a strategy that combines Spectral Feature Selection to identify relevant features and a Pair-wise Redundancy Analysis to remove those features with a high correlation with others. According to our experiments, the proposed feature selection method based on the introduced feature correlation measure provides better quality results than previous filter UFS methods for mixed data reported in the literature.

Explainable machine learning model for predicting punching shear strength of FRC flat slabs

Reinforced concrete slabs are vulnerable to punching shear failure at the slab-column joint, which can initiate catastrophic progressive collapse. The addition of steel fibers in the concrete matrix has emerged as an effective strategy to mitigate such progressive failure. However, the effects of the diverse mixture proportions of the concrete matrix with different types and dosages of fibers have made the accurate prediction of the punching shear strength (PSS) of the fiber-reinforced concrete (FRC) flat slabs a complex task, where the existing mechanical models have several limitations. Therefore, this study proposes an explainable XGBoost model for predicting PSS of flat slabs made with different types of FRC based on a newly established comprehensive database of 251 flat slabs including normal strength FRC slabs, high-performance FRC slabs, and ultra-high-performance FRC slabs. A customized procedure was proposed to establish the XGBoost model considering data preparation, feature selection, hyperparameter tuning and model validation. The performance of the XGBoost model was then compared with that of existing mechanical models. Finally, sensitivity analysis and SHapley Additive exPlanations (SHAP) analysis were applied to identify the most influential parameters on the prediction of PSS. Results show that the proposed feature selection method is effective in identifying six influential parameters from the eleven parameters related to the PSS of FRC flat slabs. The developed XGBoost model yielded highest prediction accuracy and lowest variation, which outperformed the other mechanical models. Sensitivity analysis also indicated similar trends of parameters in both the XGBoost model and the mechanical models. The PSS of FRC flat slabs can be improved by increasing the concrete compressive strength, reinforcement ratio, and fiber volume, and by decreasing the column width-to-depth ratio, water-to-binder ratio, and aggregate size ratio. The proposed XGBoost model could enhance the understanding of PSS of FRC flat slabs and guide future pertinent design code provisions.

Permutation importance based modified guided regularized random forest in human activity recognition with smartphone

Human Activity Recognition (HAR) is a burgeoning field of study due to its real-life applications in the medical field, the e-health system, and elder care or care of physically impaired people in a smart healthcare environment. Using sensors built into wearable devices, such as smartphones, HAR provides an opportunity to identify human behavior and better understand an individual’s health. Improving the classification performance of human activities is an academic and industrial focus. Feature selection can affect classification performance: redundant and irrelevant features increase the learning difficulty of the classification model, cause overfitting, reduce classification performance, decrease interpretability, and reduce generalizability. Many preceding studies showed the defectiveness of feature selection results, which causes difficulties for professionals in a variety of fields (e.g., medical practitioners) to analyze and interpret the obtained feature subsets. Random Forest (RF) based feature selection methods select more interpretable features than other methods. However, RF-based feature selection methods are highly biased. Herein, we propose a novel RF-based feature selection method, namely modified Guided Regularized Random Forest (mGRRF), using permutation importance to overcome this. To prove the effectiveness of the proposed feature selection method, we conduct experiments using a public standard HAR dataset. Five classifiers, such as random forest, k-nearest neighbors, logistic regression, support vector machine, and xgboost, are used to recognize human activities after selecting the relevant and vital features using mGRRF. Experimental results indicate that with mGRRF-based features, the recognition accuracy is generally improved to 98% or 7% better than when all the extracted statistical features are used.

Minimising redundancy, maximising relevance: HRV feature selection for stress classification

Heart rate variability serves as a valuable indicator and biomarker for stress detection and monitoring. Feature selection, which aims to identify relevant features from a large set of variables, is a crucial preprocessing step towards this. However, this task becomes challenging due to high dimensionality and the presence of irrelevant and redundant attributes. The Minimum Redundancy and Maximum Relevance (mRMR) feature selection method addresses this challenge by selecting relevant features while controlling redundancy. This paper presents extensions and evaluated versions of the mRMR feature selection methods for stress detection using Heart Rate Variability (HRV) measures. The proposed feature selection methods extend the traditional mRMR by replacing the Pearson correlation redundancy with non-linear feature redundancy measures capable of capturing more complex relationships between variables. An extensive empirical evaluation is conducted on the proposed mRMR extensions, comparing them with four other baseline feature selection methods using three publicly available datasets. The experimental results demonstrate the effectiveness of incorporating the non-linear feature redundancy measure into the feature selection process.

A comparative study of the neural network models for the stock market data classification—A multicriteria optimization approach

The aim of this paper is to explore the potential of the class of recurrent neural networks in developing a classification model for the stock market. For this research, the data that replicates the entire Nasdaq stock market limit order book is used. After extracting order book attributes from the raw dataset, feature selection approaches based on conditional entropy and Stochastic Universal Sampling are proposed, in order to highlight potentially informative features for our data classification task. Furthermore, the performances of nine presented recurrent neural network models for classifying the stock market data vector into one of the labels from the set S = {buy, sell, idle} are examined. In this study, the performances of the models based on different networks, namely the gated recurrent unit, the long short-term memory, and the recurrent neural network, are compared. Using a multi-criteria approach it is concluded that the model based on the GRU topology fed with the features selected using the newly proposed feature selection method outperforms other examined models.

An interactive feature selection method based on multi-step state transition algorithm for high-dimensional data

Feature selection (FS) has been extensively employed in classification tasks to reduce data dimensionality and enhance prediction performance effectively. Recently, hybrid filter-wrapper methods have exhibited promising results in FS problems by leveraging both advantages. However, the inadequate integration of the filter method into the wrapper method leads to the hybrid algorithms exhibiting poor efficiency in classification as datasets grow in complexity. In this paper, an interactive feature selection framework based on state transition algorithm (STA) is proposed to address high-dimensional FS problems. In this framework, prior knowledge of the features is formed via the mutual information-based filter method. The STA is a powerful search engine that traverses the feature space in the wrapper stage. Moreover, an external trainer with prior knowledge will guide the exploration direction of STA in a simple but efficient way to accelerate the search process. And a self-adaptive mechanism is proposed to adjust the prior knowledge during the search process. Specifically, the external trainer establishes the interactive loop, while the self-adaptive mechanism aims to feed feature information back to the loop. In addition, the search process is prevented from being trapped in local optima by employing a multi-step STA, which allows for continuous transformations of the solution in probability. Finally, the proposed FS method is applied to various public classification datasets. The experimental results demonstrate that the proposed method is a highly competitive FS method, outperforming several state-of-the-art algorithms in generating an optimal subset of features.

Stem Detection from Terrestrial Laser Scanning Data with Features Selected via Stem-Based Evaluation

Terrestrial laser scanning (TLS) is an effective tool for extracting stem distribution, providing essential information for forest inventory and ecological studies while also assisting forest managers in monitoring and controlling forest stand density. A feature-based method is commonly integrated into the pipelines of stem detection, facilitating the transition from stem point to stem instance, but most studies focus on feature effectiveness from the point level, neglecting the relationship between stem point extraction and stem detection. In this paper, a feature-based method is proposed to identify stems from TLS data, with features selected from stem levels. Firstly, we propose a series of voxel-based features considering the stem characteristics under the forest. Then, based on the evaluation of some commonly used and proposed features, a stem-based feature selection method is proposed to select a suitable feature combination for stem detection by constructing and evaluating different combinations. Experiments are carried out on three plots with different terrain slopes and tree characteristics, each having a sample plot size of about 8000 m2. The results show that the voxel-based features can supplement the basic features, which improve the average accuracy of stem point extraction and stem detection by 9.5% and 1.2%, respectively. The feature set obtained by the proposed feature selection method achieves a better balance between accuracy and feature number compared with the point-based feature selection method and the features used in previous studies. Moreover, the accuracies of the proposed stem detection methods are also comparable to the three methods evaluated in the international TLS benchmarking project.

A Novel Feature Selection Method for Risk Management in High-Dimensional Time Series of Cryptocurrency Market

In this study, a novel approach for feature selection has been presented in order to overcome the challenge of classifying positive and negative risk prediction in the cryptocurrency market, which contains high fluctuation. This approach is based on maximizing information gain with simultaneously minimizing the similarity of selected features to achieve a proper feature set for improving classification accuracy. The proposed method was compared with other feature selection techniques, such as sequential and bidirectional feature selection, univariate feature selection, and least absolute shrinkage and selection operator. To evaluate the feature selection techniques, several classifiers were employed: XGBoost, k-nearest neighbor, support vector machine, random forest, logistic regression, long short-term memory, and deep neural networks. The features were elicited from the time series of Bitcoin, Binance, and Ethereum cryptocurrencies. The results of applying the selected features to different classifiers indicated that XGBoost and random forest provided better results on the time series datasets. Furthermore, the proposed feature selection method achieved the best results on two (out of three) cryptocurrencies. The accuracy in the best state varied between 55% to 68% for different time series. It is worth mentioning that preprocessed features were used in this research, meaning that raw data (candle data) were used to derive efficient features that can explain the problem and help the classifiers in predicting the labels.

A ReliefF guided binary equalization optimizer based on Z-type and reverse Z-type transfer functions for feature selection

Feature selection (FS) refers to the process of finding the most relevant feature subset according to some criteria. A ReliefF guided novel binary equilibrium optimizer (RG-NBEO) feature selection method was put forward to solve the FS problem. Z-type transfer functions and reverse Z-type transfer functions are proposed to convert the continuous searching space into binary searching space. The ReliefF guidance strategy was adopted to achieve the increasing and decreasing of features in the iterative process. The proposed feature selection method was tested by using 10 UCI standard datasets. Firstly, the Z-type and Rz-type transfer functions with different parameters are compared, and the transfer function with best performance is selected to compare with S-, V- and U-type transfer functions. Then the proposed transfer function is applied to five wrapped feature selection algorithms, and the simulation results are analyzed statistically. Finally, the proposed transfer function is applied to other machine learning algorithms to verify the applicability of the proposed function. Simulation results verify the effectiveness of the proposed Z-type and Rz-type transfer function and ReliefF guided binary equalization optimizer.

A structured combination of ensemble classifier and filter-based feature selection to improve breast cancer diagnosis.

Advances in technology have led to the emergence of computerized diagnostic systems as intelligent medical assistants. Machine learning approaches cannot replace professional humans, but they can change the treatment of diseases such as cancer and be used as medical assistants. Breast cancer treatment can be very effective, especially when the disease is detected in the early stages. Feature selection and classification are common data mining techniques in machine learning that can provide breast cancer diagnosis with high speed, low cost and high precision. This paper proposes a new intelligent approach using an integrated filter-evolutionary search-based feature selection and an optimized ensemble classifier for breast cancer diagnosis. The selected features mainly relate to the viable solution as the selected features are successfully used in the breast cancer disease classification process. The proposed feature selection method selects the most informative features from the original feature set by integrating adaptive thresholder information gain-based feature selection and evolutionary gravity-search-based feature selection. Meanwhile, classification model is done by proposing a new intelligent multi-layer perceptron neural network-based ensemble classifier. The simulation results show that the proposed method provides better performance compared to the state-of-the-art algorithms in terms of various criteria such as accuracy, sensitivity and specificity. Specifically, the proposed method achieves an average accuracy of 99.42% on WBCD, WDBC and WPBC datasets from Wisconsin database with only 56.7% of features. Systems based on intelligent medical assistants configured with machine learning approaches are an important step toward helping doctors to detect breast cancer early.

Improvement detection system on complex network using hybrid deep belief network and selection features

The challenge for intrusion detection system on internet of things networks (IDS-IoT) as a complex networks is the constant evolution of both large and small attack techniques and methods. The IoT network is growing very rapidly, resulting in very large and complex data. Complex data produces large data dimensions and is one of the problems of IDS in IoT networks. In this work, we propose a dimensional reduction method to improve the performance of IDS and find out the effect of the method on IDS-IoT using deep belief network (DBN). The proposed method for feature selection uses information gain (IG) and principle component analysis (PCA). The experiment of IDS-IoT with DBN successfully detects attacks on complex networks. The calculation of accuracy, precision, and recall, shows that the performance of the combination DBN with PCA is superior to DBN with information gain for Wi-Fi datasets. Meanwhile, the Xbee dataset with information gain is superior to using PCA. The final result of measuring the average value of accuracy, precision, and recall from each IDSDBN test for IoT is 99%. Other results also show that the proposed method has better performance than previous studies increasing by 4.12%.

Combining ensemble classification and integrated filter-evolutionary search for breast cancer diagnosis.

Breast cancer is one of the most common chronic diseases and the second cause of death among women, where its timely diagnosis plays an important role in survival and treatment. Advances in technology have led to the emergence of computerized diagnostic systems as intelligent medical assistants. In recent years, the development of these systems with data mining techniques and machine learning approaches has attracted the attention of researchers. This study presents a new hybrid approach using data mining techniques including feature selection and classification. Feature selection is configured using a method based on integrated filter-evolutionary search, where this method includes an evolutionary algorithm and information gain. The proposed feature selection method can provide the most suitable features by reducing dimensions for breast cancer classification. Meanwhile, we introduce an ensemble classification approach based on neural networks whose parameters are adjusted by an evolutionary algorithm. The effectiveness of the proposed method has been evaluated by several real datasets from the UCI machine learning repository. The results of simulations in terms of various metrics such as accuracy, precision and recall show that the proposed method is 12% better than the best existing methods on average. The evaluation of the proposed method confirms its effectiveness for breast cancer diagnosis as an intelligent medical assistant.