Feature Selection Approach Research Articles

A novel profit-based validity index approach for feature selection in credit risk prediction

<abstract> <p>Establishing a reasonable and effective feature system is the basis of credit risk early warning. Whether the system design is appropriate directly determines the accuracy of the credit risk evaluation results. In this paper, we proposed a feature system through a validity index with maximum discrimination and commercial banks' loan profit maximization. First, the first objective function is the minimum validity index constructed by the intra-class, between-class, and partition coefficients. The maximum difference between the right income and wrong cost is taken as the second objective function to obtain the optimal feature combination. Second, the feature weights are obtained by calculating the change in profit after deleting each feature with replacement to the sum of all change values. An empirical analysis of 3, 425 listed companies from <italic>t</italic>-1 to <italic>t</italic>-5 time windows reveals that five groups of feature systems selected from 614 features can distinguish between defaults and non-defaults. Compared with 14 other models, it is found that the feature systems can provide at least five years' prediction and enable financial institutions to obtain the maximum profit.</p> </abstract>

A two-step feature selection approach for identifying SNPs associated with colorectal cancer

A two-step feature selection approach for identifying SNPs associated with colorectal cancer

Related Factors Mining of Diabetes Complications Based on Manifold-Constrained Multi-Label Feature Selection.

The primary cause of mortality among individuals with diabetes stems from complications. Identifying related factors for these complications holds immense potential for early prevention. Previous research predominantly employed traditional machine-learning techniques to establish prediction models utilizing medical indicators for related factor selection. However, uncovering the intricate correlations among complication labels and identifying similar characteristics among medical indicators has been challenging. We propose a novel embedded multi-label feature selection approach called LCFSM(Label Cosine and Feature Similar Manifold) to address the issue. LCFSM introduces manifold constraints into the objective function to uncover risk factors associated with diabetes complications. Label cosine similarity is set to optimize feature weights, forming label manifold constraints. Similarly, feature manifold constraints are established to utilize feature kernel similarity in optimizing feature weights. LCFSM formulates an objective function based on the l2,1 regularized Least Squares and previous manifolds constraints, employing the Sylvester equationfor convergence assurance. The experimental evaluation compares LCFSM against eight baselines, demonstrating superior performance in top-10 feature selection and feature stacking.LCFSM is applied to identify primary risk factors for diabetes complications. Related factors involve Electromyogram, Urine Routine Protein Positive, etc, offering valuable insights for early treatment.

Joint Structured Bipartite Graph and Row-Sparse Projection for Large-Scale Feature Selection.

Feature selection plays an important role in data analysis, yet traditional graph-based methods often produce suboptimal results. These methods typically follow a two-stage process: constructing a graph with data-to-data affinities or a bipartite graph with data-to-anchor affinities and independently selecting features based on their scores. In this article, a large-scale feature selection approach based on structured bipartite graph and row-sparse projection (RS 2 BLFS) is proposed to overcome this limitation. RS 2 BLFS integrates the construction of a structured bipartite graph consisting of c connected components into row-sparse projection learning with k nonzero rows. This integration allows for the joint selection of an optimal feature subset in an unsupervised manner. Notably, the c connected components of the structured bipartite graph correspond to c clusters, each with multiple subcluster centers. This feature makes RS 2 BLFS particularly effective for feature selection and clustering on nonspherical large-scale data. An algorithm with theoretical analysis is developed to solve the optimization problem involved in RS 2 BLFS. Experimental results on synthetic and real-world datasets confirm its effectiveness in feature selection tasks.

High Accuracy COVID-19 Prediction Using Optimized Union Ensemble Feature Selection Approach

High Accuracy COVID-19 Prediction Using Optimized Union Ensemble Feature Selection Approach

Enhancing Photovoltaic Reliability: A Global and Local Feature Selection Approach with Improved Harris Hawks Optimization for Efficient Hotspot Detection Using Infrared Imaging

The photovoltaic (PV) systems’ inherent ability to transform solar light directly into electrical energy has contributed to their increasing popularity. However, malfunctions can reduce system dependability. Therefore, rapid hotspot identification is critical for efficient, dependable, and risk‐free PV operation. This work presents a method for determining the most optimal hybrid features using the infrared (IR) images of PV panels for hotspot and fault detection. The information at the global (texture, HoG, and color histograms) and local (local binary pattern, SURF, and KAZE) levels were extracted from the IR images of PV panels using a uniform window size of 8 × 8. A binary improved Harris hawks optimization (b‐IHHO) optimal feature selection strategy was used to get the optimal feature subset for model training using PV IR images. The IR images of PV were acquired to test the presented framework. The findings suggested that the proposed framework can classify the IR images of solar panels with an accuracy of 98.41% with lesser feature vector size into three classes (normal, hotspot, and defective). Furthermore, the findings were also compared with the latest literature. The presented technique plays a vital role in carbon‐free cities and is simple to adopt for PV system inspection.

Minimizing time complexity for IOT-IDS based feature selection approach

Minimizing time complexity for IOT-IDS based feature selection approach

Hybrid Feature Selection Approach to Finding Optimal Feature Subsets for Vehicle Lateral Velocity Estimation

Hybrid Feature Selection Approach to Finding Optimal Feature Subsets for Vehicle Lateral Velocity Estimation

Malware Classification Using Machine Learning Models

Malware, a form of harmful software, poses a significant threat to victims by compromising data integrity and facilitating unauthorized access. Analogous to the COVID virus’s impact on the human body, untreated malware can cause ongoing internal harm until system limits are exhausted. Before proceeding with any further steps in our research, a crucial initial task is to explore various types of benchmark malware datasets. To address the challenge of malware dataset selection a comprehensive search for benchmark datasets conducted and selected CIC-MalMem-2022 dataset. Our dataset included 29,298 samples encompassing various malware families and benign instances. The proposed framework uses six different types of machine learning algorithms, namely Logistic Regression, Support Vector Machine, K-Nearest Neighbor, Random Forest, Naive Bayes, and Decision Tree for the classification of malware. Correlation based feature selection approach is used with a threshold value of 0.6 to identify the most important features and eliminate redundant and irrelevant features. The six models were cross-validated by 10-fold cross validation method. After evaluating the ML models using different classifiers, considering metrics such as accuracy, precision, recall, and F1-score, it was evident that the Decision Tree model outperformed the others. The achieved classification metrics for accuracy, precision, recall, and F1-score were as follows: 0.9994240, 0.9996162, 0.9992325, and 0.9994882, respectively. However, in the testing phase, the Decision Tree and KNN classifiers exhibited comparable performance, with both achieving an accuracy of 0. 9998293.These outcomes demonstrate the model’s ability in identifying and categorizing malware, thereby contributing to enhanced data security.

A Novel Unsupervised Feature Selection for High-Dimensional Data Based on FCM and k -Nearest Neighbor Rough Sets.

Large amounts of high-dimensional unlabeled data typically contain only a small portion of truly effective information. Consequently, the issue of unsupervised feature selection methods has gained significant attention in research. However, current unsupervised feature selection approaches face limitations when dealing with datasets that exhibit uneven density, and they also require substantial computational time. To address this problem, this research article proposes a feature extraction technique that combines the Fuzzy C-Means (FCM) and k -nearest neighbor rough sets. FCM is a clustering algorithm grounded in fuzzy theory, which takes into account the inherent data structure and the correlations between different features. Consequently, FCM is particularly well-suited for datasets with uneven density. Our proposed method consists of three steps. First, the FCM algorithm is used to cluster the unlabeled data. Second, a measure that evaluates the importance of features is defined and sorted based on the clustering results. Finally, redundant features are filtered using k -nearest neighbor rough sets while retaining important features, significantly reducing the running time. In addition, we designed the feature selection algorithm (KND-UFS) and conducted experiments on 12 public datasets. We compared KND-UFS with eight existing algorithms in terms of running time, classification accuracy, and the number of selected features. The experimental results provided strong evidence supporting the superior performance of the KND-UFS algorithm.

An equilibrium optimizer with deep recurrent neural networks enabled intrusion detection in secure cyber-physical systems

<abstract> <p>Cyber-physical systems (CPSs) are characterized by their integration of physical processes with computational and communication components. These systems are utilized in various critical infrastructure sectors, including energy, healthcare, transportation, and manufacturing, making them attractive targets for cyberattacks. Intrusion detection system (IDS) has played a pivotal role in identifying and mitigating cyber threats in CPS environments. Intrusion detection in secure CPSs is a critical component of ensuring the integrity, availability, and safety of these systems. The deep learning (DL) algorithm is extremely applicable for detecting cyberattacks on IDS in CPS systems. As a core element of network security defense, cyberattacks can change and breach the security of network systems, and then an objective of IDS is to identify anomalous behaviors and act properly to defend the network from outside attacks. Deep learning (DL) and Machine learning (ML) algorithms are crucial for the present IDS. We introduced an Equilibrium Optimizer with a Deep Recurrent Neural Networks Enabled Intrusion Detection (EODRNN-ID) technique in the Secure CPS platform. The main objective of the EODRNN-ID method concentrates mostly on the detection and classification of intrusive actions from the platform of CPS. During the proposed EODRNN-ID method, a min-max normalization algorithm takes place to scale the input dataset. Besides, the EODRNN-ID method involves EO-based feature selection approach to choose the feature and lessen high dimensionality problem. For intrusion detection, the EODRNN-ID technique exploits the DRNN model. Finally, the hyperparameter related to the DRNN model can be tuned by the chimp optimization algorithm (COA). The simulation study of the EODRNN-ID methodology is verified on a benchmark data. Extensive results display the significant performance of the EODRNN-ID algorithm when compared to existing techniques.</p> </abstract>

Heart Disease Diagnostics Using Meta‐Learning‐Based Hybrid Feature Selection

Heart disease, encompassing a range of conditions affecting the heart, remains a leading cause of morbidity and mortality worldwide. The urgent need for precise diagnostic techniques is crucial for improving patient outcomes, as early and accurate diagnosis can significantly influence the effectiveness of treatment and management strategies. This study introduces an innovative approach to diagnosing heart disease by combining classifiers in a meta‐learning‐based approach and utilizing advanced feature selection methods in a hybrid model. Using the extensive heart disease dataset provided by the IEEE DataPort, our method aims to enhance the accuracy of diagnosis by gradually refining the selection of relevant features at two separate stages. Initially, in level 0, a collection of filter‐based algorithms, such as mutual information, relief, ANOVA, systematic uncertainty, and minimum redundancy‐maximum relevance, is utilized to identify a subset of relevant features. Subsequently, the Sequential Forward Selection (SFS) wrapper method is employed in level 1 to further fine‐tune the feature set. The model utilizes a meta‐learning‐based ensemble strategy for classification, combining basic classifiers such as K‐nearest neighbor (KNN), Naïve Bayes (NB), decision tree (DT), and extreme gradient boost (XGBoost) in level 0. In addition, a logistic regression (LR) meta‐classifier is incorporated at level 1. The proposed hybrid technique has been tested on the heart disease dataset, and the results show that the meta‐learning‐based hybrid feature selection approach performs exceptionally well in terms of performance metrics. The model produces impressive results with only eight precisely chosen features, including an accuracy of 96.2185%, precision of 96.0317%, recall of 96.8%, F1‐score of 96.4143%, and an area under the curve (AUC) of 0.9619. Furthermore, our approach significantly outperforms state‐of‐the‐art techniques, indicating its potential to revolutionize heart disease diagnosis and improve patient care.

A New Feature Selection Metric Based on Rough Sets and Information Gain in Text Classification

In text classification, taking words in text documents as features creates a very high dimensional feature space. This is known as the high dimensionality problem in text classification. The most common and effective way to solve this problem is to select an ideal subset of features using a feature selection approach. In this paper, a new feature selection approach called Rough Information Gain (RIG) is presented as a solution to the high dimensionality problem. Rough Information Gain extracts hidden and meaningful patterns in text data with the help of Rough Sets and computes a score value based on these patterns. The proposed approach utilizes the selection strategy of the Information Gain Selection (IG) approach when pattern extraction is completely uncertain. To demonstrate the performance of the Rough Information Gain in the experimental studies, the Micro-F1 success metric is used to compare with Information Gain Selection (IG), Chi-Square (CHI2), Gini Coefficient (GI), Discriminative Feature Selector (DFS) approaches. The proposed Rough Information Gain approach outperforms the other methods in terms of performance, according to the results.

Offline Handwriting Signature Verification: A Transfer Learning and Feature Selection Approach

Offline Handwriting Signature Verification: A Transfer Learning and Feature Selection Approach

Heterogeneous Ensemble Feature Selection and Multilevel Ensemble Approach to Machine Learning Phishing Attack Detection

Over the decade, technology has presented human facets with easiest means of accomplishing complex tasks seamlessly, especially in the area of communication. Malicious and vicious links are consciously doctored to resemble the original and sent through emails to millions of users at once at a lower price. Since the emergence of phishing and its cohorts, every solution and means to mitigate the attacks has proven unsuccessful due to the dynamic nature of the attacks. Meanwhile, machine learning (ML) is adopted as the right antidote to phishing detection, with its performance based on diverse steps, especially feature selection. Most studies in the problem domain concentrate more on model optimization than sourcing for a reliable feature selection system and fail to integrate a reliable feature selection along with the classification model. The systems are fed with low-quality data that hampers the performance of such models. The authors noticed the contribution of feature selection to the performance of machine learning models and developed a novel Heterogeneous Ensemble Feature Selection (HEFS) framework for multilevel ensemble machine learning-based phishing detection. In HEFS, three filter-based statistical techniques were exploited to produce a primary subset of phishing features, and the variable selected by each of the techniques was automatically aggregated to produce the baseline features. The selection of the techniques is to overcome each limitation since their ranking principles are different. The experiment revealed that the multilevel ensemble (stacked) on the baseline features outperformed others with an accuracy of 98.8%., including multilevel model on each filter-based method.

Union With Recursive Feature Elimination: A Feature Selection Framework to Improve the Classification Performance of Multicategory Causes of Death in Colorectal Cancer

Despite the use of machine learning tools, it is challenging to properly model cause-specific deaths in colorectal cancer (CRC) patients and choose appropriate treatments. Here, we propose an interesting feature selection framework, namely union with recursive feature elimination (U-RFE), to select the union feature sets that are crucial in CRC progression-specific mortality using The Cancer Genome Atlas (TCGA) dataset. Based on the union feature sets, we compared the performance of 5 classification algorithms, including logistic regression (LR), support vector machines (SVM), random forest (RF), eXtreme gradient boosting (XGBoost), and Stacking, to identify the best model for classifying 4-category deaths. In the first stage of U-RFE, LR, SVM, and RF were used as base estimators to obtain subsets containing the same number of features but not exactly the same specific features. Union analysis of the subsets was then performed to determine the final union feature set, effectively combining the advantages of different algorithms. We found that the U-RFE framework could improve various models’ performance. Stacking outperformed LR, SVM, RF, and XGBoost in most scenarios. When the target feature number of the RFE was set to 50 and the union feature set contained 298 deterministic features, the Stacking model achieved F1_weighted, Recall_weighted, Precision_weighted, Accuracy, and Matthews correlation coefficient of 0.851, 0.864, 0.854, 0.864, and 0.717, respectively. The performance of the minority categories was also significantly improved. Therefore, this recursive feature elimination–based approach of feature selection improves performances of classifying CRC deaths using clinical and omics data or those using other data with high feature redundancy and imbalance.

A hybrid feature selection approach for urinary tract infection detection and prediction in IoT-Fog environment

Urinary Tract Infections (UTIs) are a prevalent health concern experienced by millions of individuals worldwide and have a significant impact on overall well-being. The clinical presentation of UTIs varies depending on their location and severity. Common symptoms include discomfort during urination, increased frequency, a compelling urge to urinate, discomfort in the lower abdomen, and the presence of blood in the urine (hematuria). Severe cases may be accompanied by fever, pain in the flank region, and other systemic symptoms, indicative of an upper UTI. Precise diagnosis of UTIs is achieved through clinical evaluation and laboratory tests. The primary aim of this research is to utilize appropriate Machine Learning (ML)-based algorithms to predict Urinary Tract Infections (UTIs). The ultimate goal is to develop a predictive model that can be effectively implemented in a smart toilet system. By achieving this objective, this research seeks to provide an innovative and pragmatic solution for UTI prediction, harnessing the potential of ML algorithms in IoT-Fog settings to enhance healthcare and promote public health. This paper introduces a hybrid approach that combines feature selection methods and employs the Guided Regularized Random Forest (GRRF) classification algorithm to aid in the diagnosis of UTIs. A UTI dataset was created using data from routine examinations and definitive diagnostic outcomes for UTI patients. Dimensionality reduction was carried out using Principal Component Analysis (PCA), while feature selection was performed using K-best and Lasso CV techniques. Through the proposed strategy, this study achieved a remarkable accuracy rate of 98.8% and a precision rate of 98.90% in UTI identification. Future UTI prevention and treatment plans must be optimized via further research and ongoing efforts to overcome antibiotic resistance.

Elastic net-based high dimensional data selection for regression

High-dimensional feature selection is of particular interest to researchers. In some domains, such as microarray data, it is quite common for a group of highly correlated explanatory variables to be of equal importance for inclusion in the predictive model. This paper proposes a new hybrid feature selection approach that integrates feature screening based on Kendall’s tau and Elastic Net regularized regression (K-EN). K-EN as an approach that embeds the Elastic Net, has the advantage of the grouping effect, which automatically includes all the highly correlated variables in the group. The K-EN approach offers insightful solutions to high-dimensional regression problems and improves Elastic Net performance since the screening phase is preceded by a step that further reduces the number of explanatory variables by removing those that disagree with the target based on Kendall’s tau. The use of Kendall’s tau further enhances Elastic Net performance, as it is robust enough to handle heavy-tailed distributions, non-parametric models, outliers, and non-normal data with greater ease. K-EN is therefore a time-saving approach. The proposed algorithm is evaluated on four simulation scenarios and four publicly available datasets, including riboflavin, eyedata, Longley, and Boston Housing, and achieves 0.2528, 0.0098, 0.1007, and 0.4121 respectively as the Mean Squared Error (MSE). K-EN’s MSEs are the best compared to those achieved by the state-of-the-art approaches reviewed in this paper. In addition, K-EN selects up to 100% of relevant features when run on simulated data.

A comparative study of cyber security intrusion detection in healthcare systems

Due to the proliferation of network devices and the presence of sensitive information, healthcare systems have become prime targets for cyber attackers. Therefore, it is crucial to design an efficient and accurate intrusion detection system (IDS) specifically tailored for healthcare systems. In this regard, we conducted a comprehensive comparative study on network security intrusion detection in healthcare systems. In order to tackle the challenges arising from information redundancy and noise in feature selection, we developed the Maximum Information Coefficient (MIC) method to effectively analyse the nonlinear relationships among traffic features. This method was utilized in a comparative analysis involving ten models on three datasets. The experiments demonstrated that the detection models using MIC-based feature selection outperformed other feature selection approaches, especially when applied to the WUSTL-EHMS-2020 dataset, which includes patients' biometric features. The MIC-enhanced Extreme Gradient Boosting detection model achieved remarkable results, attaining an accuracy of 95.01%, precision of 94.94%, and recall of 95.01%. These findings underscore the efficacy of our comparative study in safeguarding healthcare systems against cyber attacks. Furthermore, our study highlights the importance of feature selection and the incorporation of patient biometric features in healthcare IDS. It is imperative for medical managers to consider these factors when making informed decisions regarding cyber security measures.

Golden eagle based improved Att-BiLSTM model for big data classification with hybrid feature extraction and feature selection techniques

ABSTRACT The remarkable development in technology has led to the increase of massive big data. Machine learning processes provide a way for investigators to examine and particularly classify big data. Besides, several machine learning models rely on powerful feature extraction and feature selection techniques for their success. In this paper, a big data classification approach is developed using an optimized deep learning classifier integrated with hybrid feature extraction and feature selection approaches. The proposed technique uses local linear embedding-based kernel principal component analysis and perturbation theory, respectively, to extract more representative data and select the appropriate features from the big data environment. In addition, the feature selection task is fine-tuned by using perturbation theory through heuristic search based on their output accuracy. This feature selection heuristic search method is analysed with five recent heuristic optimization algorithms for deciding the final feature subset. Finally, the data are categorized through an attention-based bidirectional long short-term memory classifier that is optimized with a golden eagle-inspired algorithm. The performance of the proposed model is experimentally verified on publicly accessible datasets. From the experimental outcomes, it is demonstrated that the proposed framework is capable of classifying large datasets with more than 90% accuracy.