Effective Feature Subset Research Articles

FSCME: A Feature Selection Method Combining Copula Correlation and Maximal Information Coefficient by Entropy Weights.

Feature selection is a critical component of data mining and has garnered significant attention in recent years. However, feature selection methods based on information entropy often introduce complex mutual information forms to measure features, leading to increased redundancy and potential errors. To address this issue, we propose FSCME, a feature selection method combining Copula correlation (Ccor) and the maximum information coefficient (MIC) by entropy weights. The FSCME takes into consideration the relevance between features and labels, as well as the redundancy among candidate features and selected features. Therefore, the FSCME utilizes Ccor to measure the redundancy between features, while also estimating the relevance between features and labels. Meanwhile, the FSCME employs MIC to enhance the credibility of the correlation between features and labels. Moreover, this study employs the Entropy Weight Method (EWM) to evaluate and assign weights to the Ccor and MIC. The experimental results demonstrate that FSCME yields a more effective feature subset for subsequent clustering processes, significantly improving the classification performance compared to the other six feature selection methods.

Enhancing IoT (Internet of Things) feature selection: A two-stage approach via an improved whale optimization algorithm

Feature selection is a critical task for optimizing system performance and reducing computational overhead in the context of Internet of Things (IoT) applications. This paper presents a two-stage feature selection approach specifically designed for IoT scenarios. In the first stage, a variety of feature dimensionality reduction techniques are employed to significantly reduce the dimensionality of the original feature set by more than 50%. This process results in the creation of a highly effective feature subset, which serves as a solid foundation for subsequent feature selection in the second stage. In the second stage, feature selection is performed on the feature subset by an evolutionary algorithm to obtain high accuracy. Notably, we propose an improved whale optimization algorithm (WOA-HA), which incorporates several improvement factors such as a chaotic Hénon map mechanism (HMM), adaptive coefficient vector (ACV), and a binary operator. To assess the effectiveness of our approach, we compare the performance of WOA-HA with other evolutionary algorithms in terms of feature selection outcomes. Through extensive experiments, our proposed approach achieves an average accuracy of up to 95.5% on Aalto IoT dataset and 98.8% on RT-IoT 2022 dataset, respectively. Meanwhile, the average number of selected features reduced by about 82.5% on Aalto IoT dataset and about 62.3% in the RT-IoT 2022 dataset, respectively. Our proposed approach consistently outperforms other methods and achieves the best performance on most datasets with higher accuracy and fewer features.

Swing Trend Prediction of Main Guide Bearing in Hydropower Units Based on MFS-DCGNN.

Hydropower units are the core equipment of hydropower stations, and research on the fault prediction and health management of these units can help improve their safety, stability, and the level of reliable operation and can effectively reduce costs. Therefore, it is necessary to predict the swing trend of these units. Firstly, this study considers the influence of various factors, such as electrical, mechanical, and hydraulic swing factors, on the swing signal of the main guide bearing y-axis. Before swing trend prediction, the multi-index feature selection algorithm is used to obtain suitable state variables, and the low-dimensional effective feature subset is obtained using the Pearson correlation coefficient and distance correlation coefficient algorithms. Secondly, the dilated convolution graph neural network (DCGNN) algorithm, with a dilated convolution graph, is used to predict the swing trend of the main guide bearing. Existing GNN methods rely heavily on predefined graph structures for prediction. The DCGNN algorithm can solve the problem of spatial dependence between variables without defining the graph structure and provides the adjacency matrix of the graph learning layer simulation, avoiding the over-smoothing problem often seen in graph convolutional networks; furthermore, it effectively improves the prediction accuracy. The experimental results showed that, compared with the RNN-GRU, LSTNet, and TAP-LSTM algorithms, the MAEs of the DCGNN algorithm decreased by 6.05%, 6.32%, and 3.04%; the RMSEs decreased by 9.21%, 9.01%, and 2.83%; and the CORR values increased by 0.63%, 1.05%, and 0.37%, respectively. Thus, the prediction accuracy was effectively improved.

Inline classification of polymer films using Machine learning methods

Improving the sortability of plastic packaging film waste (PPFW) is crucial for increasing the recycling rate in Austria as they account for 150,000 t of the annually produced 300,000 t of plastic packaging waste. Currently PPFW is thermally recovered, as it is impossible to separate the mechanically recyclable monomaterial films from the non mechanically-recyclable multimaterial films. In this study, machine learning models capable of classifying inline into monolayer and multilayer films of PPFW according to their spectral fingerprint taken in transflection were created. Feature selection methods, like PCA and MRMR F-Tests, identified the most relevant spectral ranges for classification, that show the least redundancy and highest relevance. This effective subset of features decreases the required complexity of the model while reducing prediction time without compromising accuracy. The resulting models achieved a prediction accuracy of 85 % on unseen specimens with minimal prediction latency, effectively showing the inline applicability of these models in sorting aggregates.

Research on the Prediction of Tire Radial Load Based on 1D CNN and BiGRU

As an important indicator of vehicle systems, tire load is a key factor in the structural design and safety assessment of vehicles. Direct measurement methods for tire loads are expensive and complicated, while conventional load identification methods are limited by low accuracy and poor robustness. This study aimed to propose a radial load identification method for rubber-tired vehicles based on a one-dimensional convolutional neural network (1D CNN) and bidirectional gated recurrent unit (BiGRU). Considering a priori information of the radial load data of tires and based on the observability of the vehicle vibration system, the proposed method selected feature sets and then retained the effective feature subsets through feature selection to construct samples with multiple time steps as input and with a single time step as output for network training. In doing so, the load prediction results were obtained, and the theoretical model was modified by integrating prediction accuracy, generalization performance, and robustness. Compared with traditional algorithms, the proposed method could effectively reduce the error of load identification, improve adaptability under different operating conditions, and handle the measurement error of different noise levels, which are of practical application value in the engineering field.

Early Detection of Coronary Microvascular Dysfunction Using Machine Learning Algorithm Based on Vectorcardiography and Cardiodynamicsgram Features

PurposeAs a main etiology of myocardial ischemia, coronary microvascular dysfunction (CMD) can occur in patients with or without obstructive coronary artery disease. Currently, there is a lack of a non-invasive approach for early detection of CMD. AimWe aim to develop a multilayer perceptron (MLP) algorithm to achieve non-invasive early detection of CMD based on vectorcardiography (VCG) and cardiodynamicsgram (CDG) features. MethodsElectrocardiograms of 82 CMD patients and 107 healthy controls were collected and synthesized into VCGs. The VCGs' ST-T segments were extracted and fed into a deterministic learning algorithm to develop CDGs. Temporal heterogeneity index, spatial heterogeneity index, sample entropy, approximate entropy, and complexity index were extracted from VCGs' ST-T segments and CDGs, entitled as STT- and CDG-based features, respectively. The most effective feature subsets were determined from CDG-based, STT-based, and the combined features (i.e., all features) via the sequential backward selection algorithm as inputs for CDG-, STT-, and CDG-STT-based MLP models optimized with an improved sparrow search algorithm, respectively. Finally, the classification capacity of the corresponding models was evaluated via five-fold cross-validations and tested on a testing dataset to verify the optimal one. ResultsThe CDG-STT-based MLP model had significantly higher evaluated metrics than CDG- and STT-based ones on the validation dataset, with the accuracy, sensitivity, specificity, F1 score, and AUC of 0.904, 0.925, 0.870, 0.870, and 0.897 on the testing dataset respectively. ConclusionsThe MLP model based on VCG and CDG features showed high efficiency in identifying CMD. The CDG-STT-based MLP model may afford a potential computer-aided tool for non-invasive detection of CMD.

An Improved Binary Quantum-based Avian Navigation Optimizer Algorithm to Select Effective Feature Subset from Medical Data: A COVID-19 Case Study

An Improved Binary Quantum-based Avian Navigation Optimizer Algorithm to Select Effective Feature Subset from Medical Data: A COVID-19 Case Study

Combining deep features and hand-crafted features for abnormality detection in WCE images

In this paper, a computer-aided method is proposed for abnormality detection in Wireless Capsule Endoscopy (WCE) video frames. Common abnormalities in WCE images include ulcers, bleeding, Angiodysplasia, Lymphoid Hyperplasia, and polyp. In this paper, deep features and Hand-crafted features are combined to detect these abnormalities in WCE images. There are not sufficient images available to train deep structures, therefore the ResNet50 pre-trained model is used to extract deep features. Hand-crafted features are associated with color, shape, and texture. We used a novel idea to reveal unexpected color changes in the background due to existing lesions as a color feature set. Histogram of gradient (HOG) and local binary pattern (LBP) were used respectively for shape and texture features. They are extracted from the region of interest (ROI), i.e. suspicious region. The expectation Maximization (EM) algorithm is used to extract more distinct areas in the background as ROI. The expectation Maximization (EM) algorithm is configured in a way that can extract areas with a distinct texture and color as ROI. The EM algorithm is also initialized with a new fast method which leads to an increase in the accuracy of the method. A large number of features are created by the method, so the minimum redundancy maximum relevance approach is used to select a subset of more effective features. These selected features are then fed to a Support Vector Machine for classification. The results show that the proposed approach can detect mentioned abnormalities in WCE frames with the accuracy of 97.82%

E2H Distance-Weighted Minimum Reference Set for Numerical and Categorical Mixture Data and a Bayesian Swap Feature Selection Algorithm

Generally, when developing classification models using supervised learning methods (e.g., support vector machine, neural network, and decision tree), feature selection, as a pre-processing step, is essential to reduce calculation costs and improve the generalization scores. In this regard, the minimum reference set (MRS), which is a feature selection algorithm, can be used. The original MRS considers a feature subset as effective if it leads to the correct classification of all samples by using the 1-nearest neighbor algorithm based on small samples. However, the original MRS is only applicable to numerical features, and the distances between different classes cannot be considered. Therefore, herein, we propose a novel feature subset evaluation algorithm, referred to as the “E2H distance-weighted MRS,” which can be used for a mixture of numerical and categorical features and considers the distances between different classes in the evaluation. Moreover, a Bayesian swap feature selection algorithm, which is used to identify an effective feature subset, is also proposed. The effectiveness of the proposed methods is verified based on experiments conducted using artificially generated data comprising a mixture of numerical and categorical features.

Effective hybrid feature subset selection for multilevel datasets using decision tree classifiers

Feature selection is one of the most significant procedures in machine learning algorithms. It is particularly to improve the performance and prediction accuracy for complex data classification. This paper discusses a hybrid feature selection technique with the decision tree-based classification algorithm. The feature selected using information gain (IG) is combined with the features selected from ReliefF which generates the resultant feature subset. Then the resultant feature subset is in turn combined with a correlation-based feature selection (CFS) method to generate the aggregated feature subset. To perform classification accuracy on the aggregated feature subset, different decision trees-based classification algorithm such as C4.5, decision stumps, naive Bayes tree, and random forest with ten-fold cross-validation have been deployed. To check the prediction accuracy of the proposed work eight different multilevel University of California, Irvine (UCI) machine learning datasets have been used with minimum to maximum numbers of features. The main objective of the hybrid feature selection is to improve the classification accuracy, prediction and to reduce the execution time using standard datasets.

Effective hybrid feature subset selection for multilevel datasets using decision tree classifiers

Effective hybrid feature subset selection for multilevel datasets using decision tree classifiers

Research on Mutual Information Feature Selection Algorithm Based on Genetic Algorithm

<p>Feature selection is an important part of data preprocessing. Feature selection algorithms that use mutual information as evaluation can effectively handle different types of data, so it has been widely used. However, the potential relationship between relevance and redundancy in the evaluation criteria is often ignored, so that effective feature subsets cannot be selected. Optimize the evaluation criteria of the mutual information feature selection algorithm and propose a mutual information feature selection algorithm based on dynamic penalty factors (Dynamic Penalty Factor Mutual Information Feature Selection Algorithm, DPMFS). The penalty factor is dynamically calculated with different selected features, so as to achieve a relative balance between relevance and redundancy, and effectively play the synergy between relevance and redundancy, and select a suitable feature subset. Experimental results verify that the DPMFS algorithm can effectively improve the classification accuracy of the feature selection algorithm. Compared with the traditional chi-square, MIM and MIFS feature selection algorithms, the average classification accuracy of the random forest classifier for the six standard datasets is increased by 3.73%, 3.51% and 2.44%, respectively.</p> <p> </p>

Short-Term Wind Power Prediction Based on LightGBM and Meteorological Reanalysis

With the expansion of wind power grid integration, the challenges of sharp fluctuations and high uncertainty in preparing the power grid day-ahead plan and short-term dispatching are magnified. These challenges can be overcome through accurate short-term wind power process prediction based on mining historical operation data and taking full advantage of meteorological forecast information. In this paper, adopting the ERA5 reanalysis dataset as input, a short-term wind power prediction framework is proposed, combining light gradient boosting machine (LightGBM), mutual information coefficient (MIC) and nonparametric regression. Primarily, the reanalysis data of ERA5 provide more meteorological information for the framework, which can help improve the model input features. Furthermore, MIC can identify effective feature subsets from massive feature sets that significantly affect the output, enabling concise understanding of the output. Moreover, LightGBM is a prediction method with a stronger ability of goodness-of-fit, which can fully mine the effective information of wind power historical operation data to improve the prediction accuracy. Eventually, nonparametric regression expands the process prediction to interval prediction, which significantly improves the utility of the prediction results. To quantitatively analyze the prediction results, five evaluation criteria are used, namely, the Pearson correlation coefficient (CORR), the root mean square error (RMSE), the mean absolute error (MAE), the index of agreement (IA) and Kling–Gupta efficiency (KGE). Compared with support vector regression (SVR), random forest (RF) and extreme gradient boosting (XGBoost) models, the present framework can make full use of meteorological information and effectively improve the prediction accuracy, and the generated output prediction interval can also be used to promote the safe operation of power systems.

Fuzzy-rough hybrid dimensionality reduction

Dimensionality reduction including feature selection and feature extraction is a significant preprocessing method for data analysis, which is an important application of fuzzy rough set theory. Linear discriminant analysis (LDA) is a commonly used supervised feature extraction method, which can generate transformations while retaining as much class discrimination information as possible. However, it is worth noting that linear discriminant analysis is not interpretable. To improve the interpretability of the model and avoid losing part of the original feature information when only using feature selection for dimensionality reduction, this paper proposes a hybrid dimensionality reduction method based on fuzzy rough set and linear discriminant analysis. It can obtain a reduction set that contains complementary transformation features and original features. First, the semi-feature set is generated by criterion based on the normalized fuzzy joint mutual information, and the hybrid feature set is further induced by applying LDA to the feature set. Then, the redundancy between features is evaluated by the normalized fuzzy mutual information. The significance of a feature is defined by the proposed normalized effective classification information amount and the symmetric effective classification information amount. Next, the priority value between features according to the significance of them is defined to delete the redundant feature from the hybrid feature set. Furthermore, a hybrid dimensionality reduction algorithm based on fuzzy rough set is designed further. Finally, the performance of the proposed method is compared with some methods. The experimental results show that the proposed algorithm can obtain effective feature subset, which improve the performance of a classifier.

Binary Aquila Optimizer for Selecting Effective Features from Medical Data: A COVID-19 Case Study

Medical technological advancements have led to the creation of various large datasets with numerous attributes. The presence of redundant and irrelevant features in datasets negatively influences algorithms and leads to decreases in the performance of the algorithms. Using effective features in data mining and analyzing tasks such as classification can increase the accuracy of the results and relevant decisions made by decision-makers using them. This increase can become more acute when dealing with challenging, large-scale problems in medical applications. Nature-inspired metaheuristics show superior performance in finding optimal feature subsets in the literature. As a seminal attempt, a wrapper feature selection approach is presented on the basis of the newly proposed Aquila optimizer (AO) in this work. In this regard, the wrapper approach uses AO as a search algorithm in order to discover the most effective feature subset. S-shaped binary Aquila optimizer (SBAO) and V-shaped binary Aquila optimizer (VBAO) are two binary algorithms suggested for feature selection in medical datasets. Binary position vectors are generated utilizing S- and V-shaped transfer functions while the search space stays continuous. The suggested algorithms are compared to six recent binary optimization algorithms on seven benchmark medical datasets. In comparison to the comparative algorithms, the gained results demonstrate that using both proposed BAO variants can improve the classification accuracy on these medical datasets. The proposed algorithm is also tested on the real-dataset COVID-19. The findings testified that SBAO outperforms comparative algorithms regarding the least number of selected features with the highest accuracy.

A feature selection method with feature ranking using genetic programming

Feature selection is a data processing method which aims to select effective feature subsets from original features. Feature selection based on evolutionary computation (EC) algorithms can often achieve better classification performance because of their global search ability. However, feature selection methods using EC cannot get rid of invalid features effectively. A small number of invalid features still exist till the termination of the algorithms. In this paper, a feature selection method using genetic programming (GP) combined with feature ranking (FRFS) is proposed. It is assumed that the more the original features appear in the GP individuals' terminal nodes, the more valuable these features are. To further decrease the number of selected features, FRFS using a multi-criteria fitness function which is named as MFRFS is investigated. Experiments on 15 datasets show that FRFS can obtain higher classification performance with smaller number of features compared with the feature selection method without feature ranking. MFRFS further reduces the number of features while maintaining the classification performance compared with FRFS. Comparisons with five benchmark techniques show that MFRFS can achieve better classification performance.

An effective feature subset selection approach based on Jeffries-Matusita distance for multiclass problems

Jeffries-Matusita (JM) distance, a transformation of the Bhattacharyya distance, is a widely used measure of the spectral separability distance between the two class density functions and is generally used as a class separability measure. It can be considered to have good potential to be used for evaluation of the effectiveness of a feature in discriminating two classes. The capability of JM distance as a ranking based feature selection technique for binary classification problems has been verified in some research works as well as in our earlier work. It was found by our simulation experiments with benchmark data sets that JM distance works equally well compared to other popular feature ranking methods based on mutual information, information gain or Relief. Extension of JM distance measure for feature ranking in multiclass problems has also been reported in the literature. But all of them are basically rank based approaches which deliver the ranking of the features and do not automatically produce the final optimal feature subset. In this work, a novel heuristic approach for finding out the optimum feature subset from JM distance based ranked feature lists for multiclass problems have been developed without explicitly using any specific search technique. The proposed approach integrates the extension of JM measure for multiclass problems and the selection of the final optimal feature subset in a unified process. The performance of the proposed algorithm has been evaluated by simulation experiments with benchmark data sets in comparison with two other previously developed multiclass JM distance measures (weighted average JM distance and another multiclass extension equivalent to Bhattacharyya bound) and some other popular filter based feature ranking algorithms. It is found that the proposed algorithm performs better in terms of classification accuracy, F-measure, AUC with a reduced set of features and computational cost.

An Intelligent Ensemble Classification Method For Spam Diagnosis in Social Networks

In recent years, the destructive behavior of social networks spammers has seriously threatened the information security of ordinary users. To reduce this threat, many researchers have extracted the behavioral characteristics of spam and obtained good results based on machine learning algorithms to identify them. However, most of these studies use a single classification technique that often works differently for different spam data. In this paper, an intelligent ensemble classification method for social networks spam detection is introduced. The proposed heterogeneous ensemble learning framework is based on stack generalization and uses an evolutionary algorithm to improve the modeling process and reduce complexity. In particular, particle swarm optimization has been used as an evolutionary algorithm to optimize model parameters to reduce model complexity. These parameters include a subset of effective features and a subset of the most appropriate single classification techniques. The SPAM E-mail dataset used in this article contains the correct and effective features in spam prediction. Experimental results show that the proposed algorithm effectively improves the detection rate of spam and performs better than the methods used.

A novel TMGWO–SLBNC‐based multidimensional feature subset selection and classification framework for frequent diagnosis of breast lesion abnormalities

The selection of optimal subset of features from high-dimensional data sets still remains a major challenge during breast cancer detection and categorization. There exist several research works regarding optimal feature subset selection from high-dimensional data sets, but the obtained results are not satisfying when multidimensional data sets (MDDs) are employed in large amount during disease analysis. In this article, an effective feature subset selection and classification method suitable for MDD is proposed. At first, the important and distinct features are extracted from the mammogram images using a Deep Neural Network with wrapper-based extraction technique. Then, a novel two-phase mutation strategy integrated with grey wolf optimizer algorithm is employed for selecting the most relevant feature subsets. Finally, a learning-based semilazy Bayesian network classifier with parallel implementation is proposed for the precise categorization of the breast cancer stages. The proposed method is executed in MATLAB platform and analyzed using mammogram images taken from MAMMOSET database. The proposed method is likened with three state-of-the-art existing feature subset selection and classification approaches for validating the efficiency of the proposed approach. For Data set 1, the proposed method shows an accuracy of 90%, 92% and 98%, which is better than the existing methods taken for comparison. Also for Data set 2, the proposed method shows an accuracy of 97%, 91% and 94%, which is much better than the accuracy achieved by the existing approaches. Thus, the proposed approach outperforms the compared existing approaches by providing better precision, recall, F-measure, specificity and accuracy.

Modelling Electricity Consumption Forecasting Using the Markov Process and Hybrid Features Selection

Given the problem of electrical energy storage, it is critical to predict the amount of load required in order to have a reliable and stable power distribution network. Predicting electricity consumption of subscribers and analyzing their consumption behavior under the influence of various factors and time variables is important. Given the large volume of subscriber consumption data and the effective factors, it is only possible to analyze the data using new information technology tools such as data mining. In this paper, feature selection, clustering and Markov process techniques are used to model and predict the power consumption data of subscribers. First, the selection of a subset of effective features is based on the combined PCA approach and the Firefly algorithm. Subscribers are then clustered based on the features selected by the K-means. Finally, subscriber behavior patterns are modeled to predict consumption using the Markov process on high-risk clusters. This study is simulated based on the data of electricity subscribers in Bushehr-Iran Power Distribution Company. The simulation results show the superiority of the proposed model over other similar algorithms such as LASSO-QRNN and HyFIS. The accuracy of power consumption prediction in the proposed method is about 1% compared to LASSO-QRNN and about 0.5% compared to HyFIS.