Feature Selection Method For Classification Research Articles

Sparse Trace Ratio LDA for Supervised Feature Selection.

Classification is a fundamental task in the field of data mining. Unfortunately, high-dimensional data often degrade the performance of classification. To solve this problem, dimensionality reduction is usually adopted as an essential preprocessing technique, which can be divided into feature extraction and feature selection. Due to the ability to obtain category discrimination, linear discriminant analysis (LDA) is recognized as a classic feature extraction method for classification. Compared with feature extraction, feature selection has plenty of advantages in many applications. If we can integrate the discrimination of LDA and the advantages of feature selection, it is bound to play an important role in the classification of high-dimensional data. Motivated by the idea, we propose a supervised feature selection method for classification. It combines trace ratio LDA with l2,p -norm regularization and imposes the orthogonal constraint on the projection matrix. The learned row-sparse projection matrix can be used to select discriminative features. Then, we present an optimization algorithm to solve the proposed method. Finally, the extensive experiments on both synthetic and real-world datasets indicate the effectiveness of the proposed method.

A simple and efficient filter feature selection method via document-term matrix unitization

Text processing tasks commonly grapple with the challenge of high dimensionality. One of the most effective solutions to this challenge is to preprocess text data through feature selection methods. Feature selection can select the most advantageous features for subsequent operations (e.g., classification) from the native feature space of the text. This process effectively trims the feature space’s dimensionality, enhancing subsequent operations’ efficiency and accuracy. This paper proposes a straightforward and efficient filter feature selection method based on document-term matrix unitization (DTMU) for text processing. Diverging from previous filter feature selection methods that concentrate on scoring criteria definition, our method achieves more optimal feature selection by unitizing each column of the document-term matrix. This approach mitigates feature-to-feature influence and reinforces the role of the weighting proportion within the features. Subsequently, our scoring criterion subtracts the sum of weights for negative samples from positive samples and takes the absolute value. We conduct numerical experiments to compare DTMU with four advanced filter feature selection methods: max–min ratio metric, proportional rough feature selector, least loss, and relative discrimination criterion, along with two classical filter feature selection methods: Chi-square and information gain. The experiments are performed on four ten-thousand-dimensional feature space datasets: book, dvd, music, movie and two thousand-dimensional feature space datasets: imdb, amazon_cells, sourced from Amazon product reviews and movie reviews. Experimental findings demonstrate that DTMU selects more advantageous features for subsequent operations and achieves a higher dimensionality reduction rate than those of the other six methods used for comparison. Moreover, DTMU exhibits robust generalization capabilities across various classifiers and dimensional datasets. Notably, the average CPU time for a single run of DTMU is measured at 1.455 s.

Hibrit Sık Kullanılan Öğe Kümeleme ile Makine Öğrenmesi Tabanlı Ağ Sızma Tespiti

With the development and expansion of computer networks day by day and the diversity of software developed, the damage that possible attacks can cause is increasing beyond the predictions. Intrusion Detection Systems (STS/IDS) are one of the effective defense tools against these potential attacks that are constantly increasing and diversifying. The ultimate goal is to train these systems with various artificial intelligence methods, to detect subsequent attacks in real time and to take the necessary precautions. In this study, classical feature selection methods and Frequent Item Set Mining were used in feature selection in a hybrid model, and it was aimed to classify network traffic data for normal and attack by using many machine learning methods, including Logistic Regression, with the final features obtained. The method uses a data set originally containing 85 features to make a decision while making this classification. These attributes are extracted using CICFlowMeter from a PCAP file where network traffic is recorded. The results show that the proposed method in the study classifies more than 225000 records in the data set with a success rate of 97.68%.

Analysis of treatment response based on 1.5T magnetic resonance imaging texture analysis in stereotactic body radiotherapy of hepatocellular carcinoma

PurposeMagnetic Resonance Image (MRI)-guided stereotactic body radiotherapy (MRgSBRT) is a promising technique in the treatment of hepatocellular carcinoma (HCC). However, treatment response varies among patients. The aim of this study was to evaluate the treatment response using radiomics features extracted from 1.5 T MRI in HCC patients treated with SBRT. Materials and methods19 patients with biopsy-proven HCC who were treated with SBRT by 1.5 T MRI-guided were enrolled, all of whom were treated with 8–10 fractions with biological effective dose (BED) range of 95.2–107.1 Gy. We acquired images of pretreatment, delivered BEDs of 35–40 Gy, and delivered BEDs of 55–60 Gy. We combined three classic feature selection methods: least absolute shrinkage and selection operator (LASSO), extreme gradient boosting (XGBoost), and random forest (RF) to extract the intersection of radiomics features from the gross tumor volume (GTV) and features were averaged over the three fractions. We used the receiver operating characteristic (ROC) area under curve (AUC) obtained using leave-one-out cross-validation (LOOCV) to assess the predictive capacity. ResultsSeven patients showed response to treatment based on post-treatment imaging studies. The optimum intersection of radiomics features selected by three methods was Neighborhood Grey Tone Difference Matrix (NGTDM) contrast and NGTDM strength. The logistic regression-based model achieved an AUC of 0.821 (95% confidence interval, 0.618–1). ConclusionsRadiomics features containing biological prognostic information extracted during MRgSBRT could predict tumor treatment response, facilitating stratification of high-risk patients and providing clinical application value for individualized care of patients with different response to treatment.

A quantum feature selection framework via ground state preparation

Traditional feature selection methods face the challenges of increasing time complexity and local optima. In previous works, many classical feature selection methods were accelerated through quantum algorithms. However, these approaches still inherit the constraints of these classical methods as they do not address the issue of local minima. Here, we propose a novel quantum feature selection framework based on the classifier’s result, which utilizes Hamiltonian encoding and a ground state preparation algorithm. Numerical experiments are conducted on real-world datasets from the finance and medicine domains. Moreover, the results demonstrate that the proposed method produces the same or better classification accuracy on the classifier than the original data without feature selection. Overall, our approach presents a promising solution to feature selection using quantum computing.

A Novel Multi-Objective Based Feature Selection Method for Response Prediction

Accurate response prediction is essential towards personalized treatment in radiation therapy. Excessive imaging features, extracted from medical images, pose a great challenge in radiomic analyses. Feature selection is an essential step to remove redundant and irrelevant features for model construction. We proposed a novel multi-objective based radiomic feature selection method (MRMOPSO), where the number of features, sensitivity, and specificity are jointly considered as optimization objectives for feature selection. The MRMOPSO innovated by three aspects: 1) Fisher score initialize the feature population to speed up the convergence; 2) Min-redundancy particle generation operations to reduce the redundancy between radiomic features, a truncation strategy was also introduced; 3) Particle selection operation guided by elitism strategies to improve local search ability of the algorithm. We evaluated the effectiveness of the proposed MRMOPSO method using a cohort of oropharyngeal cancer patients from The Cancer Imaging Archive (TCIA). 357 patients were used for model training and additional 64 patients were used for independent evaluation. The proposed methods were compared with (a) classical feature selection methods, i.e., Lasso, minimal-redundancy-maximal-relevance criterion (mRMR), F-score, and mutual information (MI), (b) single-objective feature selection methods, i.e., genetic algorithm (GA), particle swarm optimization algorithm (PSO) and (c) multi-objective feature selection methods, i.e., multiple objective particle swarm optimization (MOPSO), nondominated sorting genetic algorithm II (NSGA II). The other feature selection methods yielded AUCs, sensitivity, specificity of (0.48-0.71), (0.49-0.86), (0.33-0.67), respectively. The MRMOPSO achieved significantly highly AUC of 0.84 with smaller number of selected features on the independent dataset (Table 1). Additionally, the MRMOPSO remarkably improved the sensitivity (0.81), specificity (0.81) and achieved an excellent balance between sensitively and specificity. We demonstrated a novel multi-objective based radiomic feature selection method. The proposed algorithm effectively reduced feature dimension, and achieved superior AUC with simultaneous improved sensitivity and specificity, for radiomic response prediction.

IsoFrog: a reversible jump Markov Chain Monte Carlofeature selection-based method for predicting isoform functions.

A single gene may yield several isoforms with different functions through alternative splicing. Continuous efforts are devoted to developing machine-learning methods to predict isoform functions. However, existing methods do not consider the relevance of each feature to specific functions and ignore the noise caused by the irrelevant features. In this case, we hypothesize that constructing a feature selection framework to extract the function-relevant features might help improve the model accuracy in isoform function prediction. In this article, we present a feature selection-based approach named IsoFrog to predict isoform functions. First, IsoFrog adopts a reversible jump Markov Chain Monte Carlo (RJMCMC)-based feature selection framework to assess the feature importance to gene functions. Second, a sequential feature selection procedure is applied to select a subset of function-relevant features. This strategy screens the relevant features for the specific function while eliminating irrelevant ones, improving the effectiveness of the input features. Then, the selected features are input into our proposed method modified domain-invariant partial least squares, which prioritizes the most likely positive isoform for each positive MIG and utilizes diPLS for isoform function prediction. Tested on three datasets, our method achieves superior performance over six state-of-the-art methods, and the RJMCMC-based feature selection framework outperforms three classic feature selection methods. We expect this proposed methodology will promote the identification of isoform functions and further inspire the development of new methods. IsoFrog is freely available at https://github.com/genemine/IsoFrog.

Causal Feature Selection in the Presence of Sample Selection Bias

Almost all existing causal feature selection methods are proposed without considering the problem of sample selection bias. However, in practice, as data-gathering process cannot be fully controlled, sample selection bias often occurs, leading to spurious correlations between features and the class variable, which seriously deteriorates the performance of those existing methods. In this article, we study the problem of causal feature selection under sample selection bias and propose a novel Progressive Causal Feature Selection (PCFS) algorithm which has three phases. First, PCFS learns the sample weights to balance the treated group and control group distributions corresponding to each feature for removing spurious correlations. Second, based on the sample weights, PCFS uses a weighted cross-entropy model to estimate the causal effect of each feature and removes some irrelevant features from the confounder set. Third, PCFS progressively repeats the first two phases to remove more irrelevant features and finally obtains a causal feature set. Using synthetic and real-world datasets, the experiments have validated the effectiveness of PCFS, in comparison with several state-of-the-art classical and causal feature selection methods.

New criteria for wrapper feature selection to enhance bearing fault classification

Classification is a critical task in many fields, including signal processing and data analysis. The accuracy and stability of classification results can be improved by selecting the most relevant features from the data. In this paper, a new criterion for feature selection using wrapper method is proposed, which is based on the evaluation of the classification results according to the accuracy and stability (standard deviation) of each class and the number of selected features. The proposed method is evaluated using Random Forest (RF) and Ant Colony Optimization (ACO) algorithms on a benchmark dataset. Results show that the proposed method outperforms classical feature selection methods in terms of accuracy and stability of classification results, especially for the difficult-to-classify combined damage class. This study demonstrates the effectiveness of the proposed new wrapper feature selection criterion to improve the performance of classification algorithms with higher stability (STD: C1 = 0.5, C2 = 0.8, C3 = 0.6, C4 = 1.8) and better accuracy (average C1 = 98.5%, C2 = 96.6%, C3 = 9.5%, C4 = 93) for the both; the statoric current and the vibration signal compared to other techniques. Machine learning methods had proven their efficiency in time-varying machines fault diagnosis when taking vibration signals and statoric currents extracted features as inputs. However, the use of the both demonstrated a higher robustness and a remarkable superiority.

An adaptive loss backward feature elimination method for class-imbalanced and mixed-type data in medical diagnosis

Feature selection or elimination methods have been widely used to mine high-dimensional data from many fields such as biomedicine. However, classical feature selection methods usually focus on class-balanced and numerical data, and the classification performance is poor when processing class-imbalanced or mixed-type data. Moreover, existing studies hardly consider the co-occurrence of both class-imbalance and mixture of numerical and nominal features. In this paper, we propose an adaptive loss backward feature elimination method for class-imbalanced and mixed-type data (FEIM). In FEIM, a technique to convert and decompose nominal features is employed to process mixed-type data. Meanwhile, an adaptive loss function is proposed to make the classifier pay more attention to the minority class. In experiments, the effectiveness and stability of FEIM are synthetically evaluated against seven medical diagnostic datasets in terms of the measurements of F measurement and precision, and the results of feature selection are analyzed in detail. The experimental results show that, compared with three common methods (Recursive Feature Elimination SVM (RFE-SVM), Relief, and Holdout Algorithm for Backward Feature Elimination (HO-BFE)), FEIM can achieve a higher classification accuracy and is more stable when the parameters are changed. On the other hand, the feature selection results of FEIM are more meaningful by processing class-imbalanced and mixed-type data. Therefore, FEIM is a competitive and effective method to process feature elimination for class-imbalanced and mixed-type biomedical data for in medical diagnosis. The supporting information is available in the electronic supplementary material.

A multi-objective based radiomics feature selection method for response prediction following radiotherapy

Objective. Radiomics contains a large amount of mineable information extracted from medical images, which has important significance in treatment response prediction for personalized treatment. Radiomics analyses generally involve high dimensions and redundant features, feature selection is essential for construction of prediction models. Approach. We proposed a novel multi-objective based radiomics feature selection method (MRMOPSO), where the number of features, sensitivity, and specificity are jointly considered as optimization objectives in feature selection. The MRMOPSO innovated in the following three aspects: (1) Fisher score to initialize the population to speed up the convergence; (2) Min-redundancy particle generation operations to reduce the redundancy between radiomics features, a truncation strategy was introduced to further reduce the number of features effectively; (3) Particle selection operations guided by elitism strategies to improve local search ability of the algorithm. We evaluated the effectiveness of the MRMOPSO by using a multi-institution oropharyngeal cancer dataset from The Cancer Imaging Archive. 357 patients were used for model training and cross validation, an additional 64 patients were used for evaluation. Main results. The area under the curve (AUC) of our method achieved AUCs of 0.82 and 0.84 for cross validation and independent dataset, respectively. Compared with classical feature selection methods, the AUC of MRMOPSO is significantly higher than the Lasso (AUC = 0.74, p-value = 0.02), minimal-redundancy-maximal-relevance criterion (mRMR) (AUC = 0.73, p-value = 0.05), F-score (AUC = 0.48, p-value < 0.01), and mutual information (AUC = 0.69, p-value < 0.01) methods. Compared to single-objective methods, the AUC of MRMOPSO is 12% higher than those of the genetic algorithm (GA) (AUC = 0.68, p-value = 0.02) and particle swarm optimization algorithm (AUC = 0.72, p-value = 0.05) methods. Compared to other multi-objective feature selection methods, the AUC of MRMOPSO is 14% higher than those of multiple objective particle swarm optimization (MOPSO) (AUC = 0.68, p-value = 0.02) and nondominated sorting genetic algorithm II (NSGA2) (AUC = 0.70, p-value = 0.03). Significance. We proposed a multi-objective based radiomics feature selection method. Compared to conventional feature reduction algorithms, the proposed algorithm effectively reduced feature dimension, and achieved superior performance, with improved sensitivity and specificity, for response prediction in radiotherapy.

Group-shrinkage feature selection with a spatial network for mining DNA methylation data

Identifying disease-related biomarkers from high-dimensional DNA methylation data helps in reducing early screening costs and inferring pathogenesis mechanisms. Good discovery results have been achieved through spatial correlation methods of methylation sites, group-based regularization, and network constraints. However, these methods still have some key limitations as they cannot exclude isolated differential sites and only consider adjacent site ordering. Therefore, we propose a group-shrinkage feature selection algorithm to encourage the selection of clustered sites and discourage the selection of isolated differential sites. Specifically, a network-guided group-shrinkage strategy is developed to penalize weakly-correlated isolated methylation sites through a network structure constraint. The spatial network is constructed based on spatial correlation information of DNA methylation sites, where this information accounts for the uneven site distribution. The experimental simulations and applications demonstrated that the proposed method outperforms the advanced regularization methods, especially in rejecting isolated methylation sites; hence this study provides an efficient and clinical-valuable method for biomarker candidate discovery in DNA methylation data. Additionally, the proposed method exhibits enhanced reliability due to introducing biological prior knowledge into a regularization-based feature selection framework and could promote more research in the integration between biological prior knowledge and classical feature selection methods, thus facilitating their clinical application. Our source codes will be released at https://github.com/SJTUBME-QianLab/Group-shrinkage-Spatial-Network once this manuscript is accepted for publication.

A hybrid Artificial Immune optimization for high-dimensional feature selection

For high-dimensional data, the traditional feature selection method is slightly inadequate. At present, most of the existing hybrid search methods have problems of high computational cost and unsatisfactory feature reduction rate. In this paper, a hybrid feature selection method based on artificial immune algorithm optimization (HFSIA) is proposed to solve the feature reduction problem of high-dimensional data. This method combines the filter method with the metaheuristic-based search strategy more effectively. Inspired by biological research results, the method introduces a lethal mutation mechanism and a Cauchy mutation operator with adaptive adjustment factors to improve the search performance of the algorithm. In addition, this method introduces an adaptive adjustment factor in the population update stage to improve the problem of insufficient diversity of the original algorithm. The effective combination of these mechanisms enables the algorithm to obtain better search capability at a lower computational cost. Experimental comparisons with 23 state-of-the-art feature selection methods are conducted on 22 high-dimensional benchmark datasets. The results show that the computational cost of HFSIA is comparable to 5 classical feature selection methods known for their speed. Moreover, it achieves a higher average classification accuracy than 18 hybrid feature selection methods reported in the latest literature with the best feature reduction rate.

An Efficient Hybrid Feature Selection Method Using the Artificial Immune Algorithm for High-Dimensional Data.

Feature selection provides the optimal subset of features for data mining models. However, current feature selection methods for high-dimensional data also require a better balance between feature subset quality and computational cost. In this paper, an efficient hybrid feature selection method (HFIA) based on artificial immune algorithm optimization is proposed to solve the feature selection problem of high-dimensional data. The algorithm combines filter algorithms and improves clone selection algorithms to explore the feature space of high-dimensional data. According to the target requirements of feature selection, combined with biological research results, this method introduces the lethal mutation mechanism and the Cauchy operator to improve the search performance of the algorithm. Moreover, the adaptive adjustment factor is introduced in the mutation and update phases of the algorithm. The effective combination of these mechanisms enables the algorithm to obtain a better search ability and lower computational costs. Experimental comparisons with 19 state-of-the-art feature selection methods are conducted on 25 high-dimensional benchmark datasets. The results show that the feature reduction rate for all datasets is above 99%, and the performance improvement for the classifier is between 5% and 48.33%. Compared with the five classical filtering feature selection methods, the computational cost of HFIA is lower than the two of them, and it is far better than these five algorithms in terms of the feature reduction rate and classification accuracy improvement. Compared with the 14 hybrid feature selection methods reported in the latest literature, the average winning rates in terms of classification accuracy, feature reduction rate, and computational cost are 85.83%, 88.33%, and 96.67%, respectively.

An improved feature selection method for classification on incomplete data: Non-negative latent factor-incorporated duplicate MIC

Feature selection is an important data pre-processing procedure in the classification task, whose main purpose is to remove negligible/redundant features to reduce the computational cost, while improving the performance of the subsequent machine learning method. However, most feature selection methods can only work well with the complete data, but show poorly in the presence of the missing information, especially with a high missing rate, due probably to the severe irrelevance and redundancy of features. To address this issue, we proposed a novel feature selection method, namely non-negative latent factor incorporated duplicate maximal information coefficient (NLF-DMIC), which improves the effectiveness of feature selection for the classification of incomplete data. The NLF-DMIC method is fulfilled by the following three steps: (1) select category-friendly features by MIC based on “partial sample strategy” roughly; (2) use the NLF model to make an imputation for the missing data in terms of the filtered features; and (3) select features again by an improved maximal information coefficient (i.e., low-redundancy MIC, LMIC) method on the complete dataset. Finally, experiments on an artificially synthetic dataset and 8 real datasets show that the proposed NLF-DMIC method outperforms some state-of-the-art feature selection methods.

Predicting the Risk of Overweight and Obesity in Madrid—A Binary Classification Approach with Evolutionary Feature Selection

In this paper, we experimented with a set of machine-learning classifiers for predicting the risk of a person being overweight or obese, taking into account his/her dietary habits and socioeconomic information. We investigate with ten different machine-learning algorithms combined with four feature-selection strategies (two evolutionary feature-selection methods, one feature selection from the literature, and no feature selection). We tackle the problem under a binary classification approach with evolutionary feature selection. In particular, we use a genetic algorithm to select the set of variables (features) that optimize the accuracy of the classifiers. As an additional contribution, we designed a variant of the Stud GA, a particular structure of the selection operator of individuals where a reduced set of elitist solutions dominate the process. The genetic algorithm uses a direct binary encoding, allowing a more efficient evaluation of the individuals. We use a dataset with information from more than 1170 people in the Spanish Region of Madrid. Both evolutionary and classical feature-selection methods were successfully applied to Gradient Boosting and Decision Tree algorithms, reaching values up to 79% and increasing the average accuracy by two points, respectively.

A Novel Feature Selection Method for Classification of Medical Data Using Filters, Wrappers, and Embedded Approaches

Feature selection is the process of identifying the most relevant features from the given data having a large feature space. Microarray datasets are comprised of high-quality features and very few samples of data. Feature selection is performed on such datasets to identify the optimal feature subset. The major goal of feature selection is to improve the accuracy by identifying a minimal feature subset. For this purpose, the proposed research focused on analyzing and identifying effective feature selection algorithms. A novel framework is proposed which utilizes different feature selection methods from filters, wrappers, and embedded algorithms. Furthermore, classification is then performed on selected features to classify the data using a support vector machine (SVM) classifier. Two publically available benchmark datasets are used, i.e., the Microarray dataset and the Cleveland Heart Disease dataset, for experimentation and analysis, and they are archived from the UCI data repository. The performance of SVM is analyzed using accuracy, sensitivity, specificity, and f-measure. The accuracy of 94.45% and 91% is achieved on each dataset, respectively.

A Hybrid Whale Genetic Algorithm for Feature Selection in Biomedical Dataset

One of the major concerns of biomedical datasets is high dimensionality. These dimensions may include irrelevant and redundant features that adversely affect the performance of classification algorithms. Extensive research has been done in the area of machine learning to handle high dimensionality. In literature, feature selection algorithms have been developed for this purpose. In this paper, a hybrid nature-inspired algorithm is proposed which is a combination of whale optimization algorithm and genetic algorithm for feature selection. The proposed algorithm is applied to four microarray datasets and one DNA sequence dataset and compared with classical feature selection methods. In all the algorithms decision tree classifier is mainly employed. To reach an approximate best solution and remove local solutions, the exploitation and exploration phases are balanced efficiently in the proposed algorithm. The convergence speed in the proposed algorithm is accelerated by the adaptive mechanisms. Overall results employ better performance on the majority of datasets.

A comparison between classical and new proposed feature selection methods for attention level recognition in disordered children

Lack of attention is a chronic behavior in ADHD (Attention Deficit Hyperactivity Disorder) and ASD (Autism Spectrum Disorder). Our goal is to develop a reliable method for the detection of inattention with high accuracy and low time consumption to be used in real time neurofeedback. The new applied methods for inattention in children are EMD (Empirical Mode Decomposition) with difference time series (Dt) and MRA (Multi Resolution Analysis). EMD is a method of breaking down a signal into ‘modes’ (IMFs) representing its different frequency components. Furthermore, MRA strikes balance between temporal and frequency resolution through localizing the EEG signal in frequency domain of interest (beta range) by wavelet decomposition or EMD and then retains time domain information using FD. As the results demonstrate, in intermediate and severe level cases of inattention, EMD_Dt technique is the most accurate. In mild level cases of inattention MRA (wavelet + FD) technique performance is better than EMD_Dt. However, the time consumption of the MRA (wavelet + FD) technique is fifteen times larger than EMD_Dt technique. EMD_Dt is the best technique as it requires less processing time which is the most important factor in neurofeedback, furthermore, clinician concerned more with severe and intermediate level of inattention to be treated.

Evaluation of Feature Selection Methods for Classification of Epileptic Seizure EEG Signals.

Epilepsy is a disease that decreases the quality of life of patients; it is also among the most common neurological diseases. Several studies have approached the classification and prediction of seizures by using electroencephalographic data and machine learning techniques. A large diversity of features has been extracted from electroencephalograms to perform classification tasks; therefore, it is important to use feature selection methods to select those that leverage pattern recognition. In this study, the performance of a set of feature selection methods was compared across different classification models; the classification task consisted of the detection of ictal activity from the CHB-MIT and Siena Scalp EEG databases. The comparison was implemented for different feature sets and the number of features. Furthermore, the similarity between selected feature subsets across classification models was evaluated. The best F1-score (0.90) was reported by the K-nearest neighbor along with the CHB-MIT dataset. Results showed that none of the feature selection methods clearly outperformed the rest of the methods, as the performance was notably affected by the classifier, dataset, and feature set. Two of the combinations (classifier/feature selection method) reporting the best results were K-nearest neighbor/support vector machine and random forest/embedded random forest.