Applying Feature Selection Methods Research Articles

A novel approach for the effective prediction of cardiovascular disease using applied artificial intelligence techniques.

The objective of this research is to develop an effective cardiovascular disease prediction framework using machine learning techniques and to achieve high accuracy for the prediction of cardiovascular disease. In this paper, we have utilized machine learning algorithms to predict cardiovascular disease on the basis of symptoms such as chest pain, age and blood pressure. This study incorporated five distinct datasets: Heart UCI, Stroke, Heart Statlog, Framingham and Coronary Heart dataset obtained from online sources. For the implementation of the framework, RapidMiner tool was used. The three-step approach includes pre-processing of the dataset, applying feature selection method on pre-processed dataset and then applying classification methods for prediction of results. We addressed missing values by replacing them with mean, and class imbalance was handled using sample bootstrapping. Various machine learning classifiers were applied out of which random forest with AdaBoost dataset using 10-fold cross-validation provided the high accuracy. The proposed model provides the highest accuracy of 99.48% on Heart Statlog, 93.90% on Heart UCI, 96.25% on Stroke dataset, 86% on Framingham dataset and 78.36% on Coronary heart disease dataset, respectively. In conclusion, the results of the study have shown remarkable potential of the proposed framework. By handling imbalance and missing values, a significantly accurate framework has been established that could effectively contribute to the prediction of cardiovascular disease at early stages.

A Comparative Study of Feature Selection Technique for Predicting the Professional Tennis Matches Outcome in a Grand Slam Tournament

Tennis is one of the world's most played sports, attracting many spectators to participate in the game. One of the most essential strokes in a tennis match is serve performance. This research is intended to determine the most critical strokes in tennis serve performance in predicting the tennis match outcome. This research focuses on the Grand Slam Tournaments of the Australian Open, French Open, Wimbledon, and US Open. The data are collected on the tennis serve performances such as Percentage First Serve In (PFSI), Percentage First Serve Won (PFSW), Percentage First Serve Return Won (PFSRW), Aces, and many more. For one tournament, it consists of 254 observations. This study applied feature selection methods available in R programming, such as Correlation Matrix, Relative Importance Metrics, Boruta, MARS, and cForest. Selecting the most essential and correlated variables with the match status can improve the model and help produce better results. This might help the practitioners to apply this method to obtain the closest result to the actual outcome when we include the most correlated variables in the model. From the result obtained, variables of first and second serve, either win on serve or return serve, are identified as the most critical attributes in the tennis match. As a future implication, we suggest that these are all the factors the players need to pay extra attention to in winning the tennis match.

Comparative Analysis of Polycystic Ovary Syndrome Detection Using Machine Learning Algorithms

INTRODUCTION: Polycystic Ovary Syndrome is a condition in which the ovaries manufacture androgen, seen in small traces, resulting in the production of cysts. Menstrual cycle abnormalities, clinical and/or biochemical hyperandrogenism, and the presence of polycystic ovaries on ultrasound should all be used to diagnose PCOS. PCOS appears to be a multifaceted illness influenced by both genetic and environmental factors and the symptoms include excessive hair on the face and body, weight gain, voice changes, skin type changes, and irregular periods.
 OBJECTIVES: This is the objective of this paper is to identify PCOS in its initial stage.
 METHODS: To address this issue the study proposes a comparison of various machine learning algorithms and optimization techniques Among which GSCV gave the best result of 94% accuracy, followed by TPOT with 91% accuracy. Additionally, we also applied Feature selection methods to eliminate zero-importance features to increase the accuracy of algorithms.
 RESULTS: The main results obtained in this paper This study explored various Feature selection techniques, ML and DL models. It is shown that Grid Search CV and TPOT classifier were best classifiers with 94% and 91% respectively.
 CONCLUSION: These are the conclusions of this paper and this study will explore various DL methodologies and try to find out best optimal results for the PCOS Detection. And also, to develop an PCOS detection application to keep track of menstrual cycles and track activities and symptoms for PCOS.

PREDICTING THE RISK FACTORS OF HYPERTENSION AMONG INDIAN OLDER POPULATION: A MACHINE LEARNING APPROACH

Abstract Hypertension, often known as high blood pressure, is a disorder in which the pressure inside the blood arteries remain consistently high. It is a common chronic illness that often occurs along with other morbidities. Risk prediction of hypertension further contributes to the detection, development of interventions, and treatment. The current study used different machine learning models to identify the high-risk factors of developing hypertension using a large-scale survey dataset LASI. A sample of 58,757 collected through the survey aged 45 and above used for model training and testing. We identified 35 risk factors from the literature and then applied feature selection methods to identify only the important ones. Based on the selected features, three models were built i.e., random forest, decision tree, and logistic regression, and the hyper-parameters were optimized on the training set using 10-fold cross-validation. The performance of the models was measured using area under the curve value, precision, recall, and F measure. The random forest classifier outperformed the rest of the two algorithms with an AUC score of 0.64, a precision of 0.65, a recall score of 0.57, and an F1 score of 0.61. Body Mass Index, diabetes, age, type of work, parents having hypertension, and multimorbidity are the primary risk factors for hypertension. The results can be further applied to make algorithms to predict hypertension among an older individuals based on their demographic, health and behavioral factors. Predictive models for hypertension can aid in determining the degree of community interventions required, ensuring a favorable outcome.

Feature Minimization for Diabetic Disorders High Performances Prediction System-based on Random Forest Tree

Human organ failure due to high blood sugar is considered a chronic disease. Early prediction might reduce or prevent complications due to such disorders, especially with recent machine-learning improvement techniques and the availability of electronic data from different sources. The number of diabetic patients roughly increased and may reach more than 600 million by twenty years. Transforming data into valuable and helpful information is an effort for researchers to improve the performance of ML techniques. This paper applies several types of sampling to predict 1000 samples with attributes and three diabetes class types (Random Forest tree, Hoeffding tree, LWL, NB updatable, and support vector Machine). This paper focused on most parameters that affected overall prediction accuracy. ML performances have been measured depending on the accuracy and mean absolute error for several cross-validation values before Feature reduction and after feature minimization by applying feature selection methods. It shows that Gender, Age, Blood Sugar Level (HbA1c), Triglycerides (TG), and Body Mass Index (BMI) are the most impact attributes applied. It is also shown that the Random Forest tree was the best method (97.7 and 98.6 %) with and without feature minimization, respectively, but it has a higher performance by omitting some unbalanced features from the diabetic dataset. Weight minimization has also been applied to techniques like SVM to obtain a better-searching plane and a robust model. In addition, this study specifies which parameters have weight minimization with the required analysis. Also, the feature selection method was applied to gain memory and reduce time.

Morphological Signal Processing for Phenotype Recognition of Human Pluripotent Stem Cells Using Machine Learning Methods.

Human pluripotent stem cells have the potential for unlimited proliferation and controlled differentiation into various somatic cells, making them a unique tool for regenerative and personalized medicine. Determining the best clone selection is a challenging problem in this field and requires new sensing instruments and methods able to automatically assess the state of a growing colony ('phenotype') and make decisions about its destiny. One possible solution for such label-free, non-invasive assessment is to make phase-contrast images and/or videos of growing stem cell colonies, process the morphological parameters ('morphological portrait', or signal), link this information to the colony phenotype, and initiate an automated protocol for the colony selection. As a step in implementing this strategy, we used machine learning methods to find an effective model for classifying the human pluripotent stem cell colonies of three lines according to their morphological phenotype ('good' or 'bad'), using morphological parameters from the previously published data as predictors. We found that the model using cellular morphological parameters as predictors and artificial neural networks as the classification method produced the best average accuracy of phenotype prediction (67%). When morphological parameters of colonies were used as predictors, logistic regression was the most effective classification method (75% average accuracy). Combining the morphological parameters of cells and colonies resulted in the most effective model, with a 99% average accuracy of phenotype prediction. Random forest was the most efficient classification method for the combined data. We applied feature selection methods and showed that different morphological parameters were important for phenotype recognition via either cellular or colonial parameters. Our results indicate a necessity for retaining both cellular and colonial morphological information for predicting the phenotype and provide an optimal choice for the machine learning method. The classification models reported in this study could be used as a basis for developing and/or improving automated solutions to control the quality of human pluripotent stem cells for medical purposes.

Machine learning endometrial cancer risk prediction model: integrating guidelines of European Society for Medical Oncology with the tumor immune framework

ObjectiveCurrent prognostic factors for endometrial cancer are not sufficient to predict recurrence in early stages. Treatment choices are based on the prognostic factors included in the risk classes defined by...

Cervical Cancer Diagnosis Using Stacked Ensemble Model and Optimized Feature Selection: An Explainable Artificial Intelligence Approach

Cervical cancer affects more than half a million women worldwide each year and causes over 300,000 deaths. The main goals of this paper are to study the effect of applying feature selection methods with stacking models for the prediction of cervical cancer, propose stacking ensemble learning that combines different models with meta-learners to predict cervical cancer, and explore the black-box of the stacking model with the best-optimized features using explainable artificial intelligence (XAI). A cervical cancer dataset from the machine learning repository (UCI) that is highly imbalanced and contains missing values is used. Therefore, SMOTE-Tomek was used to combine under-sampling and over-sampling to handle imbalanced data, and pre-processing steps are implemented to hold missing values. Bayesian optimization optimizes models and selects the best model architecture. Chi-square scores, recursive feature removal, and tree-based feature selection are three feature selection techniques that are applied to the dataset For determining the factors that are most crucial for predicting cervical cancer, the stacking model is extended to multiple levels: Level 1 (multiple base learners) and Level 2 (meta-learner). At Level 1, stacking (training and testing stacking) is employed for combining the output of multi-base models, while training stacking is used to train meta-learner models at level 2. Testing stacking is used to evaluate meta-learner models. The results showed that based on the selected features from recursive feature elimination (RFE), the stacking model has higher accuracy, precision, recall, f1-score, and AUC. Furthermore, To assure the efficiency, efficacy, and reliability of the produced model, local and global explanations are provided.

A Comparative Study Of Algorithmic Efficiency Of Feature Selection Algorithm On Microarray

A key challenge before classification can take place is feature selection. An effective feature selection method would increase classification accuracy and simultaneously reduce computation costs and time. A variety of filter approaches, along with different search algorithms, were considered in this study. Five traditional classifiers were evaluated on the selected gene subsets: Random Forest, Sequential minimal optimization algorithm, Naive Bayes, Decision Trees, and K-Nearest Neighbour. The datasets chosen for this analysis are the microarray gene expression data of two types of cancers: Acute Lymphocytic Leukaemia (ALL)/Acute Myeloid Leukaemia (AML) and Lung cancer. According to the experimental results, a fuzzy rough subset combined with Genetic Search selects optimal relevant gene subsets and produces significantly good classifier accuracy. Compared to classical classifiers described here, this research finds that Random Forest classifiers yield 94.33% on the raw dataset and 100% classifier accuracy after applying feature selection methods. Utilizing conventional methods like Precision, Recall, F-Score, and Region of Characteristics, MCC Matthews correlation coefficient, results are validated.

Classification of Interpretation Differences in String Quartets Based on the Origin of Performers

Music Information Retrieval aims at extracting relevant features from music material, while Music Performance Analysis uses these features to perform semi-automated music analysis. Examples of interdisciplinary cooperation are, for example, various classification tasks—from recognizing specific performances, musical structures, and composers to identifying music genres. However, some classification problems have not been addressed yet. In this paper, we focus on classifying string quartet music interpretations based on the origin of performers. Our dataset consists of string quartets from composers A. Dvořák, L. Janáček, and B. Smetana. After transferring timing information from reference recordings to all target recordings, we apply feature selection methods to rank the significance of features. As the main contribution, we show that there are indeed origin-based tempo differences, distinguishable by measure durations, by which performances may be identified. Furthermore, we train a machine learning classifier to predict the performers’ origin. We evaluate three different experimental scenarios and achieve higher classification accuracy compared to the baseline using synchronized measure positions.

LIME-based ensemble machine for predicting performance status of patients with liver cancer.

The Eastern Cooperative Oncology Group performance status (ECOG PS) is a widely recognized measure used to assess the functional abilities of cancer patients and predict their prognosis. It plays a crucial role in guiding treatment decisions made by physicians. This study aimed to build a stacking ensemble-based prognosis predictor model for predicting the ECOG PS of a liver cancer patient undergoing treatment. We used Light Gradient Boosting Machine (LightGBM) as the meta-model, and five base models, including Random Forest (RF), Extra Trees (ET), AdaBoost (Ada), Gradient Boosting Machine (GBM), and Extreme Gradient Boosting (XGBoost). After preprocessing the data and applying feature selection method, the stacking ensemble model was trained using 1622 liver cancer patients' data and 46 variables. We also integrated the stacking ensemble model with a LIME-based explainable model to obtain model prediction explainability. According to the research, the best combination of the stacking ensemble model is ET + XGBoost + RF + GBM + Ada + LightGBM and achieved a ROC AUC of 0.9826 on the training set and 0.9675 on the test set. This explainable stacking ensemble model can become a helpful tool for objectively predicting ECOG PS in liver cancer patients and aiding healthcare practitioners to adapt their treatment approach more effectively.

An Application of Feature Selection Methods to Compare the Performances of Classification Algorithms

In this study, it is aimed to determine fewer and significant variables with the help of feature selection methods among a large number of variables in the data discussed. Feature selection methods are effective methods that have great importance in statistics in recent years and provide great convenience to researchers. Depending on the technique used in the method, different numbers of variables are included in the model, but the correct classification rates may vary. In this context, being able to express the variables in a data set with a large number of variables of interest with a high classification percentage and fewer new variables makes positive contributions to issues such as time and cost. The variables in the data set discussed in this study were firstly analyzed with different feature selection methods and new data sets were created. Afterwards, these new data sets containing different numbers of variables were analyzed with different machine learning techniques and the best machine learning technique was determined. In this study, chronic kidney disease data were handled and the variables in the data set were classified with different feature selection methods. When the results of the study are examined, the highest classification rate with 99.75% was obtained from the correlation-based feature selection method, which includes the random forest and multilayer perceptron technique, and the filter method, which includes the k-nearest neighbor technique, with the same rate. The results of the study show that the percentage of correct classification obtained from this study is higher than that of other studies, when compared with other studies using the same dataset.

Active Learning for Multi-way Sensitivity Analysis with Application to Disease Screening Modeling.

Sensitivity analysis is an important aspect of model development as it can be used to assess the level of confidence that is associated with the outcomes of a study. In many practical problems, sensitivity analysis involves evaluating a large number of parameter combinations which may require an extensive amount of time and resources. However, such a computational burden can be avoided by identifying smaller subsets of parameter combinations that can be later used to generate the desired outcomes for other parameter combinations. In this study, we investigate machine learning-based approaches for speeding up the sensitivity analysis. Furthermore, we apply feature selection methods to identify the relative importance of quantitative model parameters in terms of their predictive ability on the outcomes. Finally, we highlight the effectiveness of active learning strategies in improving the sensitivity analysis processes by reducing the total number of quantitative model runs required to construct a high-performance prediction model. Our experiments on two datasets obtained from the sensitivity analysis performed for two disease screening modeling studies indicate that ensemble methods such as Random Forests and XGBoost consistently outperform other machine learning algorithms in the prediction task of the associated sensitivity analysis. In addition, we note that active learning can lead to significant speed-ups in sensitivity analysis by enabling the selection of more useful parameter combinations (i.e., instances) to be used for prediction models.

EVALUATION OF FEATURE SELECTION METHODS FOR VEGETATION MAPPING USING MULTITEMPORAL SENTINEL IMAGERY

Abstract. With the recent advances in remote sensing technologies for Earth observation (EO), many different remote sensors (e.g., optical, radar) collect data with distinctive properties. EO data have been employed to monitor croplands and forested areas, oceans and seas, urban settlements, and natural hazards. The spectral, spatial, and temporal resolutions of remote sensors have been continuously improving, making geospatial monitoring more accurate and comprehensive than ever before. To tackle this issue, various variable selection methods (e.g., filter, wrapper, and embedded methods) are nowadays used to reduce data complexity, and hence improve classification accuracy. Therefore, the goal of this research was twofold. Firstly, to assess the performance of the random forest (RF) classifier in a large heterogeneous landscape with diverse land-cover categories using multi-seasonal Sentinel imagery (i.e., Sentinel-1; S1 and Sentinel-2; S2) and ancillary data. Secondly, to compare RF variable selection methods to identify a subset of predictor variables that will be included in a final, simpler model. Using mean decrease accuracy (MDA) as a feature selection (FS) method, an original dataset was reduced from 114 to 34 input features, and its classification performance outperformed all-feature (114 features) and band-only (36 features) model with an OA of 90.91%. The most pertinent input features for vegetation mapping were S2 spectral bands (14 features), followed by the spectral indices derived from S2, texture features, and S1 bands. This research improved vegetation mapping by integrating radar and optical imagery, especially after applying FS methods which removed redundant and noisy features from the original dataset. Future research should address additional feature selection methods (i.e., filter, wrapper, or the embedded) for vegetation mapping, combined with advanced deep learning methods.

Evaluation of chemometric classification and regression models for the detection of syrup adulteration in honey

Honey is a highly desirable commodity hence a target of adulteration to increase its bulk. Spectroscopic methods and chemometrics offer a fast, easy, and simple approach to the detection of syrup adulterations in honey. The idiosyncrasies associated with different instrumental data suggest varying performance for each algorithm. The objective of this study was to evaluate the performance of five commonly used classification and regression algorithms for the detection of syrup adulteration in honey. The prediction performance of the classification and regression algorithms was evaluated using data obtained from Fourier-transform infrared spectrometer with a Horizontal Attenuated Total Reflectance (HATR) Accessory analysis of honey samples. Gradient boosted discriminant analysis (GBDA) and Support vector machines discriminant analysis (SVMDA) were able to differentiate between adulterated and pure honey samples with an external validation set prediction accuracies of 0.988 and 0.981, respectively. Application of feature selection method did not lead to improved prediction accuracies. Gradient boosted regression (GBR) initialized with a ridge regression predicted the percentage level of adulteration with a regression coefficient of 1.000. The RMSE for the optimal GBR was 2.183 and 0.018 for before and after feature selection with the partial least squares (PLS) algorithm, respectively. Ensemble methods are generally better for both classification and regression of honey using FTIR-HATR spectroscopic data.

An ensemble framework-stacking and feature selection technique for detection of breast cancer

Breast cancer is the second most common cancer in women worldwide. The machine learning (ML) method is a modern and accurate technique that researchers have recently applied to predict and diagnose breast cancer. In this research article, we developed stack-based ensemble techniques and feature selection methods for the comprehensive performance of the algorithm and comparative analysis of breast cancer datasets with reduced attributes and all attributes. This article uses five-feature selection technique because it affects the overall performance of the model. After applying feature selection method, now we have a dataset with reduced features as well as all features. We implemented logistic regression on a dataset with all features and a dataset with reduced features. Finally, we see that the dataset with reduced features have got improved accuracy.

Student Profile Modeling Using Boosting Algorithms

The student profile has become an important component of education systems. Many systems objectives, as e-recommendation, e-orientation, e-recruitment and dropout prediction are essentially based on the profile for decision support. Machine learning plays an important role in this context and several studies have been carried out either for classification, prediction or clustering purpose. In this paper, the authors present a comparative study between different boosting algorithms which have been used successfully in many fields and for many purposes. In addition, the authors applied feature selection methods Fisher Score, Information Gain combined with Recursive Feature Elimination to enhance the preprocessing task and models’ performances. Using multi-label dataset predict the class of the student performance in mathematics, this article results show that the Light Gradient Boosting Machine (LightGBM) algorithm achieved the best performance when using Information gain with Recursive Feature Elimination method compared to the other boosting algorithms.

FUSION OF SELECTED MULTI-MODAL FEATURES FOR ACCURATE FACE RECOGNITION

Yüz tanıma çok sayıda uygulama alanı olan popüler bir bilgisayarla görü problemidir. Farklı ışık koşulları ve değişen yüz ifadeleri yüz tanıma problemini zorlaştıran etkenlerdir. Yüz tanıma işlemi için çeşitli yöntemlerle elde edilen öznitelikler yüze ait farklı karakteristik özellikleri yansıtır. Bu karakteristik özelliklerden faydalanılarak yüz tanıma işlemi gerçekleştirilir. Bu çalışmada, örüntü ve doku tanımada sıklıkla kullanılan Yerel İkili Örüntü ve Felzenszwalb Yönelimli Gradyan Histogram özniteliklerinin birleştirilmesi ile yüz tanıma problemine çok kipli bir çözüm sunulmuştur. Yüz imgesi bölgelere ayrılarak, her iki yöntem ile bölgelerden öznitelik vektörleri elde edilmiştir. Bununla birlikte elde edilen vektörlere öznitelik seçim yöntemleri uygulanarak hem vektör boyutu azaltılmış hem de başarım arttırılmıştır. Öznitelik seçimi sonucu her iki yöntem için seçilen öznitelik alt kümeleri birleştirilerek uzamsal ve spektral öznitelikleri içeren tek bir öznitelik vektörü haline getirilmiştir. Seçilen öznitelikler Ki-kare sınıflandırıcısı kullanılarak sınıflandırılmıştır. Önerilen yöntemin başarımı FERET veri setinde ölçülmüş, %89.45 yüz doğrulama ve %94.55 yüz tanıma başarımı elde edilmiştir.

Short term soil moisture forecasts for potato crop farming: A machine learning approach

Agricultural decision-making is crucial for future yields. In the context of smart farming, grower combine information from sensors located close to their crops with agronomic models to help them to better understand their crops. Irrigation management is therefore based on extrapolation of data and/or agronomic model responses. This problem can be seen as a learning task for which machine learning techniques have proven their relevance in many and diverse applications. In this paper we place ourselves in the context of potato farming, a crop for which irrigation plays a crucial role. We model the problem of soil water potential prediction as a learning problem solved by supervised leaning algorithms. The problem appears to be difficult since there are several potential inputs, and several outputs to predict. Experiments are conducted on several scenarios with data acquired during 3 years. We demonstrate the possibility of applying feature selection method to automatically design models with features relevant to the problem at hand while having good performances. We have also demonstrated the relevance of the machine learning for this kind of problem, since the methods are able to correctly predict the next water potential values.

Vehicle logo recognition using histograms of oriented gradient descriptor and sparsity score

Most of vehicle have the similar structures and designs. It is extremely complicated and difficult to identify and classify vehicle brands based on their structure and shape. As we requirea quick and reliable response, so vehicle logos are an alternative method of determining the type of a vehicle. In this paper, we propose a method for vehicle logo recognition based on feature selection method in a hybrid way. Vehicle logo images are first characterized by histograms of oriented gradient descriptors and the final features vector are then applied feature selection method to reduce the irrelevant information. Moreover, we release a new benchmark dataset for vehicle logo recognition and retrieval task namely, VLR-40. The experimental results are evaluated on this database which show the efficiency of the proposed approach.