Recursive Feature Elimination Technique Research Articles

Hyperparameter Tuning Decision Tree and Recursive Feature Elimination Technique for Improved Chronic Kidney Disease Classification

Purpose: This study has the purpose of classifying patients with chronic kidney disease based on specific features and improving the classification models by tuning hyperparameters. This study aims to detect chronic kidney disease at an early stage. Methods: In this study, a machine learning classifier in the form of a decision tree is used to classify chronic kidney disease on the Risk Factor Prediction of Chronic Kidney Disease dataset. After that, the performance of the classifier model is improved by using feature selection, namely Recursive Feature Elimination and Hyperparameter tuning with GridSearchCV. Result: After tests were conducted 3 times namely testing with Decision Tree, Recursive Feature Elimination, and Hyperparameter tuning GridSearchCV which is the proposed method, then compared to other tests. The results from this study is using that method can improve the Decision Tree classifier in classifying chronic kidney disease patients. Novelty: Dataset that have been used in this study is from UCI machine learning repository namely Risk Factor Prediction of Chronic Kidney Disease that have 202 instances and 28 feature and after being processess and conducting test, Recursive Feature Elimination and Hyperparameter tuning GridSearchCV can improve the Decision Tree classifier in classifying chronic kidney disease.

Deep learning insights into spatial patterns of stable isotopes in Iran’s precipitation: a novel approach to climatological mapping

ABSTRACT Stable isotope techniques are precise methods for studying various aspects of hydrology, such as precipitation characteristics. However, understanding the variations in the stable isotope content in precipitation is challenging in Iran due to numerous climatic and geographic factors. To address this, forty-two precipitation sampling stations were selected across Iran to assess the fractional importance of these climatic and geographic parameters influencing stable isotopes. Additionally, deep learning models were employed to simulate the stable isotope content, with missing data initially addressed using the predictive mean matching (PMM) method. Subsequently, the recursive feature elimination (RFE) technique was applied to identify influential parameters impacting Iran’s precipitation stable isotope content. Following this, long short-term memory (LSTM) and deep neural network (DNN) models were utilized to predict stable isotope values in precipitation. Interpolated maps of these values across Iran were developed using inverse distance weighting (IDW), while an interpolated reconstruction error (RE) map was generated to quantify deviations between observed and predicted values at study stations, offering insights into model precision. Validation using evaluation metrics demonstrated that the model based on DNN exhibited higher accuracy. Furthermore, RE maps confirmed acceptable accuracy in simulating the stable isotope content, albeit with minor weaknesses observed in simulation maps. The methodology outlined in this study holds promise for application in regions worldwide characterized by diverse climatic conditions.

Bi-GRU-APSO: Bi-Directional Gated Recurrent Unit with Adaptive Particle Swarm Optimization Algorithm for Sales Forecasting in Multi-Channel Retail

In the present scenario, retail sales forecasting has a great significance in E-commerce companies. The precise retail sales forecasting enhances the business decision making, storage management, and product sales. Inaccurate retail sales forecasting can decrease customer satisfaction, inventory shortages, product backlog, and unsatisfied customer demands. In order to obtain a better retail sales forecasting, deep learning models are preferred. In this manuscript, an effective Bi-GRU is proposed for accurate sales forecasting related to E-commerce companies. Initially, retail sales data are acquired from two benchmark online datasets: Rossmann dataset and Walmart dataset. From the acquired datasets, the unreliable samples are eliminated by interpolating missing data, outlier’s removal, normalization, and de-normalization. Then, feature engineering is carried out by implementing the Adaptive Particle Swarm Optimization (APSO) algorithm, Recursive Feature Elimination (RFE) technique, and Minimum Redundancy Maximum Relevance (MRMR) technique. Followed by that, the optimized active features from feature engineering are given to the Bi-Directional Gated Recurrent Unit (Bi-GRU) model for precise retail sales forecasting. From the result analysis, it is seen that the proposed Bi-GRU model achieves higher results in terms of an R2 value of 0.98 and 0.99, a Mean Absolute Error (MAE) of 0.05 and 0.07, and a Mean Square Error (MSE) of 0.04 and 0.03 on the Rossmann and Walmart datasets. The proposed method supports the retail sales forecasting by achieving superior results over the conventional models.

Enhancing Seismic Landslide Susceptibility Analysis for Sustainable Disaster Risk Management through Machine Learning

The accuracy of Seismic Landslide Susceptibility Maps (SLSMs) is imperative for the prevention of seismic landslide disasters. This study enhances the precision of SLSMs by integrating nine distinct machine learning methodologies with the GeoDetector version 0.0.4 tool to filter both numerical and physical factors contributing to landslide susceptibility. The dataset comprises 2317 landslide instances triggered by the 2013 Minxian Ms = 6.6 earthquake, from which redundant factors were pruned using the Recursive Feature Elimination technique. Subsequent evaluations of the optimized factors, both individually and in combination, were conducted through Frequency Ratio analysis and Factor Interaction assessment. The study juxtaposes the Area Under the Receiver Operating Characteristic Curve (AUC) and the accuracy of nine machine learning models before and after factor optimization. The findings indicate an increase in AUC from a maximum of 0.989 to 0.992 in the Random Forest model, and an 8.37% increase in AUC for the SVM model, signifying a notable enhancement in the stability across all models. The establishment of the SLSM notably elevated the frequency ratio in high-risk zones from 50.40 to 85.14, underscoring the efficacy of combining machine learning and detector optimization techniques in sustainable practices. This research proposes a universal framework aimed at eliminating redundancy and noise in SLSMs and hazard risk assessments, thereby facilitating sustainable geological disaster risk management.

A novel hybrid deep CNN model for breast cancer classification using Lipschitz-based image augmentation and recursive feature elimination

Breast cancer is the leading cause of death for women worldwide. Despite its difficulties, the histopathological diagnosis must be used because modern imaging techniques cannot differentiate between benign and malignant tumors. This article introduces a hybrid deep learning method with recursive feature elimination for categorizing histopathology images of breast lesions as benign or malignant, using standard datasets, to increase diagnostic accuracy. The proposed four-stage model includes several novel techniques. First, a modified Lipschitz-based image augmentation technique is utilized to ensure a proportionate representation of various sample classes. K-fold cross-validation is employed to mitigate the bias that emerges from the selection of training and testing datasets. Then, for efficient feature extraction, a hybrid convolutional neural network (CNN) approach is used by integrating three CNNs (PResNet-34, FE-VGG-16, and M-AlexNet). With the proposed universum twin support vector machine-based recursive feature elimination technique, the initial 1,000 features extracted by each deep model are further reduced to 100. Finally, the deep features are subsequently combined into a set of 300 features, which are then fed into machine learning classifiers such as decision tree, k-nearest neighborhood, linear discriminant analysis, linear regression, and support vector machine. The results demonstrate that the proposed hybrid CNN model outperforms state-of-the-art techniques in accurately classifying breast lesions as benign or malignant, with an accuracy of 99.99 %, sensitivity of 99.56 %, specificity of 99.53 %, precision of 99.55 %, and F1 score of 99.77 %.

Predicting the Conversion from Mild Cognitive Impairment to Alzheimer’s Disease Using an Explainable AI Approach

Mild Cognitive Impairment (MCI) is a cognitive state frequently observed in older adults, characterized by significant alterations in memory, thinking, and reasoning abilities that extend beyond typical cognitive decline. It is worth noting that around 10–15% of individuals with MCI are projected to develop Alzheimer’s disease, effectively positioning MCI as an early stage of Alzheimer’s. In this study, a novel approach is presented involving the utilization of eXtreme Gradient Boosting to predict the onset of Alzheimer’s disease during the MCI stage. The methodology entails utilizing data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Through the analysis of longitudinal data, spanning from the baseline visit to the 12-month follow-up, a predictive model was constructed. The proposed model calculates, over a 36-month period, the likelihood of progression from MCI to Alzheimer’s disease, achieving an accuracy rate of 85%. To further enhance the precision of the model, this study implements feature selection using the Recursive Feature Elimination technique. Additionally, the Shapley method is employed to provide insights into the model’s decision-making process, thereby augmenting the transparency and interpretability of the predictions.

Subnetwork prediction approach for aircraft schedule recovery

The combinatorial optimization techniques for the aircraft schedule recovery problem are too expensive to compute. However, we require a reasonable solution within 1–2 min for real-time application. The computational time of traditional optimization algorithms will drastically reduce if we can accurately predict the subnetwork affected by the disruption. In this study, we advocate the use of machine learning (ML) as a promising tool to predict the subnetwork. The optimal recovery schedules for a large number of disruptions are used for offline training, and subsequently, the trained ML model is employed to reduce the search space for new disruption cases substantially. To achieve this, we employ a unique feature space specifically designed to capture the essential characteristics of the problem. A notable contribution is constructing a novel feature space based on revised aircraft schedules designed to capture the essential characteristics of the problem. The recursive feature elimination technique is employed for optimal feature selection. Five machine learning classifiers: Logistic Regression, Random Forest, Extreme gradient boosting, Light gradient boosting, and Categorical Boosting are compared. The performance is evaluated on real data obtained from an airline company. Our study demonstrates that, with the subnetwork of aircraft predicted by the classifier, the computational time of the column generation approach is remarkably reduced without deteriorating the solution quality in all the disruption instances tested, with more than 98% of the instances being solved within 60 s. The results highlight the remarkable advantage of integrating ML in the pre-optimization phase.

A machine learning diagnosis of the severe accident progression

A Machine Learning (ML) platform is proposed to assist the operator in diagnosing the severe accident progression, where some of the signal is corrupted, and/or estimation of key parameters for accident management is needed. To predict the key parameters for accident management such as lost signals for the pressure and core water level, the amount of hydrogen generation, and the mass of nuclear aerosols, a Machine Learning (ML) platform with long short term memory (LSTM) network is proposed, where multiple accident scenarios can be accumulated and used for training. Training and test data were produced by MELCOR simulations of the Fukushima Daiichi Nuclear Power Plant (FDNPP) accident at unit 3. The MELCOR simulation data is presented as time series at equal intervals. Feature variables were selected among many plant parameters, where the importance ranking was determined by a recursive feature elimination technique using RandomForestRegressor. We performed a sensitivity study of the different choices of training and test scenarios, the number of feature variables, the number of neurons, and target variables on the prediction accuracy of the ML platform. Among eight MELCOR simulation results, different combinations of training and test data are selected in terms of similarity between them. It is found that when the number of feature variables is more than five, the proposed ML platform was able to predict not only similar test data but also unseen test data with reasonable accuracy. It is also found that the proposed ML platform consistently predicts any of the lost signals and key parameters for accident management with reasonable accuracy. Pre-trained ML model predicted the target variable within several minutes on a desktop computer.

A Hybrid Dimensionality Reduction for Network Intrusion Detection

Due to the wide variety of network services, many different types of protocols exist, producing various packet features. Some features contain irrelevant and redundant information. The presence of such features increases computational complexity and decreases accuracy. Therefore, this research is designed to reduce the data dimensionality and improve the classification accuracy in the UNSW-NB15 dataset. It proposes a hybrid dimensionality reduction system that does feature selection (FS) and feature extraction (FE). FS was performed using the Recursive Feature Elimination (RFE) technique, while FE was accomplished by transforming the features into principal components. This combined scheme reduced a total of 41 input features into 15 components. The proposed systems’ classification performance was determined using an ensemble of Support Vector Classifier (SVC), K-nearest Neighbor classifier (KNC), and Deep Neural Network classifier (DNN). The system was evaluated using accuracy, detection rate, false positive rate, f1-score, and area under the curve metrics. Comparing the voting ensemble results of the full feature set against the 15 principal components confirms that reduced and transformed features did not significantly decrease the classifier’s performance. We achieved 94.34% accuracy, a 93.92% detection rate, a 5.23% false positive rate, a 94.32% f1-score, and a 94.34% area under the curve when 15 components were input to the voting ensemble classifier.

SOC content of global Mollisols at a 30 m spatial resolution from 1984 to 2021 generated by the novel ML-CNN prediction model

Carbon cycle is influenced by agricultural soils, and accurately mapping the soil organic carbon (SOC) content of global Mollisols at a 30 m spatial resolution can contribute to clarifying the carbon sequestration capacity of each region, facilitate the quantification of agroecosystems and contribute to global food security. However, the high heterogeneity of environmental variables in global regions, coupled with the challenges posed by small-sample tasks, creates significant obstacles to producing reliable SOC content datasets. In this study, we collected 191,465 scenes of Landsat TM and OLI images and elevation model data to calculate spectral indices that can represent soil formation information based on a soil-pedogenic model. Second, a local strategy (LS) was proposed to reduce the influence of the high heterogeneity of SOC content and environmental variables on the prediction results. More importantly, the first meta-learning convolutional neural network (ML-CNN) model was proposed. It provides high prediction accuracy for small-sample tasks and was used to generate the first high-resolution global Mollisol region SOC content product (GMR-MCNN). Finally, we compared GMR-MCNN with the existing SoilGrids250m and Soil SubCenter products. The results showed that long-term, high-accuracy and high-resolution prediction of the SOC content in global Mollisol regions was achieved by the ML-CNN model (RMSE = 4.84 g kg−1, R2 = 0.75, RPIQ = 2.43). Compared with a CNN, ML-CNN can continuously optimize the meta-task, thus improving the performance of the model in small-sample tasks. Compared to the prediction model that combined the recursive feature elimination technique with the random forest model (RFE-RF), ML-CNN can efficiently extract high-level features from time-series data, thus improving the model performance. Compared with that of the global strategy, the RMSE of the LS decreased by 0.20 g kg−1, and R2 and RPIQ increased by 13.00% and 0.22, respectively. In addition, the GMR-MCNN results illustrated that the SOC content in the global Mollisol regions shows a decreasing trend, and the trend can be divided into significant decrease (1984–2000) and moderate decrease (2001−2021) phases. Different products were tested based on laboratory-measured SOC contents, and GMR-MCNN (RMSE = 6.13 g kg−1, R2 = 0.63) displayed better performance than SoilGrids250m (RMSE = 23.37 g kg−1, R2 = 0.28) and the Soil SubCenter map (RMSE = 8.59 g kg−1, R2 = 0.43). The developed methodology can provide a reference for the long-term observation of soil and crop properties at moderate and high resolutions globally.

Enhancing Breast Cancer Detection and Classification Using Advanced Multi-Model Features and Ensemble Machine Learning Techniques.

Breast cancer (BC) is the most common cancer among women, making it essential to have an accurate and dependable system for diagnosing benign or malignant tumors. It is essential to detect this cancer early in order to inform subsequent treatments. Currently, fine needle aspiration (FNA) cytology and machine learning (ML) models can be used to detect and diagnose this cancer more accurately. Consequently, an effective and dependable approach needs to be developed to enhance the clinical capacity to diagnose this illness. This study aims to detect and divide BC into two categories using the Wisconsin Diagnostic Breast Cancer (WDBC) benchmark feature set and to select the fewest features to attain the highest accuracy. To this end, this study explores automated BC prediction using multi-model features and ensemble machine learning (EML) techniques. To achieve this, we propose an advanced ensemble technique, which incorporates voting, bagging, stacking, and boosting as combination techniques for the classifier in the proposed EML methods to distinguish benign breast tumors from malignant cancers. In the feature extraction process, we suggest a recursive feature elimination technique to find the most important features of the WDBC that are pertinent to BC detection and classification. Furthermore, we conducted cross-validation experiments, and the comparative results demonstrated that our method can effectively enhance classification performance and attain the highest value in six evaluation metrics, including precision, sensitivity, area under the curve (AUC), specificity, accuracy, and F1-score. Overall, the stacking model achieved the best average accuracy, at 99.89%, and its sensitivity, specificity, F1-score, precision, and AUC/ROC were 1.00%, 0.999%, 1.00%, 1.00%, and 1.00%, respectively, thus generating excellent results. The findings of this study can be used to establish a reliable clinical detection system, enabling experts to make more precise and operative decisions in the future. Additionally, the proposed technology might be used to detect a variety of cancers.

Transformer-based deep learning models for adsorption capacity prediction of heavy metal ions toward biochar-based adsorbents

Biochar adsorbents synthesized from food and agricultural wastes are commonly applied to eliminate heavy metal (HM) ions from wastewater. However, biochar’s diverse characteristics and varied experimental conditions make the accurate estimation of their adsorption capacity (qe) challenging. Herein, various machine-learning (ML) and three deep learning (DL) models were built using 1518 data points to predict the qe of HM on various biochars. The recursive feature elimination technique with 28 inputs suggested that 14 inputs were significant for model building. FT-transformer with the highest test R2 (0.98) and lowest root mean square error (RMSE) (0.296) and mean absolute error (MAE) (0.145) outperformed various ML and DL models. The SHAP feature importance analysis of the FT-transformer model predicted that the adsorption conditions (72.12%) were more important than the pyrolysis conditions (25.73%), elemental composition (1.39%), and biochar’s physical properties (0.73%). The two-feature SHAP analysis proposed the optimized process conditions including adsorbent loading of 0.25 g, initial concentration of 12 mg/L, and solution pH of 9 using phosphoric-acid pre-treated biochar synthesized from banana-peel with a higher O/C ratio. The t-SNE technique was applied to transform the 14-input matrix of the FT-Transformer into two-dimensional data. Finally, we outlined the study’s environmental implications.

Decision support framework for predicting rate of gait recovery with optimized treatment planning

Rate of Recovery during the rehabilitation procedure is not effectively evaluated due to existing limitations of measurement tools and large human variability.The conventional way to treat lower-limb injuries is by conducting a physician-guided rehabilitation process that could take several weeks or months. But the process of estimating the rate of recovery is entirely based on the expertise of a therapist. Moreover, it remains unclear to a large extent about the factors have contributed to the success of treatment, the most effective rehabilitation duration, and recovery progression rate. To address these limitations, a random subspace ensemble k-nearest neighbor (RSEnkNN) model is proposed to predict the rehabilitation duration of an individual after initial baseline assessment of gait trajectories. The model is trained using an open-source dataset comprising 211 healthy and 2084 patients with functional gait disorders.The proposed model is able to predict rehabilitation duration with an accuracy of 88–89 % in case of ankle, calcaneus, and hip injury while the success rate of 82 % is achieved for knee disorders. Features computed from ground reaction force, measures of postural steadiness, and bilateral symmetry turn out to be effective in predicting the early, moderate, or late recovery of an individual. The significantly contributing features for each anomaly class are reported using recursive feature elimination (RFE) technique to support clinicians. Thereafter, recovery rate is determined using a novel metric Rate of Gait Recovery (RoGR) index that could be effective for treatment planning and re-scheduling of therapy sessions (if required). This work determined that the most effective rehabilitation period is 2–6 weeks post-surgery for restoration to normal walk. Thus, a prediction tool for strengthening clinical decision-making is proposed to support early rehabilitation.

BaPreS: a software tool for predicting bacteriocins using an optimal set of features

BackgroundAntibiotic resistance is a major public health concern around the globe. As a result, researchers always look for new compounds to develop new antibiotic drugs for combating antibiotic-resistant bacteria. Bacteriocin becomes a promising antimicrobial agent to fight against antibiotic resistance, due to cases of both broad and narrow killing spectra. Sequence matching methods are widely used to identify bacteriocins by comparing them with the known bacteriocin sequences; however, these methods often fail to detect new bacteriocin sequences due to their high diversity. The ability to use a machine learning approach can help find new highly dissimilar bacteriocins for developing highly effective antibiotic drugs. The aim of this work is to develop a machine learning-based software tool called BaPreS (Bacteriocin Prediction Software) using an optimal set of features for detecting bacteriocin protein sequences with high accuracy. We extracted potential features from known bacteriocin and non-bacteriocin sequences by considering the physicochemical and structural properties of the protein sequences. Then we reduced the feature set using statistical justifications and recursive feature elimination technique. Finally, we built support vector machine (SVM) and random forest (RF) models using the selected features and utilized the best machine learning model to implement the software tool.ResultsWe applied BaPreS to an established dataset and evaluated its prediction performance. Acquired results show that the software tool can achieve a prediction accuracy of 95.54% for testing protein sequences. This tool allows users to add new bacteriocin or non-bacteriocin sequences in the training dataset to further enhance the predictive power of the tool. We compared the prediction performance of the BaPreS with a popular sequence matching-based tool and a deep learning-based method, and our software tool outperformed both.ConclusionsBaPreS is a bacteriocin prediction tool that can be used to discover new highly dissimilar bacteriocins for developing highly effective antibiotic drugs. This software tool can be used with Windows, Linux and macOS operating systems. The open-source software package and its user manual are available at https://github.com/suraiya14/BaPreS.

Testing machine learning algorithms for the prediction of depositional fluxes of the radionuclides 7Be, 210Pb and 40K

The monthly depositional fluxes of 7Be, 210Pb and 40K were measured at Malaga, (Southern Spain) from 2005 to 2018. In this work, the depositional fluxes of these radionuclides are investigated and their relations with several atmospheric variables have been studied by applying two popular machine learning methods: Random Forest and Neural Network algorithms. We extensively test different configurations of these algorithms and demonstrate their predictive ability for reproducing depositional fluxes. The models derived with Neural Networks achieve slightly better results, in average, although similar, having into account the uncertainties. The mean Pearson-R coefficients, evaluated with a k-fold cross-validation method, are around 0.85 for the three radionuclides using Neural Network models, while they go down to 0.83, 0.79 and 0.8 for 7Be, 210Pb and 40K, respectively, for the Random Forest models. Additionally, applying the Recursive Feature Elimination technique we determine the variables more correlated with the depositional fluxes of these radionuclides, which elucidates the main dependences of their temporal variability.

Development and validation of radiomic signature for predicting overall survival in advanced-stage cervical cancer.

The role of artificial intelligence and radiomics in prediction model development in cancer has been increasing every passing day. Cervical cancer is the 4th most common cancer in women worldwide, contributing to 6.5% of all cancer types. The treatment outcome of cervical cancer patients varies and individualized prediction of disease outcome is of paramount importance. The purpose of this study is to develop and validate the digital signature for 5-year overall survival prediction in cervical cancer using robust CT radiomic and clinical features. Pretreatment clinical features and CT radiomic features of 68 patients, who were treated with chemoradiation therapy in our hospital, were used in this study. Radiomic features were extracted using an in-house developed python script and pyradiomic package. Clinical features were selected by the recursive feature elimination technique. Whereas radiomic feature selection was performed using a multi-step process i.e., step-1: only robust radiomic features were selected based on our previous study, step-2: a hierarchical clustering was performed to eliminate feature redundancy, and step-3: recursive feature elimination was performed to select the best features for prediction model development. Four machine algorithms i.e., Logistic regression (LR), Random Forest (RF), Support vector classifier (SVC), and Gradient boosting classifier (GBC), were used to develop 24 models (six models using each algorithm) using clinical, radiomic and combined features. Models were compared based on the prediction score in the internal validation. The average prediction accuracy was found to be 0.65 (95% CI: 0.60-0.70), 0.72 (95% CI: 0.63-0.81), and 0.77 (95% CI: 0.72-0.82) for clinical, radiomic, and combined models developed using four prediction algorithms respectively. The average prediction accuracy was found to be 0.69 (95% CI: 0.62-0.76), 0.79 (95% CI: 0.72-0.86), 0.71 (95% CI: 0.62-0.80), and 0.72 (95% CI: 0.66-0.78) for LR, RF, SVC and GBC models developed on three datasets respectively. Our study shows the promising predictive performance of a robust radiomic signature to predict 5-year overall survival in cervical cancer patients.

Intrusion detection using decision tree classifier with feature reduction technique

The number of internet users and network services is increasing rapidly in the recent decade gradually. A Large volume of data is produced and transmitted over the network. Number of security threats to the network has also been increased. Although there are many machine learning approaches and methods are used in intrusion detection systems to detect the attacks, but generally they are not efficient for large datasets and real time detection. Machine learning classifiers using all features of datasets minimized the accuracy of detection for classifier. A reduced feature selection technique that selects the most relevant features to detect the attack with ML approach has been used to obtain higher accuracy. In this paper, we used recursive feature elimination technique and selected more relevant features with machine learning approaches for big data to meet the challenge of detecting the attack. We applied this technique and classifier to NSL KDD dataset. Results showed that selecting all features for detection can maximize the complexity in the context of large data and performance of classifier can be increased by feature selection best in terms of efficiency and accuracy.

Impact of Data Balancing and Feature Selection on Machine Learning-based Network Intrusion Detection

Unbalanced datasets are a common problem in supervised machine learning. It leads to a deeper understanding of the majority of classes in machine learning. Therefore, the machine learning model is more effective at recognizing the majority classes than the minority classes. Naturally, imbalanced data, such as disease data and data networking, has emerged in real life. DDOS is one of the network intrusions found to happen more often than R2L. There is an imbalance in the composition of network attacks in Intrusion Detection System (IDS) public datasets such as NSL-KDD and UNSW-NB15. Besides, researchers propose many techniques to transform it into balanced data by duplicating the minority class and producing synthetic data. Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic (ADASYN) algorithms duplicate the data and construct synthetic data for the minority classes. Meanwhile, machine learning algorithms can capture the labeled data's pattern by considering the input features. Unfortunately, not all the input features have an equal impact on the output (predicted class or value). Some features are interrelated and misleading. Therefore, the important features should be selected to produce a good model. In this research, we implement the recursive feature elimination (RFE) technique to select important features from the available dataset. According to the experiment, SMOTE provides a better synthetic dataset than ADASYN for the UNSW-B15 dataset with a high level of imbalance. RFE feature selection slightly reduces the model's accuracy but improves the training speed. Then, the Decision Tree classifier consistently achieves a better recognition rate than Random Forest and KNN.

A Preliminary Predictive Model for Proliferative Lupus Nephritis in Juvenile Systemic Lupus Erythematosus

Proliferative lupus nephritis, which is diagnosed by renal biopsy, has significant impact on the treatment choices and long-term prognosis of juvenile SLE (jSLE). Renal biopsies are however not always possible or available, thus leading to an ongoing search for alternative biomarkers. This study aimed to develop a clinical predictive machine learning model using routine standard parameters as an alternative tool to evaluate the probability of proliferative lupus nephritis (ISN/RPS Class III or IV). Data were collected retrospectively from jSLE patients seen at Selayang Hospital from 2004 to 2021. A total of 22 variables including demographic, clinical and laboratory features were analyzed. A recursive feature elimination technique was used to identify factors to predict pediatric proliferative lupus nephritis. Various models were then used to build predictive machine learning models and assessed for sensitivity, specificity and accuracy. There were 194 jSLE patients (165 females), of which 111 had lupus nephritis (54 proliferative pattern). A combination of 11 variables consisting of gender, ethnicity, fever, nephrotic state, hypertension, urine red blood cells (RBC), C3, C4, duration of illness, serum albumin, and proteinuria demonstrated the highest accuracy of 79.4% in predicting proliferative lupus nephritis. A decision-tree model performed the best with an AROC of 69.9%, accuracy of 73.85%, sensitivity of 78.72% and specificity of 61.11%. A potential clinically useful predictive model using a combination of 11 non-invasive variables to collectively predict pediatric proliferative lupus nephritis in daily practice was developed.

Towards a versatile mental workload modeling using neurometric indices.

Researchers have been working to magnify mental workload (MWL) modeling for a long time. An important aspect of its modeling is feature selection as itinterprets bulky and high-dimensional EEG data and enhances the accuracy of the classification model. In this study, a feature selection technique is proposed to obtain an optimized feature set with multiple domain features that can contribute to classifying the MWL at three distinct levels. The brain signals from thirteen healthy subjects were examined while they attended an intrinsic MWL of spotting differences in a set of similar pictures. The Recursive Feature Elimination (RFE) technique selects therobust features from the feature matrix by eliminating all the least contributing features. Along with the Support Vector Machine (SVM), the overall classification accuracy with the proposed RFE reached 0.913 from 0.791 surpassing the other techniques mentioned. The results of the study also significantly display the variation in the mean values of the selected features at the three workload levels (p<0.05). This model can become the principle for defining the workload level quantification applicable to diverse fields like neuroergonomics study, intelligent assistive devices (ADs) development, blue-chip technology exploration, cognitive evaluation of students, power plant operators, traffic operators, etc.