XGBoost Classifier Research Articles

Game Theory-Based Dynamic Weighted Ensemble for Retinal Disease Classification

An automated retinal disease detection system has long been in existence and it provides a safe, no-contact and cost-effective solution for detecting this disease. This paper presents a game theory-based dynamic weighted ensemble of a feature extraction-based machine learning model and a deep transfer learning model for automatic retinal disease detection. The feature extraction-based machine learning model uses Gaussian kernel-based fuzzy rough sets for reduction of features, and XGBoost classifier for the classification. The transfer learning model uses VGG16 or ResNet50 or Inception-ResNet-v2. A novel ensemble classifier based on the game theory approach is proposed for the fusion of the outputs of the transfer learning model and the XGBoost classifier model. The ensemble approach significantly improves the accuracy of retinal disease prediction and results in an excellent performance when compared to the individual deep learning and feature-based models.

Machine Learning Techniques for Breast Cancer Prediction

Breast cancer is main reason for mortality in woman. Prediction of breast cancer is a challenging task in medical data analysis. Doctors and pathologist required some automated tools to take decision and to differentiate between malignant and benign tumour. A machine learning (ML) algorithm helps lot to take decisions and to perform diagnosis from the data collected by medical field. Various researches show that ML techniques are helpful for decision making in breast cancer prediction. In this paper, we used various ML Classification techniques: Naïve Bayes(NB), Logistic regression (LR),Support vector machine(SVM),K-Nearest Neighbor (KNN), Decision Tree(DT), and ensemble techniques: Random forest(RF), Adaboost, XGBoost on breast cancer dataset and evaluated by using different performance measure. It has been found that both decision tree and XGBoost classifier has highest accuracy 97% among all model and highest AUC 0.999 obtained for XGBoost classifier.

User Authentication by Eye Movement Features Employing SVM and XGBoost Classifiers

User Authentication by Eye Movement Features Employing SVM and XGBoost Classifiers

Assessment of mental workload using XGBoost classifier from optimised EEG features

Assessment of mental workload using XGBoost classifier from optimised EEG features

Automatic Anomaly Monitoring in Public Surveillance Areas

With the dramatic increase in video surveillance applications and public safety measures, the need for an accurate and effective system for abnormal/suspicious activity classification also increases. Although it has multiple applications, the problem is very challenging. In this paper, a novel approach for detecting normal/abnormal activity has been proposed. We used the Gaussian Mixture Model (GMM) and Kalman filter to detect and track the objects, respectively. After that, we performed shadow removal to segment an object and its shadow. After object segmentation we performed occlusion detection method to detect occlusion between multiple human silhouettes and we implemented a novel method for region shrinking to isolate occluded humans. Fuzzy c-mean is utilized to verify human silhouettes and motion based features including velocity and optical flow are extracted for each identified silhouettes. Gray Wolf Optimizer (GWO) is used to optimize feature set followed by abnormal event classification that is performed using the XG-Boost classifier. This system is applicable in any surveillance application used for event detection or anomaly detection. Performance of proposed system is evaluated using University of Minnesota (UMN) dataset and UBI (University of Beira Interior)-Fight dataset, each having different type of anomaly. The mean accuracy for the UMN and UBI-Fight datasets is 90.14% and 76.9% respectively. These results are more accurate as compared to other existing methods.

Integrated extreme gradient boost with c4.5 classifier for high level synthesis in very large scale integration circuits

High-level synthesis (HLS) is utilized for high-performance and energy-efficient heterogeneous systems designing. HLS is assist in field-programmable gate array circuits designing where hardware implementations are refined and replaced in target device. However, the power-process-voltage-temperature-delay (PPVTD) variation in VLSI circuits undergoes many problems and reduced the performance. In order to address these problems, C4.5 with eXtreme Gradient Boosting Classification based High Level Synthesis (C4.5-XGBCHLS) Method is designed for afford better runtime adaptability (RA) with minimal error rate. VLSI circuits are designed using the behavioral input and results are measured at running condition. When VLSI circuit’s results get reduced, the language description of the circuit is considered as an input. Then, compilation process convert high level specification into Intermediate Representation (IR) in control/data flow graph (CDFG). CDFG computes data and control dependencies among operations. eXtreme Gradient Boosting (XGBoost) Classifier is exploited in C4.5-XGBCHLS method to classify the error causing functional unit (FU) with minimal error rate. XGBoost Classifier exploited C4.5 decision tree as base classifier to enhance classification of error causing FU in VLSI circuits. After that, FU gets allocated in place of error causing FU from functional library based on the design objectives and PPVTD variations. Finally, operation scheduling and binding process is executed for register transfer level (RTL) generation to form VLSI circuits with improved RA. The simulation results shows that the C4.5-XGBCHLS method enhances the performance of functional unit selection accuracy (FUSA) with minimal error rate (ER) and circuit adaptability time (CAT).

Detecting the most critical clinical variables of COVID-19 breakthrough infection in vaccinated persons using machine learning.

COVID-19 vaccines offer different levels of immune protection but do not provide 100% protection. Vaccinated persons with pre-existing comorbidities may be at an increased risk of SARS-CoV-2 breakthrough infection or reinfection. The aim of this study is to identify the critical variables associated with a higher probability of SARS-CoV-2 breakthrough infection using machine learning. A dataset comprising symptoms and feedback from 257 persons, of whom 203 were vaccinated and 54 unvaccinated, was used for the investigation. Three machine learning algorithms - Deep Multilayer Perceptron (Deep MLP), XGBoost, and Logistic Regression - were trained with the original (imbalanced) dataset and the balanced dataset created by using the Random Oversampling Technique (ROT), and the Synthetic Minority Oversampling Technique (SMOTE). We compared the performance of the classification algorithms when the features highly correlated with breakthrough infection were used and when all features in the dataset were used. The results show that when highly correlated features were considered as predictors, with Random Oversampling to address data imbalance, the XGBoost classifier has the best performance (F1 = 0.96; accuracy = 0.96; AUC = 0.98; G-Mean = 0.98; MCC = 0.88). The Deep MLP had the second best performance (F1 = 0.94; accuracy = 0.94; AUC = 0.92; G-Mean = 0.70; MCC = 0.42), while Logistic Regression had less accurate performance (F1 = 0.89; accuracy = 0.88; AUC = 0.89; G-Mean = 0.89; MCC = 0.68). We also used Shapley Additive Explanations (SHAP) to investigate the interpretability of the models. We found that body temperature, total cholesterol, glucose level, blood pressure, waist circumference, body weight, body mass index (BMI), haemoglobin level, and physical activity per week are the most critical variables indicating a higher risk of breakthrough infection. These results, evident from our unique data source derived from apparently healthy volunteers with cardiovascular risk factors, follow the expected pattern of positive or negative correlations previously reported in the literature. This information strengthens the body of knowledge currently applied in public health guidelines and may also be used by medical practitioners in the future to reduce the risk of SARS-CoV-2 breakthrough infection.

Comparison of different machine learning algorithms to classify patients suspected of having sepsis infection in the intensive care unit

BackgroundSepsis is a life-threatening disease that occurs as a result of the body's response to an infection. This study aims to develop a classification model for predicting patients at risk of sepsis using clinical findings and demographic information. MethodsThe study was conducted using a MIMICIII dataset which is freely available as open-access data. The synthetic minority oversampling technique (SMOTE) was applied to address the imbalanced data problem in our dataset. Through preprocessing, the dataset was cleaned and missing values were imputed. Split validation was done by dividing the dataset into training and test data for developing classification models. Six algorithms including Gaussian Naïve Bayes (NB), decision tree (DT), random forest (RF), logistic regression (LR), KNN algorithm, and XGBoost classifier were developed. A combination of evaluation metrics was employed to evaluate the performance of the proposed models. ResultsOur dataset includes 1,552,210 entries with 44 features of critically ill patients who were admitted to the ICU. Comparing the performance of developed models using different metrics showed that the RF model had the best performance in terms of F-Measure and the area under the ROC curve. The 20 top features with high importance were determined based on the RF model. ConclusionOur analysis showed that the RF model predicted sepsis with significantly higher performance in comparison to other classification models using the MIMICIII dataset. Due to the high mortality of sepsis, these kinds of studies could be supportive to prevent the side effects of the disease and lessen the risk of mortality in hospitalized patients by providing early sepsis prediction.

Automation of Credit Card Customer Churn Analysis using Hybrid Machine Learning Models

Credit Card Customer Churn Analysis (C4A) is a phenomenon where customers stop using a specific business credit card service. Predicting customer churn is crucial for Credit Card (CC) companies because it enables them to spot at-risk customers and take precautions to retain them. The aim of the paper named C4A is to create a model that accurately predicts customers who are most likely to stop using CC. The paper involves gathering and analyzing customer information from Kaggle, including transaction history, demographics and credit card usage patterns for prediction. Machine learning algorithms namely, Logistic Regression, KNN, XGBoost Classifier, Decision Tree and Hybrid Models integrating Logistic Regression and KNN, Logistic Regression and Decision Tree are used to train to find patterns and correlations that point to customer churn. The accuracy of the proposed method is 0.846 with LR, 0.849 with KNN, 0.90 with a hybrid model integrating LR and KNN, 0.928 by integrating LR and DT, 0.91 with DT, and 0.93 with XGBoost.

Attention monitoring of students during online classes using XGBoost classifier

With the ever-growing influence of globalization and digitization on education, online learning has gained immense popularity, driven in large part by the global COVID-19 pandemic. In response, educational institutions have rapidly expanded their virtual course offerings and assessment methods. One significant challenge in the realm of online education is the effective assessment of student involvement, engagement, and attentiveness during virtual classes. To address this challenge, we introduce a specialized system tailored to identifying and comprehending the complex range of student behaviors exhibited during online classes. Our system takes into account the diverse contexts and individual considerations that shape students' responses to the online learning environment. It employs user authentication through facial recognition technology and seamlessly integrates vital components, including facial detection, hand tracking, mobile phone detection, and pose estimation modules. Leveraging these components, we extract high-level features to construct a comprehensive dataset for training machine-learning models designed to detect students' attention levels during online classes. After evaluating six prominent machine learning algorithms, we have selected the extreme gradient boosting (XGBoost) algorithm, which demonstrated an impressive 99.75% accuracy on the test data. Finally, the system generates a comprehensive anonymous report on the level of attentiveness of students, accessible through a dedicated webpage that includes a summary of the behavior of students throughout online classes, providing valuable insights for personalized interventions and enhanced online learning experiences.

Classification and comparison of human activities by machine learning

The current study of human activity recognition and classification has been an important part of promoting the development of science and technology in society. Human activity recognition and classification are in several fields, such as competitive sports, criminal investigation field, etc. As the field of micro-electromechanics continues to evolve, more accurate human recognition is becoming possible, with wearable multi-axis inertial sensors that allow us to visually detect the desired data. In this paper, the data of 19 human activities for 8 testers are feature extracted and normalized. The data are divided into training and test sets by machine learning models: support vector machine (SVR) classification, XGBoost classification, and logistic regression. The experiment was repeated 10 times to take the average value. The models were then scored, and by comparing integrated machine learning with traditional machine learning, it was found that integrated learning improved by 5%−29% in terms of accuracy compared to traditional machine learning.

HEGANLDA: A Computational Model for Predicting Potential Lncrna-Disease Associations Based On Multiple Heterogeneous Networks.

Long non-coding RNAs (lncRNAs) play vital regulatory roles in many human complex diseases, however, the number of validated lncRNA-disease associations is notable rare so far. How to predict potential lncRNA-disease associations precisely through computational methods remains challenging. In this study, we proposed a novel method, LDVCHN (LncRNA-Disease Vector Calculation Heterogeneous Networks), and also developed the corresponding model, HEGANLDA (Heterogeneous Embedding Generative Adversarial Networks LncRNA-Disease Association), for predicting potential lncRNA-disease associations. In HEGANLDA, the graph embedding algorithm (HeGAN) was introduced for mapping all nodes in the lncRNA-miRNA-disease heterogeneous network into the low-dimensional vectors which severed as the inputs of LDVCHN. HEGANLDA effectively adopted the XGBoost (eXtreme Gradient Boosting) classifier, which was trained by the low-dimensional vectors, to predict potential lncRNA-disease associations. The 10-fold cross-validation method was utilized to evaluate the performance of our model, our model finally achieved an area under the ROC curve of 0.983. According to the experiment results, HEGANLDA outperformed any one of five current state-of-the-art methods. To further evaluate the effectiveness of HEGANLDA in predicting potential lncRNA-disease associations, both case studies and robustness tests were performed and the results confirmed its effectiveness and robustness. The source code and data of HEGANLDA are available at https://github.com/HEGANLDA/HEGANLDA.

Deep Transfer Learning Driven Automated Fall Detection for Quality of燣iving of Disabled Persons

Mobile communication and the Internet of Things (IoT) technologies have recently been established to collect data from human beings and the environment. The data collected can be leveraged to provide intelligent services through different applications. It is an extreme challenge to monitor disabled people from remote locations. It is because day-to-day events like falls heavily result in accidents. For a person with disabilities, a fall event is an important cause of mortality and post-traumatic complications. Therefore, detecting the fall events of disabled persons in smart homes at early stages is essential to provide the necessary support and increase their survival rate. The current study introduces a Whale Optimization Algorithm Deep Transfer Learning-Driven Automated Fall Detection (WOADTL-AFD) technique to improve the Quality of Life for persons with disabilities. The primary aim of the presented WOADTL-AFD technique is to identify and classify the fall events to help disabled individuals. To attain this, the proposed WOADTL-AFD model initially uses a modified SqueezeNet feature extractor which proficiently extracts the feature vectors. In addition, the WOADTL-AFD technique classifies the fall events using an extreme Gradient Boosting (XGBoost) classifier. In the presented WOADTL-AFD technique, the WOA approach is used to fine-tune the hyperparameters involved in the modified SqueezeNet model. The proposed WOADTL-AFD technique was experimentally validated using the benchmark datasets, and the results confirmed the superior performance of the proposed WOADTL-AFD method compared to other recent approaches.

Ensemble Learning for Fetal Health Classification

<b>:</b> Cardiotocography (CTG) represents the fetus’s health inside the womb during labor. However, assessment of its readings can be a highly subjective process depending on the expertise of the obstetrician. Digital signals from fetal monitors acquire parameters (i.e., fetal heart rate, contractions, acceleration). <b>Objective::</b> This paper aims to classify the CTG readings containing imbalanced healthy, suspected, and pathological fetus readings. <b>Method::</b> We perform two sets of experiments. Firstly, we employ five classifiers: Random Forest (RF), Adaptive Boosting (AdaBoost), Categorical Boosting (CatBoost), Extreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LGBM) without over-sampling to classify CTG readings into three categories: healthy, suspected, and pathological. Secondly, we employ an ensemble of the above-described classifiers with the over-sampling method. We use a random over-sampling technique to balance CTG records to train the ensemble models. We use 3602 CTG readings to train the ensemble classifiers and 1201 records to evaluate them. The outcomes of these classifiers are then fed into the soft voting classifier to obtain the most accurate results. <b>Results::</b> Each classifier evaluates accuracy, Precision, Recall, F1-scores, and Area Under the Receiver Operating Curve (AUROC) values. Results reveal that the XGBoost, LGBM, and CatBoost classifiers yielded 99% accuracy. <b>Conclusion::</b> Using ensemble classifiers over a balanced CTG dataset improves the detection accuracy compared to the previous studies and our first experiment. A soft voting classifier then eliminates the weakness of one individual classifier to yield superior performance of the overall model.

Development and validation of a prognostic 15-gene signature for stratifying HER2+/ER+ breast cancer

BackgroundHuman epidermal growth receptor 2-positive (HER2+) breast cancer (BC) is a heterogeneous subgroup. Estrogen receptor (ER) status is emerging as a predictive marker within HER2+ BCs, with the HER2+/ER+ cases usually having better survival in the first 5 years after diagnosis but have higher recurrence risk after 5 years compared to HER2+/ER-. This is possibly because sustained ER signaling in HER2+ BCs helps escape the HER2 blockade. Currently HER2+/ER+ BC is understudied and lacks biomarkers. Thus, a better understanding of the underlying molecular diversity is important to find new therapy targets for HER2+/ER+ BCs. MethodsIn this study, we performed unsupervised consensus clustering together with genome-wide Cox regression analyses on the gene expression data of 123 HER2+/ER+ BC from The Cancer Genome Atlas Breast Invasive Carcinoma (TCGA-BRCA) cohort to identify distinct HER2+/ER+ subgroups. A supervised eXtreme Gradient Boosting (XGBoost) classifier was then built in TCGA using the identified subgroups and validated in another two independent datasets (Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) and Gene Expression Omnibus (GEO) (accession number GSE149283)). Computational characterization analyses were also performed on the predicted subgroups in different HER2+/ER+ BC cohorts. ResultsWe identified two distinct HER2+/ER+ subgroups with different survival outcomes using the expression profiles of 549 survival-associated genes from the Cox regression analyses. Genome-wide gene expression differential analyses found 197 differentially expressed genes between the two identified subgroups, with 15 genes overlapping the 549 survival-associated genes.XGBoost classifier, using the expression values of the 15 genes, achieved a strong cross-validated performance (Area under the curve (AUC) = 0.85, Sensitivity = 0.76, specificity = 0.77) in predicting the subgroup labels. Further investigation partially confirmed the differences in survival, drug response, tumor-infiltrating lymphocytes, published gene signatures, and CRISPR-Cas9 knockout screened gene dependency scores between the two identified subgroups. ConclusionThis is the first study to stratify HER2+/ER+ tumors. Overall, the initial results from different cohorts showed there exist two distinct subgroups in HER2+/ER+ tumors, which can be distinguished by a 15-gene signature. Our findings could potentially guide the development of future precision therapies targeted on HER2+/ER+ BC.

Revolutionizing Fertilization Strategies with Machine Learning-Driven Nutrient Prediction

In the pursuit of sustainable agriculture, the effective management of fertilization strategies stands as a critical imperative. This abstract explores the transformative potential encapsulated within "Revolutionizing Fertilization Strategies with Machine Learning-Driven Nutrient Prediction." Traditional agricultural practices often grapple with imprecise fertilization, leading to inefficiencies, overuse of resources, and environmental ramifications. This study introduces an innovative approach that integrates advanced Machine Learning (ML) techniques with agronomic insights to accurately predict plant nutrient requirements. By harnessing comprehensive datasets encompassing soil properties, crop categorizations, and historical growth data, an intricate ML model is formulated. Employing sophisticated algorithms such as Random Forest, XG Boost Classifier the model uncovers intricate interdependencies shaping nutrient absorption. Through an iterative process of training, validation, and optimization, the model attains the capability to anticipate nuanced nutrient demands across diverse growth stages and crop typologies. By supplying real-time, data-informed intelligence on nutrient requirements, farmers are empowered to tailor fertilization approaches with precision, curbing resource wastage and diminishing nutrient runoff. Additionally, this ML-Centered methodology aligns seamlessly with the ambitions of sustainable agriculture, channelling efforts toward resource efficiency and environmental stewardship.

FrDSM: An Ensemble Predictor With Effective Feature Representation for Deleterious Synonymous Mutation in Human Genome.

With the discovery of causality between synonymous mutations and diseases, it has become increasingly important to identify deleterious synonymous mutations for better understanding of their functional mechanisms. Although several machine learning methods have been proposed to solve the task, an effective feature representation method that can make use of the inner difference and relevance between deleterious and benign synonymous mutations is still challenging considering the vast number of synonymous mutations in human genome. In this work, we developed a robust and accurate predictor called frDSM for deleterious synonymous mutation prediction using logistic regression. More specifically, we introduced an effective feature representation learning method which exploits multiple feature descriptors from different perspectives including functional scores obtained from previously computational methods, evolutionary conservation, splicing and sequence feature descriptors, and these features descriptors were input into the 76 XGBoost classifiers to obtain the predictive probabilities values. These probabilities were concatenated to generate the 76-dimension new feature vector, and feature selection method was used to remove redundant and irrelevant features. Experimental results show that frDSM enables robust and accurate prediction than the competing prediction methods with 31 optimal features, which demonstrated the effectiveness of the feature representation learning method. frDSM is freely available at http://frdsm.xialab.info.

Fingertip Detection Algorithm Based on Maximum Discrimination HOG Feature in Complex Background

Gesture fingertip detection plays a vital role in human-computer interaction applications such as VR and robot control. To improve the accuracy of fingertip detection in complex background, this paper proposes a fingertip detection algorithm based on a novel maximum discrimination HOG feature. Firstly, the Holistically-nested edge detection algorithm is used to detect the edge of hand images and non-hand images, and the HOG features of the contour edge are extracted to obtain the training set of positive and negative samples to reduce the influence of illumination, color, and texture on fingertip detection. Secondly, the maximum discrimination features are filtered from the positive sample set by a custom maximum discrimination feature filter and stored as a feature dictionary. The filtered maximum discrimination features and negative sample set are input into the XGBoost classifier for training and the voting rights classifier is obtained. Filtering maximum discrimination features can reduce the interference of irrelevant features and improve the performance of the classifier. Then, the KNN algorithm is used to find the best match in the dictionary, and the final fingertip position is obtained by the Meanshift algorithm. Finally, the algorithm in this paper is tested. The test results show that the accuracy of fingertip detection in this paper can reach 99 %, and the detected RMSE is kept in the range of 5 pixels, which is higher than that based on YCbCr skin color segmentation, YOLO target detection, and YOLO-YCbCr.

A novel heterogenous ensemble theory for symmetric 5G cells segmentation: Intelligent RAN analytics

MNOs are investing more in 5G, rolling out sites in urban and specific rural areas. Meanwhile, it remains imperative to consistently maintain the network performance above a certain threshold for optimal user experience. Symmetric network cells characterized by parallel attributes in terms of capacity and coverage are instrumental in planning, optimization, and resource allocation. However, the variation in environmental factors introduces divergence in network cells' behavior, symmetric or not. Therefore, the need arises for intelligent analytic processes within the RAN system to categorize symmetric cells based on their performance and behavior. Intelligent optimization and analytics in 5G rely on the accurate and automated identification of cells exhibiting symmetric behavior, enabling bulk optimization operations. In this paper, we develop and assess a clustering approach using a heterogenous ensemble method to group 5G cells based on their key performance attributes to facilitate network optimization tasks. The approach involves a synergistic integration of K-means and hierarchical clustering algorithms, enabling dynamic segmentation of cells based on their performance behavior. Leveraging the clustering output, we train an XGBoost classifier, paving the way for a comprehensive analytics framework and problematic or poor-performing cells’ detection. We apply the study model to real-world 5G RAN metrics and evaluate the proposed method in terms of clustering accuracy and convergence. The study output showcases the efficacity of the heterogenous ensemble approach compared to individual clustering algorithms, providing a valuable baseline for network performance enhancement. With such a dynamic approach for analyzing 5G new radio (NR) performance, MNOs can move toward intelligent and self-aware networks, making informed decisions regarding resource allocation and coverage optimization.

Chronic kidney disease prediction based on machine learning algorithms

Chronic kidney disease (CKD) is a dangerous ailment that can last a person’s entire life and is caused by either kidney malignancy or decreased kidney functioning. It is feasible to halt or slow the progression of this chronic disease to an end-stage wherein dialysis or surgical intervention is the only method to preserve a patient’s life. Earlier detection and appropriate therapy can increase the likelihood of this happening. Throughout this research, the potential of several different machine learning approaches for providing an early diagnosis of CKD has been investigated. There has been a significant amount of research conducted on this topic. Nevertheless, we are bolstering our approach by making use of predictive modeling. Therefore, in our approach, we investigate the link that exists between data factors as well as the characteristics of the target class. We are capable of constructing a collection of prediction models with the help of machine learning and predictive analytics, thanks to the better measures of attributes that can be introduced using predictive modeling. This study starts with 25 variables in addition to the class property, but by the end, it has narrowed the list down to 30% of those parameters as the best subset to identify CKD. Twelve different machine learning-based classifiers have been tested in a supervised learning environment. Within the confines of a supervised learning environment, a total of 12 different machine learning-based classifiers have indeed been examined, with the greatest performance indicators being an accuracy of 0.983, a precision of 0.98, a recall of 0.98, and an F1-score of 0.98 for the XgBoost classifier. The way the research was done leads to the conclusion that recent improvements in machine learning, along with the help of predictive modeling, make for an interesting way to find new solutions that can then be used to test the accuracy of prediction in the field of kidney disease and beyond.