Ensemble Of Decision Trees Research Articles

An Architecture as an Alternative to Gradient Boosted Decision Trees for Multiple Machine Learning Tasks

Deep networks-based models have achieved excellent performances in various applications for extracting discriminative feature representations by convolutional neural networks (CNN) or recurrent neural networks (RNN). However, CNN or RNN may not work when handling data without temporal/spatial structures. Therefore, finding a new technique to extract features instead of CNN or RNN is a necessity. Gradient Boosted Decision Trees (GBDT) can select the features with the largest information gain when building trees. In this paper, we propose an architecture based on the ensemble of decision trees and neural network (NN) for multiple machine learning tasks, e.g., classification, regression, and ranking. It can be regarded as an extension of the widely used deep-networks-based model, in which we use GBDT instead of CNN or RNN. This architecture consists of two main parts: (1) the decision forest layers, which focus on learning features from the input data, (2) the fully connected layers, which focus on distilling knowledge from the decision forest layers. Powered by these two parts, the proposed model could handle data without temporal/spatial structures. This model can be efficiently trained by stochastic gradient descent via back-propagation. The empirical evaluation results of different machine learning tasks demonstrate the the effectiveness of the proposed method.

Machine Learning Based Predictions of Airfoil Aerodynamic Coefficients for Reynolds Number Extrapolations

This study investigates the application of various machine learning (ML) algorithms for predicting two critical aerodynamic coefficients, i.e. the maximum lift coefficient (C l max ) and the minimum drag coefficient (C d min ), for wind turbine airfoils at any given Reynolds number. We propose to use clustering techniques to group similar airfoil shapes and use the created partitions to predict unseen airfoil properties utilizing their similarity. Here, we also represent airfoils in the PARSEC low dimensional space, rather than high dimensional airfoil points space, to remedy the small number of training data. For this purpose, an extended experimental airfoil database is created and used for training models based on five different ML algorithms. We observe that the Decision Tree Ensemble (DTE), Random Forest (RF) and multi-layer perceptron (MLP) models emerge as the most effective predictors for C l max and C d min . Testing these two ML models on three additional airfoil cases not included in the training database shows that the C l max prediction performance is generally reasonable, with error levels being around 5% on average. In contrast, the prediction error levels for C d min are usually higher, with an average of around 15%.

Early prediction of severe ICANS after standard-of-care CD19 CAR T-cell therapy using gradient-boosted classification trees.

7034 Background: Severe immune effector cell-associated neurotoxicity syndrome (ICANS) has arisen as a major complication leading to morbidity, mortality, and increased resource utilization. There is an urgent need to accurately predict the development of severe ICANS after CAR T-cell therapy. Here, we demonstrate the high performance of the XGBoost (“eXtreme Gradient Boosting”) machine learning algorithm in predicting the development of severe ICANS using commonly available laboratory and vital sign data from patients with B-cell lymphomas undergoing standard-of-care CD19 CAR T-cell therapy. Methods: We included patients who received axicabtagene ciloleucel (axi-cel) or brexucabtagene autoleucel (brexu-cel) per standard of care. ICANS was graded by CTCAE 4.03 criteria (pre-12/2018) or ASTCT Consensus Grading System (post-1/2019). Predictions were generated using XGBoost, a widely used supervised machine learning algorithm that trains ensembles of decision trees to learn iteratively from prior trees and allows for very flexible modeling (e.g. non-linear effects, complex high-order interaction effects). K-fold cross-validation was used to assess calibration and overfitting. Variables assessed included age, ferritin, CRP, LDH, IL-6, fibrinogen, platelet count, and temperature at pre-infusion, day 0, and day +3 post-infusion timepoints. Results: A total of 175 patients were included with 39 receiving brexu-cel (22%) and 136 receiving axi-cel (78%). Grade ≥3 ICANS occurred in 40 patients (23.3%). XGBoost modeling of factors demonstrated that serum ferritin level on day +3 was the most important variable predictive of grade ≥3 ICANS (accuracy gain: 0.28). Other influential factors included day 0 and +3 platelet count, patient age, day +3 IL-6 level, day 0 fibrinogen level, and day 0 and +3 CRP level, in descending order of importance. The XGBoost model maintained high discrimination (ability to distinguish high-risk from low-risk patients; mean AUROC 0.74, sensitivity 0.95, specificity 0.90). Longitudinal LOESS smoothing confirmed associations between severe ICANS and elevation in serum ferritin at day 0 (OR 3.01, 95% CI 1.41 – 6.73, p = 0.005) and day +3 (OR 5.50, 95% CI 2.31-14.5, p <0.001). Conclusions: A supervised machine learning model using XGBoost incorporating age, temperature and commonly accessible laboratory data including ferritin, CRP, LDH, IL-6, fibrinogen, and platelet count predicted severe ICANS prior to the development of symptoms with high discrimination (AUROC 0.74) in patients undergoing standard of care CAR T-cell therapy. This allows for the early identification of patients at highest risk for developing severe ICANS who may benefit from prophylactic interventions. Once externally validated, we plan to develop a user-friendly web application to generate individualized predictions from our model.

Predictive machine learning model for mechanical dilatation in transvenous lead extraction procedures

Abstract Background Transvenous lead extraction (TLE) remains a procedure that requires a high level of expertise, with a doubled risk of death and clinical failure when performed in low-volume centers compared to high-volume ones (1). Purpose The aim of this study was to create a machine learning (ML)-based risk stratification system for predicting the need for mechanical dilatation in patients undergoing TLE due to infection. Methods We designed a ML-based risk stratification system trained with data from our registry to predict the need for mechanical dilatation in patients undergoing TLE for infection. An extensive evaluation of 5 different ML models (k-nearest neighbors, support vector machine, decision tree, and decision tree ensembles, such as random forest and gradient boosting machine) was conducted to identify a classifier with the highest potential to correctly predict previously unseen patients. Results Data to train the model was extracted from our 25-year registry of patients undergoing TLE (June 1998 – March 2023), for a total of 491 patients (77.8% male; age 69.7 ± 12.8 years) and 938 leads (ICD 21.2%; pacing 78.8%; indwelling time 61 ± 60 months) removed with success in 100% of cases. Each patient was represented by a set of 21 attributes (14 clinical, 7 device-related). Manual traction (MT) was used in 27.5% of cases, and mechanical dilatation (MD) was employed in the remaining 72.5% of cases. 5-fold nested cross validation was used to estimate performances: in turn, 393 patients were used for training and model selection, and 98 patients were used for independent testing. According to the evaluation, Gradient Boosting Machine performed best, achieving test accuracy of 89% (+/- 2% std. dev.), test sensitivity of 95% (+/- 3% std. dev.), test specificity of 73% (+/- 8% std. dev.), test AUROC of 92% (+/- 1% std. dev.). A further interpretability analysis on the best performing decision tree was conducted, showing remarkable adherence between the internal decisions taken by the model to make predictions and the current clinical practice for TLE. Conclusion ML models have enhanced the prediction of MD requirements in TLE procedures. This data may support the referral of patients to specialized and high-volume centers, thereby improving preoperative risk assessment.Feature importanceROC Curves

PREDICTIVE MACHINE LEARNING MODEL FOR MECHANICAL DILATATION IN TRANSVENOUS LEAD EXTRACTION PROCEDURES

Abstract Background Transvenous lead extraction (TLE) remains a procedure that requires a high level of expertise, with a doubled risk of death and clinical failure when performed in low–volume centers compared to high–volume ones. PURPOSE The aim of this study was to create a machine learning (ML)–based risk stratification system for predicting the need for mechanical dilatation in patients undergoing TLE due to infection. Methods We designed a ML–based risk stratification system trained with data from our registry to predict the need for mechanical dilatation in patients undergoing TLE for infection. An extensive evaluation of 5 different ML models (k–nearest neighbors, support vector machine, decision tree, and decision tree ensembles, such as random forest and gradient boosting machine) was conducted to identify a classifier with the highest potential to correctly predict previously unseen patients. Results Data to train the model was extracted from our 25–year registry of patients undergoing TLE (June 1998 – March 2023), for a total of 491 patients (77.8% male; age 69.7 ± 12.8 years) and 938 leads (ICD 21.2%; pacing 78.8%; indwelling time 61 ± 60 months) removed with success in 100% of cases. Each patient was represented by a set of 21 attributes (14 clinical, 7 device–related). Manual traction (MT) was used in 27.5% of cases, and mechanical dilatation (MD) was employed in the remaining 72.5% of cases. 5–fold nested cross validation was used to estimate performances: in turn, 393 patients were used for training and model selection, and 98 patients were used for independent testing. According to the evaluation, Gradient Boosting Machine performed best, achieving test accuracy of 89% (+/– 2% std. dev.), test sensitivity of 95% (+/– 3% std. dev.), test specificity of 73% (+/– 8% std. dev.), test AUROC of 92% (+/– 1% std. dev.). A further interpretability analysis on the best performing decision tree was conducted, showing remarkable adherence between the internal decisions taken by the model to make predictions and the current clinical practice for TLE. Conclusion ML models have enhanced the prediction of MD requirements in TLE procedures. This data may support the referral of patients to specialized and high–volume centers, thereby improving preoperative risk assessment.

Lumbar spine MRI annotation with intervertebral disc height and Pfirrmann grade predictions.

Many lumbar spine diseases are caused by defects or degeneration of lumbar intervertebral discs (IVD) and are usually diagnosed through inspection of the patient's lumbar spine MRI. Efficient and accurate assessments of the lumbar spine are essential but a challenge due to the size of the clinical radiologist workforce not keeping pace with the demand for radiology services. In this paper, we present a methodology to automatically annotate lumbar spine IVDs with their height and degenerative state which is quantified using the Pfirrmann grading system. The method starts with semantic segmentation of a mid-sagittal MRI image into six distinct non-overlapping regions, including the IVD and vertebrae regions. Each IVD region is then located and assigned with its label. Using geometry, a line segment bisecting the IVD is determined and its Euclidean distance is used as the IVD height. We then extract an image feature, called self-similar color correlogram, from the nucleus of the IVD region as a representation of the region's spatial pixel intensity distribution. We then use the IVD height data and machine learning classification process to predict the Pfirrmann grade of the IVD. We considered five different deep learning networks and six different machine learning algorithms in our experiment and found the ResNet-50 model and Ensemble of Decision Trees classifier to be the combination that gives the best results. When tested using a dataset containing 515 MRI studies, we achieved a mean accuracy of 88.1%.

Hybrid Random Forest Regression and Artificial Neural Networks for Modelling and Monitoring the State of Health of Li-Ion Battery

Lithium-ion batteries play a significant role in modern energy storage systems for a number of applications, including renewable energy installations, electric cars, and portable devices. In order to guarantee the long-term dependability and safety of these battery packs, comprehensive modelling and monitoring approaches are required for accurate State of Health (SoH) evaluation. This research suggests an innovative strategy that combines the strengths of Random Forest Regression (RFR) and Artificial Neural Networks (ANN) for reliable Li-ion battery SoH modelling and monitoring. The research starts with the collection of thorough Li-ion battery data, which includes voltage, current, temperature, and cycling characteristics. The battery dataset's nonlinear relationships are captured utilising RFR as the main modelling method. In terms of feature selection and prediction precision, RFR's ensemble of decision trees shines, making it a strong contender for modelling complex battery behaviour. The SoH prediction is then enhanced by the use of ANN, a deep learning framework renowned for its ability to extract detailed patterns from data. The characteristics of RFR are complemented by ANN's capacity to discover hidden features and nonlinear correlations, improving the overall prediction performance. The suggested hybrid approach, that combines RFR and ANN, is trained and verified utilising a diverse database acquired from Li-ion battery sources under operational circumstances. To assess the prediction accuracy of a suggested framework measures like Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and R-squared (R2) values are employed. The outcomes show that the hybrid RFR-ANN model works better than the individual RFR and ANN models, producing better SoH predictions. A viable solution for real-time Li-ion battery health monitoring is provided by this combination of conventional ML and deep learning approaches, which demonstrates the synergy among complexity and interpretability. This research advances battery SoH assessment and paves the way for the practical incorporation of precise monitoring systems into applications like electric vehicles, where timely and accurate SoH data is essential for maximising battery performance and extending operational lifespan.

Creditworthiness pattern prediction and detection for GCC Islamic banks using machine learning techniques

PurposeBank creditworthiness refers to the evaluation of a bank’s ability to meet its financial obligations. It is an assessment of the bank’s financial health, stability and capacity to manage risks. This paper aims to investigate the credit rating patterns that are crucial for assessing creditworthiness of the Islamic banks, thereby evaluating the stability of their industry.Design/methodology/approachThree distinct machine learning algorithms are exploited and evaluated for the desired objective. This research initially uses the decision tree machine learning algorithm as a base learner conducting an in-depth comparison with the ensemble decision tree and Random Forest. Subsequently, the Apriori algorithm is deployed to uncover the most significant attributes impacting a bank’s credit rating. To appraise the previously elucidated models, a ten-fold cross-validation method is applied. This method involves segmenting the data sets into ten folds, with nine used for training and one for testing alternatively ten times changeable. This approach aims to mitigate any potential biases that could arise during the learning and training phases. Following this process, the accuracy is assessed and depicted in a confusion matrix as outlined in the methodology section.FindingsThe findings of this investigation reveal that the Random Forest machine learning algorithm superperforms others, achieving an impressive 90.5% accuracy in predicting credit ratings. Notably, our research sheds light on the significance of the loan-to-deposit ratio as a primary attribute affecting credit rating predictions. Moreover, this study uncovers additional pivotal banking features that intensely impact the measurements under study. This paper’s findings provide evidence that the loan-to-deposit ratio looks to be the purest bank attribute that affects credit rating prediction. In addition, deposit-to-assets ratio and profit sharing investment account ratio criteria are found to be effective in credit rating prediction and the ownership structure criterion came to be viewed as one of the essential bank attributes in credit rating prediction.Originality/valueThese findings contribute significant evidence to the understanding of attributes that strongly influence credit rating predictions within the banking sector. This study uniquely contributes by uncovering patterns that have not been previously documented in the literature, broadening our understanding in this field.

Perennial snow and ice cover change from 2001 to 2021 in the Hindu-Kush Himalayan region derived from the Landsat analysis-ready data

The changing climate directly affects spatial and temporal patterns of snow and ice cover globally and in the Hindu-Kush Himalayan (HKH) region. In the HKH, around 3.3 billion people across 11 countries depend on water originating from mountain glaciers and snowfields, and melting snow cover has a direct impact on their livelihood and well-being. Various studies have shown that the snow and ice cover in the HKH is declining at an alarming rate but have been limited in geographic, spatial and temporal scales. Here, we employed the Global Land Analysis and Discovery analysis ready Landsat time-series data (GLAD ARD) to map changes in perennial snow and ice between 2001 and 2021 at five-year epochs using decision tree ensemble models. These maps were used to create a stratified sampling design for reference data collection to estimate area and map accuracy. All five-year epoch maps have user's accuracies above 90% and producer's accuracies above 91%. Our sample data analysis showed that one-eighth of the extent of perennial snow and ice in the HKH, totalling 15,770 km2 (CI ± 3195 km2), disappeared over the last two decades with 105,935 km2 (CI ± 3396 km2) remaining in 2021. From map-based estimates, the largest decline of perennial snow and ice cover among the mountain ranges was found in the Himalayas with an estimated reduction of 5741 km2. Among HKH countries, China had the largest area of perennial snow and ice reduction of 10,654 km2, Nepal had the highest net perennial snow and ice reduction rate (31%). Among the river basins, the maximum net loss of perennial snow and ice cover between 2001 and 2021 was in the Indus (24.8%), followed by Brahmaputra (18.3%), and Tarim (15.7%). Our results confirm wide-spread loss of perennial snow and ice across the HKH region and provide both regionally consistent maps with derived area estimates at landform, national and basin-levels and methodology to continue monitoring snow and ice melt.

Semi-real-time decision tree ensemble algorithms for very short-term solar irradiance forecasting

Industrial activities are transitioning towards decarbonization, focusing on renewable energy sources, particularly photovoltaic solar energy. However, the inherent high variability of photovoltaic energy poses challenges. Some of them can be partially addressed by predicting electricity production, which in the case of photovoltaic solar energy is heavily based on solar irradiance prediction. Although extensive research has been conducted in this field, there is a noticeable gap in research regarding very short-term (intra-minute) forecasting under high-variability scenarios. In this proposal, real data from a photovoltaic solar plant in Alderville (Canada) were used to predict irradiance with a horizon of 15 and 30 s. The objective is to make this prediction in near-real time. To achieve this, we propose the use of machine learning algorithms based on decision tree ensembles, due to their low computational training cost and known effectiveness. On the other hand, we propose pre-processing the data through a temporal and spatial correlation analysis between measurements from different sensors. Feature selection analysis allows us to determine the direction of the wind and consequently identify the most relevant panels for model training. This preprocessing enhances the model retraining without the need for external information such as sky images or wind speed and direction on days with highly variable cloud cover. The presented methodology offers promising results with significantly reduced training times, demonstrating the suitability of this semi-online training approach for highly variable time series forecasting.

Classifying protein kinase conformations with machine learning.

Protein kinases are key actors of signaling networks and important drug targets. They cycle between active and inactive conformations, distinguished by a few elements within the catalytic domain. One is the activation loop, whose conserved DFG motif can occupy DFG-in, DFG-out, and some rarer conformations. Annotation and classification of the structural kinome are important, as different conformations can be targeted by different inhibitors and activators. Valuable resources exist; however, large-scale applications will benefit from increased automation and interpretability of structural annotation. Interpretable machine learning models are described for this purpose, based on ensembles of decision trees. To train them, a set of catalytic domain sequences and structures was collected, somewhat larger and more diverse than existing resources. The structures were clustered based on the DFG conformation and manually annotated. They were then used as training input. Two main models were constructed, which distinguished active/inactive and in/out/other DFG conformations. They considered initially 1692 structural variables, spanning the whole catalytic domain, then identified ("learned") a small subset that sufficed for accurate classification. The first model correctly labeled all but 3 of 3289 structures as active or inactive, while the second assigned the correct DFG label to all but 17 of 8826 structures. The most potent classifying variables were all related to well-known structural elements in or near the activation loop and their ranking gives insights into the conformational preferences. The models were used to automatically annotate 3850 kinase structures predicted recently with the Alphafold2 tool, showing that Alphafold2 reproduced the active/inactive but not the DFG-in proportions seen in the Protein Data Bank. We expect the models will be useful for understanding and engineering kinases.

Forecasting The Population of China From 2020 To 2025 Based on Random Forest and Linear Regression

The population dynamics of a country do play a vital role in its economic and political development. The COVID-19 epidemic has significantly affected the world's population. However, most people only care about the negative influence caused by COVID-19 but ignore the positive influence. This article uses two machine learning models, the random forest model, and the linear regression model, to predict the population change in China if there were no COVID-19 pandemic. With the predicted results, this article can compare the potential positive impact of the pandemic. This paper tries to fit two popular models, namely, Random Roest and Linear Regression to forecast the population of China from 2020 to 2025. The historical birth rate, death rate, and GDP growth rate of China are collected as features adding to the models to decline the error. For the Random Forest model, this paper set up an ensemble of decision trees to predict the future population of China. For the Linear Regression model, our features and population fit a linear connection. The findings of two models are compared in this article, and it is suggested that the random forest model is more suited for population forecasting. In addition, according to this study, the COVID-19 pandemic has some favorable effects on economic growth and birth rates. The article emphasizes the need to not only focus on the negative effects of the pandemic. Furthermore, the article points out that the linear regression model has poor fitting results for non-linear relationships. It suggests exploring more non-linear models for prediction and considering more influential parameters.

High-performance imbalanced learning ensembles of decision trees for detecting mineralization anomalies from geochemical exploration data

How to effectively detect geochemical anomalies associated with mineralization is a challenging task due to the extreme class-imbalance of geochemical exploration data. To address this challenge, various machine learning techniques have been employed to detect geochemical anomalies associated with mineralization. However, almost all of these machine learning techniques have their limitations when it comes to modeling class-imbalanced geochemical exploration data. To establish efficient robust high-performance models for detecting geochemical anomalies associated mineralization, a case study was carried out in the Helong area, Jilin Province, China. Decision tree (C4.5) classifiers were used as the base or weak classifiers, four imbalanced learning ensemble models, including self-paced ensemble model, under-bagging ensemble model, synthetic minority oversampling technique (SMOTE)-boost ensemble model and random under sampling (RUS)-boost ensemble model were established and compared in the detection of polymetallic mineralization anomalies from the 1:50,000 stream sediment survey data. The output of the base classifier is a vector consisting of the probability that the sample belongs to a mineralization anomaly and the probability that the sample belongs to the background. The performance of each ensemble model was evaluated using the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC). The ROC curves of the four models are close to the upper left corner of the ROC space. The AUC values of the four models are not lower than 0.9589. The polymetallic mineralization anomalies detected by the four models account for less than 7 % of the entire study area, but contain not less than 93 % of the polymetallic deposits found in the study area. In addition, the polymetallic mineralization anomalies detected by the four models spatially coincide with the regional controlling factors of polymetallic mineralization in the study area. To summarize, the four models have very high performance in detecting polymetallic mineralization anomalies, and the polymetallic mineralization anomaly detection results are consistent with the regional geological and metallogenic characteristics in the study area. Therefore, the imbalanced learning ensemble techniques are powerful tools for the establishment of high-performance ensemble classification models for detecting mineralization anomalies from geochemical exploration data.

A combined EBSD and machine learning study of predicting deformation twinning in BCC Fe81Ga19 alloy

Compared to face-centered cubic (FCC) metals, body-centered cubic (BCC) metals demonstrate more intricate deformation behavior and microstructural characteristics because of the influence of intrinsic unstable stacking faults and defect structure. To investigate the unique phenomenon of deformation twinning in Fe81Ga19 alloy with BCC structure at moderate temperature and low strain rate, four twin nucleation models were constructed using a combination of electron backscatter diffraction (EBSD) technique and machine learning. These models are aimed to predict twin nucleation of grains and explore the relationship between twin nucleation and microstructural attributes. The study revealed that amongst the 235 grains analyzed during in-situ compressive deformation, 32 grains experienced twinning. Nine attributes influencing twin nucleation were selected and ranked according to their importance. Grain area, average image quality, neighboring grain count, and the Schmid factor of twinning systems exhibited significant influence on twin nucleation. Decision tree and tree ensemble (XGBoost) achieved 94% and 96% accuracy, respectively, on the test set, showcasing robust generalization capability. Due to the optimization of hyperparameters, support vector machine (SVM) and artificial neural network (ANN) exhibited outstanding performance. The ANN model achieved an accuracy of 97% and an F1 score of 0.91 on the test set, while the SVM model achieved an accuracy of 98% and an F1 score of 0.94, indicating superior performance. Furthermore, the study revealed that the twinning stress in Fe81Ga19 alloy exhibited weak responsiveness to temperature during deformation, and the intrinsic factors of grains were identified as crucial factors influencing twin nucleation.

Finding Regions of Counterfactual Explanations via Robust Optimization

Counterfactual explanations (CEs) play an important role in detecting bias and improving the explainability of data-driven classification models. A CE is a minimal perturbed data point for which the decision of the model changes. Most of the existing methods can only provide one CE, which may not be achievable for the user. In this work, we derive an iterative method to calculate robust CEs (i.e., CEs that remain valid even after the features are slightly perturbed). To this end, our method provides a whole region of CEs, allowing the user to choose a suitable recourse to obtain a desired outcome. We use algorithmic ideas from robust optimization and prove convergence results for the most common machine learning methods, including decision trees, tree ensembles, and neural networks. Our experiments show that our method can efficiently generate globally optimal robust CEs for a variety of common data sets and classification models. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms—Discrete. Funding: This work was supported by the Nederlandse Organisatie voor Wetenschappelijk Onderzoek [Grant OCENW.GROOT.2019.015, Optimization for and with Machine Learning (OPTIMAL)]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoc.2023.0153 .

Cognitive reserve, depressive symptoms, obesity, and change in employment status predict mental processing speed and executive function after COVID-19.

The risk factors for post-COVID-19 cognitive impairment have been poorly described. This study aimed to identify the sociodemographic, clinical, and lifestyle characteristics that characterize a group of post-COVID-19 condition (PCC) participants with neuropsychological impairment. The study sample included 426 participants with PCC who underwent a neurobehavioral evaluation. We selected seven mental speed processing and executive function variables to obtain a data-driven partition. Clustering algorithms were applied, including K-means, bisecting K-means, and Gaussian mixture models. Different machine learning algorithms were then used to obtain a classifier able to separate the two clusters according to the demographic, clinical, emotional, and lifestyle variables, including logistic regression with least absolute shrinkage and selection operator (LASSO) (L1) and Ridge (L2) regularization, support vector machines (linear/quadratic/radial basis function kernels), and decision tree ensembles (random forest/gradient boosting trees). All clustering quality measures were in agreement in detecting only two clusters in the data based solely on cognitive performance. A model with four variables (cognitive reserve, depressive symptoms, obesity, and change in work situation) obtained with logistic regression with LASSO regularization was able to classify between good and poor cognitive performers with an accuracy and a weighted averaged precision of 72%, a recall of 73%, and an area under the curve of 0.72. PCC individuals with a lower cognitive reserve, more depressive symptoms, obesity, and a change in employment status were at greater risk for poor performance on tasks requiring mentalprocessing speedand executive function. Study registration: www.ClinicalTrials.gov , identifier NCT05307575.

Decision Tree Ensemble Approach for Recidivism Risk Classification Among Heinous Crime Convicts

Decision Tree Ensemble Approach for Recidivism Risk Classification Among Heinous Crime Convicts

Deep learning based decision tree ensembles for incomplete medical datasets.

In practice, the collected datasets for data analysis are usually incomplete as some data contain missing attribute values. Many related works focus on constructing specific models to produce estimations to replace the missing values, to make the original incomplete datasets become complete. Another type of solution is to directly handle the incomplete datasets without missing value imputation, with decision trees being the major technique for this purpose. To introduce a novel approach, namely Deep Learning-based Decision Tree Ensembles (DLDTE), which borrows the bounding box and sliding window strategies used in deep learning techniques to divide an incomplete dataset into a number of subsets and learning from each subset by a decision tree, resulting in decision tree ensembles. Two medical domain problem datasets contain several hundred feature dimensions with the missing rates of 10% to 50% are used for performance comparison. The proposed DLDTE provides the highest rate of classification accuracy when compared with the baseline decision tree method, as well as two missing value imputation methods (mean and k-nearest neighbor), and the case deletion method. The results demonstrate the effectiveness of DLDTE for handling incomplete medical datasets with different missing rates.

Optimizing Predictive Performance: Hyperparameter Tuning in Stacked Multi-Kernel Support Vector Machine Random Forest Models for Diabetes Identification

This study addresses the necessity for more advanced diagnostic tools in managing diabetes, a chronic metabolic disorder that leads to disruptions in glucose, lipid, and protein metabolism caused by insufficient insulin activity. The research investigates the innovative application of machine learning models, specifically Stacked Multi-Kernel Support Vector Machines Random Forest (SMKSVM-RF), to determine their effectiveness in identifying complex patterns in medical data. The innovative ensemble learning method SMKSVM-RF combines the strengths of Support Vector Machines (SVMs) and Random Forests (RFs) to leverage their diversity and complementary features. The SVM component implements multiple kernels to identify unique data patterns, while the RF component consists of an ensemble of decision trees to ensure reliable predictions. Integrating these models into a stacked architecture allows SMKSVM-RF to enhance the overall predictive performance for classification or regression tasks by optimizing their strengths. A significant finding of this study is the introduction of SMKSVM-RF, which displays an impressive 73.37% accuracy rate in the confusion matrix. Additionally, its recall is 71.62%, its precision is 70.13%, and it has a noteworthy F1-Score of 71.34%. This innovative technique shows potential for enhancing current methods and developing into an ideal healthcare system, signifying a noteworthy step forward in diabetes detection. The results emphasize the importance of sophisticated machine learning methods, highlighting how SMKSVM-RF can improve diagnostic precision and aid in the continual advancement of healthcare systems for more effective diabetes management.

METHOD FOR AGENT-ORIENTED TRAFFIC PREDICTION UNDER DATA AND RESOURCE CONSTRAINTS

Context. Problem of traffic prediction in a city is closely connected to the tasks of transportations in a city as well as air pollution detection in a city. Modern prediction models have redundant complexity when used for separate stations, require large number of measuring stations, long measurement period when predictions are made hourly. Therefore, there is a lack of method to overcome these constraints. The object of the study is a city traffic.
 Objective. The objective of the study is to develop a method for traffic prediction, providing models for traffic quantification at measuring stations in the future under data and resource constraints.
 Method. The method for agent-oriented traffic prediction under data and resource constraints was proposed in the paper. This method uses biLSTM models with input features, including traffic data obtained from agent, representing target station, and other agents, representing informative city stations. These agents are selected by ensembles of decision trees using Random Forest method. Input time period length is proposed to set using autocorrelation data.
 Results. Experimental investigation was conducted on traffic data taken in Madrid from 59 measuring stations. Models created by the proposed method had higher prediction accuracy with lower values of MSE, MAE, RMSE and higher informativeness compared to base LSTM models.
 Conclusions. Obtained models as study results have optimal number of input features compared to the known models, do not require complete system of city stations for all roads. It enables to apply these models under city traffic data and resource constraints. The proposed solutions provide high informativeness of obtained models with practically applicable accuracy level.