Ensemble Of Trees Research Articles

Evaluating SAFE AI principles using Wasserstein distance: a comparative study of Machine Learning models

Modern data-driven Artificial Intelligence (AI), powered by advanced Machine Learning (ML) models, is transforming financial technologies by enhancing financial inclusion, transparency and reducing transaction costs. However, the opaque nature of some complex ML models requires new statistical approaches to manage risks and ensure trustworthiness. In this paper, we present a novel method to evaluate the key principles of trustworthy AI - Sustainability (Robustness), Accuracy, Fairness, and Explainability (SAFE). While Babaei et al. [A Rank Graduation Box for SAFE AI. Expert Syst Appl. 259;2025:125239, 2025] introduced the Rank Graduation Box as a streamlined approach for assessing the principles of trustworthy AI, we extend this work by employing the Wasserstein distance. Our method offers a more nuanced and geometrically oriented comparison of ML models, particularly in contexts where shifts in economic or environmental conditions alter the prediction distributions. We apply this method to compare popular ML models, including Support Vector Machines, Ensemble Trees, K-Nearest Neighbours Linear and Logistic Regression. The proposal is validated using both simulated data and real-world data in the context of financial risk assessment. Our findings demonstrate that the Wasserstein distance offers nuanced and interpretable insights into model behaviour across the SAFE dimensions, making it a valuable tool for model selection and regulatory compliance in AI applications.

Comparison of ANN and XGBoost surrogate models trained on small numbers of building energy simulations.

Surrogate optimisation holds a big promise for building energy optimisation studies due to its goal to replace the use of lengthy building energy simulations within an optimisation step with expendable local surrogate models that can quickly predict simulation results. To be useful for such purpose, it should be possible to quickly train precise surrogate models from a small number of simulation results (10-100) obtained from appropriately sampled points in the desired part of the design space. Two sampling methods and two machine learning models are compared here. Latin hypercube sampling (LHS), widely accepted in building energy community, is compared to an exploratory Monte Carlo-based sequential design method mc-intersite-proj-th (MIPT). Artificial neural networks (ANN), also widely accepted in building energy community, are compared to gradient-boosted tree ensembles (XGBoost), model of choice in many machine learning competitions. In order to get a better understanding of the behaviour of these two sampling methods and two machine learning models, we compare their predictions against a large set of generated synthetic data. For this purpose, a simple case study of an office cell model with a single window and a fixed overhang, whose main input parameters are overhang depth and height, while climate type, presence of obstacles, orientation and heating and cooling set points are additional input parameters, was extensively simulated with EnergyPlus, to form a large underlying dataset of 729,000 simulation results. Expendable local surrogate models for predicting simulated heating, cooling and lighting loads and equivalent primary energy needs of the office cell were trained using both LHS and MIPT and both ANN and XGBoost for several main hyperparameter choices. Results show that XGBoost models are more precise than ANN models, and that for both machine learning models, the use of MIPT sampling leads to more precise surrogates than LHS.

Testing spatial out-of-sample area of influence for grain forecasting models

Abstract We examine the factors that determine if a grain forecasting model fit to one region can be transferred to another region. Prior research has proposed examining the area of applicability (AoA) of a model based on structurally similar characteristics in the Earth Observation predictors and weights based on the model derived feature importance. We expand on and evaluate this approach in the context of grain yield forecasting in Sub-Saharan Africa. Specifically, we evaluate an AoA methodology established for generating raster surfaces and apply it to vector supported grain data. We fit a series of ensemble tree models both within single countries and across multiple sets of countries and then test those models in countries excluded from the training set. We then calculate and decompose AoA measures and examine several different performance metrics. We find that the spatial transfer accuracy does not vary across season but does vary by average rainfall and across high, medium, and low yielding regions. In general, areas with higher yields and medium to high average rainfall tend to have higher accuracy for both model training and transfer. Finally, we find that fitting models with multiple countries provides more accurate out-of-sample estimates when compared to models fitted to a single country.

An Enhanced Tree Ensemble for Classification in the Presence of Extreme Class Imbalance

An Enhanced Tree Ensemble for Classification in the Presence of Extreme Class Imbalance

Tree smoothing: Post-hoc regularization of tree ensembles for interpretable machine learning

Random Forests (RFs) are powerful ensemble learning algorithms that are widely used in various machine learning tasks. However, they tend to overfit noisy or irrelevant features, which can result in decreased generalization performance. Post-hoc regularization techniques aim to solve this problem by modifying the structure of the learned ensemble after training. We propose a novel post-hoc regularization via tree smoothing for classification tasks to leverage the reliable class distributions closer to the root node whilst reducing the impact of more specific and potentially noisy splits deeper in the tree. Our novel approach allows for a form of pruning that does not alter the general structure of the trees, adjusting the influence of nodes based on their proximity to the root node. We evaluated the performance of our method on various machine learning benchmark data sets and on cancer data from The Cancer Genome Atlas (TCGA). Our approach demonstrates competitive performance compared to the state-of-the-art and, in the majority of cases, and outperforms it in most cases in terms of prediction accuracy, generalization, and interpretability.

A Study on the Layoff Problem After COVID-19 Based on Decision Tree Ensemble Methods

The COVID-19 pandemic, a cataclysmic event in modern history, has upended the global economy, precipitating extensive layoffs across diverse sectors. This study delves into the determinants of these workforce reductions through the lens of decision tree ensemble methods. By scrutinizing a comprehensive dataset, we harnessed statistical and machine learning algorithms to assess the significance of key variablesincluding industry type, geographic location, company development stage, and capital acquisitionon the scale of layoffs. This analysis further entailed predicting the total number of employees laid off and examining their distribution. The use of decision tree ensemble methods, such as random forests and gradient boosting, provided robust insights into the complex interplay of factors influencing layoff decisions. We discovered that industry type and company development stage were particularly critical in predicting layoff patterns, while geographic location and capital acquisition also played notable roles. This research offers a data-driven perspective on the layoff phenomena, shedding light on the multifaceted influences at play and offering a foundation for further inquiry and strategic policy development in response to economic downturns. By understanding these dynamics, policymakers and business leaders can better navigate future economic crises, potentially mitigating the adverse impacts on the workforce and promoting more resilient economic structures.

Data‐driven prediction of prolonged air leak after video‐assisted thoracoscopic surgery for lung cancer: Development and validation of machine‐learning‐based models using real‐world data through the ePath system

AbstractIntroductionThe reliability of data‐driven predictions in real‐world scenarios remains uncertain. This study aimed to develop and validate a machine‐learning‐based model for predicting clinical outcomes using real‐world data from an electronic clinical pathway (ePath) system.MethodsAll available data were collected from patients with lung cancer who underwent video‐assisted thoracoscopic surgery at two independent hospitals utilizing the ePath system. The primary clinical outcome of interest was prolonged air leak (PAL), defined as drainage removal more than 2 days post‐surgery. Data‐driven prediction models were developed in a cohort of 314 patients from a university hospital applying sparse linear regression models (least absolute shrinkage and selection operator, ridge, and elastic net) and decision tree ensemble models (random forest and extreme gradient boosting). Model performance was then validated in a cohort of 154 patients from a tertiary hospital using the area under the receiver operating characteristic curve (AUROC) and calibration plots.ResultsTo mitigate bias, variables with missing data related to PAL or those with high rates of missing data were excluded from the dataset. Fivefold cross‐validation indicated improved AUROCs when utilizing key variables, even post‐imputation of missing data. Dichotomizing continuous variables enhanced performance, particularly when fewer variables were employed in the decision tree ensemble models. Consequently, regression models incorporating seven key variables in complete case analysis demonstrated superior discriminatory ability for both internal (AUROCs: 0.77–0.84) and external cohorts (AUROCs: 0.75–0.84). These models exhibited satisfactory calibration in both cohorts.ConclusionsThe data‐driven prediction model implementing the ePath system exhibited adequate performance in predicting PAL post‐video‐assisted thoracoscopic surgery, optimizing variables and considering population characteristics in a real‐world setting.

Optimal iterative learning PI controller for SISO and MIMO processes with machine learning validation for performance prediction

The multivariable process plays a significant role in industrial applications, and designing a controller for the Multi-Input Multi-Output process is challenging due to dynamic process changes and interactions between system variables. Traditionally, the PI family of controllers has been used for its simple design, easy tuning, and quick deployment. However, these processes require complex control actions due to multiple loops in process plants. Thus, this paper proposes an Iterative Learning Controller Dead-time compensating PI, which utilizes the newly developed hybrid Simulated Annealing-Ant Lion Optimization algorithm for Single-Input Single-Output process simulation and real-time experimentation on the Quadruple Tank System. To validate the effectiveness of the developed controller, Machine Learning techniques such as regression and ensemble trees are used to accurately predict the actual system response using error values from respective processes. The simulation and experimental results demonstrate that the proposed controller achieved better performance. The regression and ensemble tree algorithm models effectively predicted the actual response. The obtained data shows that the proposed controller improved system stability and robustness by minimizing nearly half of the overshoot and improving settling time, with an average of 29.9596% faster than the other controller in the SISO process and 14.6116% in the MIMO process.

Ensembles of decision trees and gradient-based learning for employee turnover rate prediction

Employee turnover has a negative impact on business profitability. To tackle this issue, we can utilize computational advancements to forecast attrition and minimize expenses. We employed an HR Analytics dataset to investigate the feasibility of using these predictive models in decision support systems. We developed an ensemble of gradient-based decision trees that accurately predicted employee turnover and performed better than other sophisticated techniques. This approach demonstrates exceptional performance in handling structured and imbalanced data, effectively capturing intricate patterns. Gradient-based decision trees provide scalable solutions that effectively balance predictive accuracy and computational efficiency, making them well-suited for strategic business analysis. The importance of our findings lies in their ability to offer dependable insights for making well-informed decisions in business settings.

Thermal Runaway Diagnosis of Lithium-Ion Cells Using Data-Driven Method

Fault diagnosis is crucial to guarantee safe operation and extend the operating time while preventing the thermal runaway of the lithium-ion battery. This study presents a data-driven thermal runaway diagnosis framework where Bayesian optimization techniques are applied to optimize the hyperparameter of various machine learning techniques. We use different machine learning models such as support vector machine, naive Bayes, decision tree ensemble, and multi-layer perceptron to estimate a high likelihood of causes of thermal runaway by using the experimental measurements of open-source battery failure data. We analyze different evaluation metrics, including the prediction accuracy, confusion metrics, and receiver operating characteristic curves of different models. An experimental evaluation shows that the classification accuracy of the decision tree ensemble outperforms that of other models. Furthermore, the decision tree ensemble provides robust prediction accuracy even with the strictly limited dataset.

Application of SHAP and Multi-Agent Approach for Short-Term Forecast of Power Consumption of Gas Industry Enterprises

Currently, machine learning methods are widely applied in the power industry to solve various tasks, including short-term power consumption forecasting. However, the lack of interpretability of machine learning methods can lead to their incorrect use, potentially resulting in electrical system instability or equipment failures. This article addresses the task of short-term power consumption forecasting, one of the tasks of enhancing the energy efficiency of gas industry enterprises. In order to reduce the risks of making incorrect decisions based on the results of short-term power consumption forecasts made by machine learning methods, the SHapley Additive exPlanations method was proposed. Additionally, the application of a multi-agent approach for the decomposition of production processes using self-generation agents, energy storage agents, and consumption agents was demonstrated. It can enable the safe operation of critical infrastructure, for instance, adjusting the operation modes of self-generation units and energy-storage systems, optimizing the power consumption schedule, and reducing electricity and power costs. A comparative analysis of various algorithms for constructing decision tree ensembles was conducted to forecast power consumption by gas industry enterprises with different numbers of categorical features. The experiments demonstrated that using the developed method and production process factors reduced the MAE from 105.00 kWh (MAPE of 16.81%), obtained through expert forecasting, to 15.52 kWh (3.44%). Examples were provided of how the use of SHapley Additive exPlanation can increase the safety of the electrical system management of gas industry enterprises by improving experts’ confidence in the results of the information system.

Prediction of Flotation Deinking Performance: A Comparative Analysis of Machine Learning Techniques

Flotation deinking is one of the most widely used techniques for the separation of ink particles from cellulose fibers during the process of paper recycling. It is a complex process influenced by a variety of factors, and is difficult to represent and usually results in models that are inconvenient to implement and/or interpret. In this paper, a comprehensive study of several machine learning methods for the prediction of flotation deinking performance is carried out, including support vector regression, regression tree ensembles (random forests and boosting) and Gaussian process regression. The prediction relies on the development of a limited dataset that assumes representative data samples obtained under a variety of laboratory conditions, including different reagents, pH values and flotation residence times. The results obtained in this paper confirm that the machine learning methods enable the accurate prediction of flotation deinking performance even when the dataset used for training the model is limited, thus enabling the determination of optimal conditions for the paper recycling process, with only minimal costs and effort. Considering the low complexity of the Gaussian process regression compared to the aforementioned ensemble models, it should be emphasized that the Gaussian process regression gave the best performance in estimating fiber recovery (R2 = 97.77%) and a reasonable performance in estimating the toner recovery (R2 = 86.31%).

Hybrid lane change strategy of autonomous vehicles based on SOAR cognitive architecture and deep reinforcement learning

Research on lane change strategies for autonomous vehicles holds paramount importance in optimizing traffic flow efficiency, enhancing driving safety, and adapting to complex traffic environments. While numerous rule-based or machine-learning approaches have been explored to tackle the challenge of lane change on highways, they frequently exhibit limited performance owing to the complexity of driving environments. This study proposes a novel lane change strategy for autonomous vehicles, which utilizes a hybrid framework integrating the SOAR cognitive architecture and deep reinforcement learning (DRL) to address the lane change challenge on highways. First, we introduce a rule extraction algorithm, the RuleCOSI+, which is based on tree ensemble algorithms, designed to extract concise lane change rules from large-scale human driving data. These straightforward rules, together with traffic regulations and safety rules, constitute the long-term memory of the SOAR cognitive architecture, enabling transparent decision-making processes. Next, by analyzing the clipping mechanism of the proximal policy optimization (PPO) algorithm, we propose an Adaptive Clipping PPO (ACPPO) algorithm which is based on the importance of samples. This algorithm adopts different clipping strategies for SOAR samples and ACPPO samples during the training process, enabling the algorithm to more effectively utilize samples with different levels of importance. Then, we propose a hybrid decision-making algorithm: SOAR-ACPPO, which combines the SOAR cognitive architecture with the ACPPO algorithm. This algorithm leverages SOAR’s prior knowledge to effectively and safely guide agent learning. Finally, by selecting appropriate intervention probability and weaning strategy, the system avoids inappropriate knowledge intervention and ensures adequate environment exploration. Simulation experiments conducted using the CARLA simulator illustrate that the proposed strategy not only improves model learning efficiency but also enhances driving efficiency and safety. Additionally, it demonstrates a certain degree of human-like characteristics and interpretability.

The use of ensembles trees machine learning method in estimating damping of granular materials

Granular Materials (GM) employed within the mechanism of particle dampers attenuate the vibration energy due to their interparticle and particle-wall interactions. Estimating their damping effect using the analytical equivalent single mass approach overlooked the particles' individual losses that are built into the total damping. Alternatively, the numerical techniques (e.g., Discrete Element) are time-inefficient and computationally demanding. Therefore, this study explores the implementation of Machine learning (ML) algorithms to estimate the damping effect of GM. The ML model in this study will rely on a Data-Driven Modeling approach (DDM) incorporating the ensemble tress nonlinear regressor method. The models' training and testing data were obtained from an experimental setup of an acrylic beam internally integrated with Stainless Steel (SS) and glass spheres undergoing low excitation amplitude (RMS <1N). The main aim was to map between seven input features (e.g. filling ratio) and one targeted output: the beam's damped frequency response. Three ensemble trees' algorithms were used to create the DDM; Decision Tree, Random Forest, and XGBoost. The hyperparameter combination based on the Gridsearch CV function increases the prediction accuracy of each model. The developed ML models provided high accuracy (86-93%) in predicting the damping effect of the granular materials spheres.

Enhancing long-term vegetation monitoring in Australia: a new approach for harmonising the Advanced Very High Resolution Radiometer normalised-difference vegetation (NVDI) with MODIS NDVI

Abstract. Long-term, reliable datasets of satellite-based vegetation condition are essential for understanding terrestrial ecosystem responses to global environmental change, particularly in Australia, which is characterised by diverse ecosystems and strong interannual climate variability. We comprehensively evaluate several existing global Advanced Very High Resolution Radiometer (AVHRR) normalised-difference vegetation index (NDVI) products for their suitability for long-term vegetation monitoring in Australia. Comparisons with the MODIS NDVI highlight significant deficiencies, particularly over densely vegetated regions. Moreover, all the assessed products failed to adequately reproduce the interannual variability in the pre-MODIS era as indicated by Landsat NDVI anomalies. To address these limitations, we propose a new approach to calibrating and harmonising NOAA's Climate Data Record of AVHRR NDVI to the MODIS MCD43A4 NDVI for Australia using a gradient-boosting decision tree ensemble method. Two versions of the datasets are developed, one incorporating climate data in the predictors (“AusENDVI-clim”: Australian Empirical NDVI-climate) and another that is independent of climate data (“AusENDVI-noclim”). These datasets, spanning 1982–2013 at a spatial resolution of 0.05° and with a monthly time step, exhibit strong correlations (r2=0.89–0.94) and low mean errors compared with MODIS MCD43A4 NDVI (mean absolute error (MAE) = 0.014–0.028, RMSE = 0.021–0.046), accurately reproducing seasonal cycles over densely vegetated regions. Furthermore, they closely replicate the interannual variability in vegetation condition in the pre-MODIS era. A reliable method for gap-filling the AusENDVI record is also developed that leverages climate, atmospheric CO2 concentration, and woody-cover fraction predictors. The resulting synthetic NDVI dataset shows excellent agreement with the MODIS MCD43A4 NDVI and the recalibrated AVHRR NDVI time series (r2=0.82–0.95, MAE = 0.016–0.029, RMSE = 0.039–0.041). Finally, we provide a complete 41-year dataset where the gap-filled AusENDVI-clim from January 1982 to February 2000 is joined with the MODIS MCD43A4 NDVI from March 2000 to December 2022. Analysing 40-year per-pixel trends in Australia's annual maximum NDVI revealed increasing values, and shifts in the timing, of the annual peak NDVI across most of the continent, underscoring the dataset's potential to address crucial questions regarding the changing vegetation phenology and its drivers. The AusENDVI dataset can be used for studying Australia's changing vegetation dynamics and downstream impacts on the terrestrial carbon and water cycles, and it provides a reliable foundation for further research into the drivers of vegetation change. AusENDVI is open access and available at https://doi.org/10.5281/zenodo.10802703 (Burton et al., 2024).

A Comparative Study of Pavement Roughness Prediction Models under Different Climatic Conditions

Predicting the International Roughness Index (IRI) is crucial for maintaining road quality and ensuring the safety and comfort of road users. Accurate IRI predictions help in the timely identification of road sections that require maintenance, thus preventing further deterioration and reducing overall maintenance costs. This study aims to develop robust predictive models for the IRI using advanced machine learning techniques across different climatic conditions. Data were sourced from the Ministry of Energy and Infrastructure in the UAE for localized conditions coupled with the Long-Term Pavement Performance (LTPP) database for comparison and validation purposes. This study evaluates several machine learning models, including regression trees, support vector machines (SVMs), ensemble trees, Gaussian process regression (GPR), artificial neural networks (ANNs), and kernel-based methods. Among the models tested, GPR, particularly with rational quadratic specifications, consistently demonstrated superior performance with the lowest Root Mean Square Error (RMSE) and highest R-squared values across all datasets. Sensitivity analysis identified age, total pavement thickness, precipitation, temperature, and Annual Average Daily Truck Traffic (AADTT) as key factors influencing the IRI. The results indicate that pavement age and higher traffic loads significantly increase roughness, while thicker pavements contribute to smoother surfaces. Climatic factors such as temperature and precipitation showed varying impacts depending on the regional conditions. The developed models provide a powerful tool for predicting pavement roughness, enabling more accurate maintenance planning and resource allocation. The findings highlight the necessity of tailoring pavement management practices to specific environmental and traffic conditions to enhance road quality and longevity. This research offers a comprehensive framework for understanding and predicting pavement performance, with implications for infrastructure management both locally and worldwide.

Human-AI collaboration improves adults’ oral biomechanical functions: A multi-centre, self-controlled clinical trial

ObjectivesMaintenance of oral muscle functions is important for survival and communication. Utilizing Artificial Intelligence (AI) as a self-health-management material has shown promise. Here we developed a functional and AI-enabled smartphone e-Oral APP that provided real-time feedback features to let humans collaborate with AI, targeting to improve oral biomechanical and speech functions of adults. MethodsA total of 113 neurologically healthy Hong Kong citizens over 50 years old were recruited in this study from June to October 2022. A set of 12 exercises of the tongue, lips, and jaw were conducted with guidance provided in the self-developed e-Oral APP, in which visual-audio demonstrations and immediate feedback on facial movements by an open-source AI-pretrained 68-point Ensemble of Regression Tree (ERT) face landmark detector was given to the users. After 8-week exercises, assessments after 1-week and 4-week post-training were conducted. Primary outcomes were measured on oral functions, including occlusal force, masticatory efficiency, tongue strength, along with dry mouth condition, and oral health-related quality of life (OHQoL), while secondary outcomes were comprised of oral diadochokinesis (DDK) and swallowing ability. The usability of the APP was evaluated by a self-written questionnaire. ResultsAfter exercising a set of 12 exercises on the tongue, lips, and jaw for 8 weeks, 70 out of 113 participants (average age of 67.70±4.93 y.o., 52 female and 18 male) had a significant improvement in the occlusal force (p < 0.001), while masticatory efficiency (p = 0.002), tongue pressure (p < 0.001) and endurance (p = 0.004) were also improved. 80.3 % of respondents rated the APP with an overall rating of 3 out of 5 points, and 68.6 % would recommend the APP to others. ConclusionsAI-based APP can be an effective approach to help healthy adults improve their occlusal force, masticatory efficiency, tongue functions, and oral diadochokinesis after 8-week home oral exercises. Furthermore, these improvements can be sustained for at least four weeks. Clinical significanceThis is the first AI-assisted APP developed for oral muscle training. Our findings demonstrated that a self-administrated human-AI collaboration APP can improve clinically oral muscle biomechanics and functions for healthy adults. AI technology in smartphone gadgets provides a cost-effective, convenient, and reliable means for oral muscles training for adults.

Automated CFRP impact damage detection with statistical thermographic data and machine learning

The study is focused on the use of machine learning models for the automated detection of impact damage in carbon fiber reinforced polymer (CFRP) by flash-pulse thermographic testing. A new method for thermographic data pre-processing, which is based on statistical features, was proposed. Nine machine learning models for the automated detection of impact damage in CFRP samples were applied to the raw thermographic data, data pre-processed by the suggested method and data pre-processed by the widely used thermographic signal reconstruction (TSR) method. The machine learning models were tested to provide a binary classification of impact damage in CFRP. The results presented in this study show improved performance of the classification if the data are pre-processed by the proposed method. The best results were obtained by a Bagged tree ensemble trained with statistical features. The final balanced accuracy achieved for the Bagged trees model trained on 40 statistical features was 99.8 % which indicates a very good performance.

Statistical and machine learning models for predicting spalling in CRCP

Continuously reinforced concrete pavement (CRCP), crucial for the resilience of transportation infrastructure owing to its continuous steel reinforcement, confronts a critical challenge in the form of spalling—a distress phenomenon posing a threat to pavement durability and overall structural integrity. The detachment or breakage of concrete from the surface compromises CRCP’s functionality and raises safety concerns and escalating maintenance costs. To address this pressing issue, our study investigates the multifaceted factors influencing spalling, employing a comprehensive approach that integrates statistical and machine learning techniques for predictive modeling. Descriptive statistics meticulously profile the dataset, emphasizing age, thickness, precipitation, temperature, and traffic parameters. Regression analysis unveils key relationships, emphasizing the significance of age, annual temperature, annual precipitation, maximum humidity, and the initial International Roughness Index (IRI) as influential factors. The correlation matrix heatmap guides feature selection, elucidating intricate interdependencies. Simultaneously, feature importance analysis identifies age, Average Annual Daily Traffic (AADT), and total pavement thickness as crucial contributors to spalling. In machine learning, adopting models, including Gaussian Process Regression and ensemble tree models, is grounded in their diverse capabilities and suitability for the complex task at hand. Their varying predictive accuracies underscore the importance of judicious model selection. This research advances pavement engineering practices by offering nuanced insights into factors influencing spalling in CRCP, refining our understanding of spalling influences. Consequently, the study not only opens avenues for developing improved predictive methodologies but also enhances the durability of CRCP infrastructure, addressing broader implications for informed decision-making in transportation infrastructure management. The selection of Gaussian Process Regression and ensemble tree models stems from their adaptability to capture intricate relationships within the dataset, and their comparative performance provides valuable insights into the diverse predictive capabilities of these models in the context of CRCP spalling.

From ensemble learning to deep ensemble learning: A case study on multi-indicator prediction of pavement performance

Recently, Big data analytics approaches were combined with emerging machine learning techniques, which can provide more sophisticated insights into information-intensive activity. Compared to traditional shallow-architecture machine learning algorithms, Deep learning could excavate more potential information from the raw features. However, its powerful representational capacity relies on the support of enormous samples. The ensemble trees system performs superior on small sample problems due to better generalization capacity. To merge both benefits of deep learning and ensemble tree system, this paper developed a deep ensemble algorithm applied to multiple indicators prediction of pavement performance, including International Roughness Index and pavement 3-layer modulus. The deep ensemble algorithm is developed by merging a deep neural network (DNN) with the decision manifold property of the decision trees (TabNet) into a cascade ensemble system, combined with a sliding window algorithm to extract dependency information from raw data. During the training stage, the Bayesian Optimization Algorithm (BOA) is used to search for the optimal combination of sub-decision makers in the cascade ensemble. And equipped with GPU, it can speed up by 2.6–4.0 times. In the case study of pavement engineering, with sufficient training samples, it can achieve an average accuracy of 98.74 %, higher than DNN (97.49 %) and XGBoost (96.12 %) in predicting pavement indicators. With insufficient training samples, it can achieve an accuracy improvement of 12 % than XGBoost (75 %) and 24.5 % than DNN (62.5 %).