Tree-based Machine Learning Techniques Research Articles

On the pricing of capped volatility swaps using machine learning techniques

A capped volatility swap is a forward contract on an asset's capped, annualized, realized volatility, over a predetermined period of time. This paper presents data-driven machine learning techniques for pricing such capped volatility swaps, using unique data sets comprising both the strike price of contracts at initiation and the daily observed prices of running contracts. Additionally, the developed model can serve as a validation tool for external volatility swap prices, flagging prices that deviate significantly from the estimated value. In order to predict the capped, future, realized volatility, we explore distributional information on the underlying asset, specifically by extracting information from the implied volatilities and market-implied moments of the asset. The pricing performance of tree-based machine learning techniques and a Gaussian process regression model is evaluated in a validation setting tailored to the use of financial data.

A New Practical Formula for Bond-Slip Stress of Cold-Formed CFSTs Stub-Columns in a Post-fire Based on Four Cycles of Push-Out Tests Using Decision Tree-Based Machine Learning Techniques

A New Practical Formula for Bond-Slip Stress of Cold-Formed CFSTs Stub-Columns in a Post-fire Based on Four Cycles of Push-Out Tests Using Decision Tree-Based Machine Learning Techniques

Use of machine learning techniques for identifying ischemic stroke instead of the rule-based methods: a nationwide population-based study

BackgroundMany studies have evaluated stroke using claims data; most of these studies have defined ischemic stroke using an operational definition following the rule-based method. Rule-based methods tend to overestimate the number of patients with ischemic stroke.ObjectivesWe aimed to identify an appropriate algorithm for identifying stroke by applying machine learning (ML) techniques to analyze the claims data.MethodsWe obtained the data from the Korean National Health Insurance Service database, which is linked to the Ilsan Hospital database (n = 30,897). The performance of prediction models (extreme gradient boosting [XGBoost] or gated recurrent unit [GRU]) was evaluated using the area under the receiver operating characteristic curve (AUROC), the area under precision–recall curve (AUPRC), and calibration curve.ResultsIn total, 30,897 patients were enrolled in this study, 3145 of whom (10.18%) had ischemic stroke. XGBoost, a tree-based ML technique, had the AUROC was 94.46% and AUPRC was 92.80%. GRU showed the highest accuracy (99.81%), precision (99.92%) and recall (99.69%).ConclusionsWe proposed recurrent neural network-based deep learning techniques to improve stroke phenotyping. This can be expected to produce rapid and more accurate results than the rule-based methods.

A Global Forecasting Approach to Large-Scale Crop Production Prediction with Time Series Transformers

Accurate prediction of crop production is essential in effectively managing the food security and economic resilience of agricultural countries. This study evaluates the performance of statistical and machine learning-based methods for large-scale crop production forecasting. We predict the quarterly production of 325 crops (including fruits, vegetables, cereals, non-food, and industrial crops) across 83 provinces in the Philippines. Using a comprehensive dataset of 10,949 time series over 13 years, we demonstrate that a global forecasting approach using a state-of-the-art deep learning architecture, the transformer, significantly outperforms popular tree-based machine learning techniques and traditional local forecasting approaches built on statistical and baseline methods. Our results show a significant 84.93%, 80.69%, and 79.54% improvement in normalized root mean squared error (NRMSE), normalized deviation (ND), and modified symmetric mean absolute percentage error (msMAPE), respectively, over the next-best methods. By leveraging cross-series information, our proposed method is scalable and works well even with time series that are short, sparse, intermittent, or exhibit structural breaks/regime shifts. The results of this study further advance the field of applied forecasting in agricultural production and provide a practical and effective decision-support tool for policymakers that oversee crop production and the agriculture sector on a national scale.

Tree-Based Machine Learning Techniques for Automated Human Sleep Stage Classification

Tree-Based Machine Learning Techniques for Automated Human Sleep Stage Classification

Prediction and exploration of contributory factors related to fatigue driving among long-haul truck drivers traveling across India: A tree-based machine learning approach

IntroductionLong-haul Truck drivers (LHTDs) have long working hours, insufficient rest, and poor health conditions and often experience fatigue that substantially may lead to crashes and injuries. Despite its potential harmfulness, we have little understanding on non-linear hidden patterns of influential factors on fatigue driving, especially in developing nations like India. ObjectivesThis paper aimed to predict fatigue driving among LHTDs using four tree-based machine learning techniques including Decision tree (DT), Random Forest (RF), Adaptive boosting (AdaBoost), and Extreme gradient boosting (XGBoost) to analyze the non-linear hidden pattern of most influential variables contributing to fatigue driving among Indian LHTDS. MethodsUsing a cross-sectional study design, a face-to-face interview was conducted among LHTDs using carefully designed questionnaire in Salem, Tamil Nadu, India. A total of 756 responses were obtained from LHTDs based on four aspects including socio-economic characteristics, work and vehicle characteristics, health-related lifestyle characteristics, and fatigue-inducing characteristics using a convenience sampling technique. Four tree-based machine learning algorithms namely DT, RF, Adaboost, and Xgboost were employed to predict fatigue driving. The influence of predictors on fatigue driving for the most suitable model was determined through variable importance plot and their causality effect on probability of fatigue were examined using Partial dependency plot (PDP). All the analysis were carried out using IBM SPSS statistics Version 27.0 and R programming language version 4.2.1. ResultsFrom the analysis, it was found that RF model outperformed other investigated classifiers (Accuracy = 81.2%, F1 score = 58.82%, AUROC = 0.854). Furthermore, variable importance plot of befitting RF classifier showed that type of commodity carried, pressured delivery of goods, countermeasure mostly followed, and education level of the LHTD as some of most influential predictors causing fatigue. ConclusionThese findings provide insights to state and highway transportation officials and Indian trucking industries for framing effective strategies to promote safety and well-being among LHTDs.

Tree-Based Machine Learning Models with Optuna in Predicting Impedance Values for Circuit Analysis.

The transmission characteristics of the printed circuit board (PCB) ensure signal integrity and support the entire circuit system, with impedance matching being critical in the design of high-speed PCB circuits. Because the factors affecting impedance are closely related to the PCB production process, circuit designers and manufacturers must work together to adjust the target impedance to maintain signal integrity. Five machine learning models, including decision tree (DT), random forest (RF), extreme gradient boosting (XGBoost), categorical boosting (CatBoost), and light gradient boosting machine (LightGBM), were used to forecast target impedance values. Furthermore, the Optuna algorithm is used to determine forecasting model hyperparameters. This study applied tree-based machine learning techniques with Optuna to predict impedance. The results revealed that five tree-based machine learning models with Optuna can generate satisfying forecasting accuracy in terms of three measurements, including mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination (R2). Meanwhile, the LightGBM model with Optuna outperformed the other models. In addition, by using Optuna to tune the parameters of machine learning models, the accuracy of impedance matching can be increased. Thus, the results of this study suggest that the tree-based machine learning techniques with Optuna are a viable and promising alternative for predicting impedance values for circuit analysis.

Artificial Intelligence and Machine Learning in Prediction of Total Hip Arthroplasty Outcome: A Bibliographic Review

This study investigates the current research trends on the adoption of artificial intelligence and machine learning techniques to predict the outcome of total hip arthroplasty (THA) or total hip replacement (THR) procedure using bibliometric analysis. A total of 102 publications from articles, review, and conference papers were included. The study analysed the network of authors, keywords, citations, and collaboration between authors on the application of artificial intelligence and machine learning to predict the outcome of THA. Regression-based and tree-based machine learning techniques were utilized in the majority of research because they are simpler to comprehend when there are elements involved in the prediction of results. All models had moderate to excellent (AUROC values from 0.71 to 0.97) discrimination ability in making the prediction.

2018 Kerala Flood Damage: Survey, Identification, and Damage Prediction Models Using Machine Learning

The 2018 flood experienced by Kerala, the southernmost state of India, has affected thousands of lives, resulting in heavy financial loss (248,930 million INR) and structural damage (129,420 residential buildings have been reported to be damaged). Depth versus damage relationships have been used conventionally for flood emergency decision-making; however, these are approximate and do not account for all the significant factors involved in the flood damage assessment. An extensive questionnaire survey was conducted in this study to collect data comprising significant parameters of flood-damaged residential buildings due to the 2018 Kerala flood event. The damage to the buildings was expressed in terms of damage ratios, and the buildings were classified into various damage states according to their damage ratios. Machine learning (ML) techniques, such as random forest, naïve Bayes, decision tree, K-nearest neighbors, AdaBoost, XGBoost, support vector machine, LightGBM, and CatBoost, were used in this study to develop models to predict and classify the buildings into various flood damage states. Results indicated that almost all the tree-based ML techniques performed well, and the random forest model was obtained as the best-performing classification model for flood damage prediction with 84% accuracy. Further SHAP (SHapely Additive exPlanations) analysis was conducted to explore the relative importance of the different parameters involved in damage prediction.

Tree Based Fault Classification in Underground Cable

This paper presents tree-based machine learning techniques like decision trees, and random forest algorithms to categorize various permanent faults such as a line to ground, line to line, line-line to ground, and line-line-line to ground fault simulated in an underground cable. The simulation has been performed with the variation in fault types, fault location, fault resistance, and fault inception angle and the sending end current signals are presented as input to the classifier for classifying the fault. A relative comparison between the decision tree technique and random forest technique has been performed and the random forest algorithm turns out to be the best for classifying the permanent faults in an underground cable. Both the algorithms are deliberated by precision, F1 score, and recall. The best result is presented by Random Forest Algorithm with an accuracy of 98.8%.

A Fuzzy Random Survival Forest for Predicting Lapses in Insurance Portfolios Containing Imprecise Data

We propose a fuzzy random survival forest (FRSF) to model lapse rates in a life insurance portfolio containing imprecise or incomplete data such as missing, outlier, or noisy values. Following the random forest methodology, the FRSF is proposed as a new machine learning technique for solving time-to-event data using an ensemble of multiple fuzzy survival trees. In the learning process, the combination of methods such as the c-index, fuzzy sets theory, and the ensemble of multiple trees enable the automatic handling of imprecise data. We analyse the results of several experiments and test them statistically; they show the FRSF’s robustness, verifying that its generalisation capacity is not reduced when modelling imprecise data. Furthermore, the results obtained using a real portfolio of a life insurance company demonstrate that the FRSF has a better performance in comparison with other state-of-the-art algorithms such as the traditional Cox model and other tree-based machine learning techniques such as the random survival forest.

Intelligent transportation system for internet of vehicles based vehicular networks for smart cities

A popular research area is internet traffic analysis as it has many applications, mainly for classifying internet traffic. Innovative technologies have been developed for predicting and identifying traffic congestion in the intelligent Internet of vehicles (IOVs). In this paper, an intelligent transport system for the IOVs-based vehicular network traffic for smart city scenario is proposed based on tree-based Decision Tree (DT), Random Forest (RF), and Extra Tree (ET), and XGBoost machine learning (ML) models. Simulation results indicate that the proposed system can provide high detection accuracy and low computational costs thanks to ensemble learning and averaging important feature selection. The tree-based ML techniques with feature selection performed better than those without feature selection for IOV-based vehicular network traffic. The Stacking model shows higher classification accuracy, 99.05%, compared to the lowest KNN accuracy, 96.6%, and SVM accuracy, 98.01%.

Evaluation of Machine Learning Techniques for Traffic Flow-Based Intrusion Detection

Cybersecurity is one of the great challenges of today's world. Rapid technological development has allowed society to prosper and improve the quality of life and the world is more dependent on new technologies. Managing security risks quickly and effectively, preventing, identifying, or mitigating them is a great challenge. The appearance of new attacks, and with more frequency, requires a constant update of threat detection methods. Traditional signature-based techniques are effective for known attacks, but they are not able to detect a new attack. For this reason, intrusion detection systems (IDS) that apply machine learning (ML) techniques represent an alternative that is gaining importance today. In this work, we have analyzed different machine learning techniques to determine which ones permit to obtain the best traffic classification results based on classification performance measurements and execution times, which is decisive for further real-time deployments. The CICIDS2017 dataset was selected in this work since it contains bidirectional traffic flows (derived from traffic captures) that include benign traffic and different types of up-to-date attacks. Each traffic flow is characterized by a set of connection-related attributes that can be used to model the traffic and distinguish between attacks and normal flows. The CICIDS2017 also contains the raw network traffic captures collected during the dataset creation in a packet-based format, thus permitting to extract the traffic flows from them. Various classification techniques have been evaluated using the Weka software: naive Bayes, logistic, multilayer perceptron, sequential minimal optimization, k-nearest neighbors, adaptive boosting, OneR, J48, PART, and random forest. As a general result, methods based on decision trees (PART, J48, and random forest) have turned out to be the most efficient with F1 values above 0.999 (average obtained in the complete dataset). Moreover, multiclass classification (distinguishing between different types of attack) and binary classification (distinguishing only between normal traffic and attack) have been compared, and the effect of reducing the number of attributes using the correlation-based feature selection (CFS) technique has been evaluated. By reducing the complexity in binary classification, better results can be obtained, and by selecting a reduced set of the most relevant attributes, less time is required (above 30% of decrease in the time required to test the model) at the cost of a small performance loss. The tree-based techniques with CFS attribute selection (six attributes selected) reached F1 values above 0.990 in the complete dataset. Finally, a conventional tool like Zeek has been used to process the raw traffic captures to identify the traffic flows and to obtain a reduced set of attributes from these flows. The classification results obtained using tree-based techniques (with 14 Zeek-based attributes) were also very high, with F1 above 0.997 (average obtained in the complete dataset) and low execution times (allowing several hundred thousand flows/s to be processed). These classification results obtained on the CICIDS2017 dataset allow us to affirm that the tree-based machine learning techniques may be appropriate in the flow-based intrusion detection problem and that algorithms, such as PART or J48, may offer a faster alternative solution to the RF technique.

아파트 대량 평가를 위한 기계 학습 알고리즘과 스태킹 앙상블 기법의 응용

Purpose The goal is to find an optimal data-driven model for apartment price appraisal system. We compares the predictive power of several machine learning algorithms and the ensemble model by using 17,779 apartment transaction data in Gangnam from 2006 to 2017. Methods We use three tree-based machine learning techniques, random forest, Adaboost, and Catboost. Also, an ensemble model is developed through the stacking technique using the above algorithm. 5-fold cross validation is used as model validation techniques. Results By applying the algorithm, it was found that the ensemble model constructed from stacking method performs better than individual machine learning models. Conclusion The results illustrate that the quality of mass appraisals or apartment price indices can be improved by using the ensemble model.

Machine learning for recovery factor estimation of an oil reservoir: A tool for derisking at a hydrocarbon asset evaluation

Well-known oil recovery factor estimation techniques such as analogy, volumetric calculations, material balance, decline curve analysis, hydrodynamic simulations have certain limitations. Those techniques are time-consuming, and require specific data and expert knowledge. Besides, though uncertainty estimation is highly desirable for this problem, the methods above do not include this by default. In this work, we present a data-driven technique for oil recovery factor (limited to water flooding) estimation using reservoir parameters and representative statistics. We apply advanced machine learning methods to historical worldwide oilfields datasets (more than 2000 oil reservoirs). The data-driven model might be used as a general tool for rapid and completely objective estimation of the oil recovery factor. In addition, it includes the ability to work with partial input data and to estimate the prediction interval of the oil recovery factor. We perform the evaluation in terms of accuracy and prediction intervals coverage for several tree-based machine learning techniques in application to the following two cases: (1) using parameters only related to geometry, geology, transport, storage and fluid properties, (2) using an extended set of parameters including development and production data. For both cases, the model proved itself to be robust and reliable. We conclude that the proposed data-driven approach overcomes several limitations of the traditional methods and is suitable for rapid, reliable and objective estimation of oil recovery factor for hydrocarbon reservoir.

Applications of Decision Tree and Random Forest as Tree-Based Machine Learning Techniques for Analyzing the Ultimate Strain of Spliced and Non-Spliced Reinforcement Bars

The performance of both non-spliced and spliced steel bars significantly affects the overall performance of structural reinforced concrete elements. In this context, the mechanical properties of reinforcement bars (i.e., their ultimate strength and strain) should be determined in order to evaluate their reliability prior to the construction procedure. In this study, the application of Tree-Based machine learning techniques is implemented to analyze the ultimate strain of non-spliced and spliced steel reinforcements. In this regard, a database containing the results of 225 experimental tests was collected based on the research investigations available in peer-reviewed international publications. The database included the mechanical properties of both non-spliced and mechanically spliced bars. For better accuracy, the databases of other splicing methods such as lap and welded-spliced methods were excluded from this research. The database was categorized as two sub-databases: training (85%) and testing (15%) of the developed models. Various effective parameters such as splice technique, steel grade of the bar, diameter of the steel bar, coupler geometry—including length and outer diameter along with the testing temperatures—were defined as the input variables for analyzing the ultimate strain using tree-based approaches including Decision Trees and Random Forest. The predicted outcomes were compared to the actual values and the precision of the prediction models was assessed via performance metrics, along with a Taylor diagram. Based on the reported results, the reliability of the proposed ML-based methods was acceptable (with an R2 ≥ 85%) and they were time-saving and cost-effective compared to more complicated, time-consuming, and expensive experimental examinations. More importantly, the models proposed in this study can be further considered as a part of a comprehensive prediction model for estimating the stress-strain behavior of steel bars.

Forecasting Global Maize Prices From Regional Productions

This study analyses the quality of six regression algorithms in forecasting the monthly price of maize in its primary international trading market, using publicly available data of agricultural production at a regional scale. The forecasting process is done between one and twelve months ahead, using six different forecasting techniques. Three (CART, RF, and GBM) are tree-based machine learning techniques that capture the relative influence of maize-producing regions on global maize price variations. Additionally, we consider two types of linear models—standard multiple linear regression and vector autoregressive (VAR) model. Finally, TBATS serves as an advanced time-series model that holds the advantages of several commonly used time-series algorithms. The predictive capabilities of these six methods are compared by cross-validation. We find RF and GBM have superior forecasting abilities relative to the linear models. At the same time, TBATS is more accurate for short time forecasts when the time horizon is shorter than three months. On top of that, all models are trained to assess the marginal contribution of each producing region to the most extreme price shocks that occurred through the past 60 years of data in both positive and negative directions, using Shapley decompositions. Our results reveal a strong influence of North-American yield variation on the global price, except for the last months preceding the new-crop season.

Exploring the additional value of class imbalance distributions on interpretable flash flood susceptibility prediction in the Black Warrior River basin, Alabama, United States

This study proposes a novel flash flood susceptibility prediction framework with a particular emphasis on the extent of imbalance between the number of flooding and non-flooding events as majority of the events result in non-flooding. The class imbalance issue and the magnitude of the imbalance was explored in this study to highlight the uncertain nature of the flooding phenomenon. Therefore, the Random Forest (RF) was initially adopted to evaluate five imbalance class distribution scenarios (i.e., 1x, 10x, 25x, 50x, 100x non-flood events, for each x flood event). Parameter configurations of developed models were determined with the state-of-the-art metaheuristic, the Cuckoo Search (CS) algorithm. The CS-RF model showed the highest (0.8455) prediction capability with regards to the area under the receiver operating characteristic (AUROC) once the extent of imbalance was set as 50x. The CS-RF model was then benchmarked with another bagging, i.e., Extra Trees, and two boosting, i.e., Adaptive Boosting (Adaboost) and eXtreme Gradient Boosting (XGBoost) algorithms, all integrated with the CS technique. Analysis results showed that the CS-RF is the most promising tree-based machine learning technique in flash flood susceptibility projection for the selected study area. According to the predictions, a flash flood susceptibility map was generated, where 9.35% of the basin was under very high flash flood risk. A recently developed model-agnostic game-theoretical method, SHapley Additive exPlanations (SHAP), was used for anatomizing the flash flood conditioning factors to highlight the contribution of each feature on the incident outcome prediction ensuring the transparency of the model findings. Overall, this study contributes to both theory and practice with particular focus on the model interpretability and existence of imbalance in the occurrence of flash flood events, assisting decision-makers in enhancing strategies to combat hazardous impacts of floods.

Tree-based machine learning performed in-memory with memristive analog CAM

Tree-based machine learning techniques, such as Decision Trees and Random Forests, are top performers in several domains as they do well with limited training datasets and offer improved interpretability compared to Deep Neural Networks (DNN). However, these models are difficult to optimize for fast inference at scale without accuracy loss in von Neumann architectures due to non-uniform memory access patterns. Recently, we proposed a novel analog content addressable memory (CAM) based on emerging memristor devices for fast look-up table operations. Here, we propose for the first time to use the analog CAM as an in-memory computational primitive to accelerate tree-based model inference. We demonstrate an efficient mapping algorithm leveraging the new analog CAM capabilities such that each root to leaf path of a Decision Tree is programmed into a row. This new in-memory compute concept for enables few-cycle model inference, dramatically increasing 103 × the throughput over conventional approaches.

Designing a Network Intrusion Detection System Based on Machine Learning for Software Defined Networks

Software-defined Networking (SDN) has recently developed and been put forward as a promising and encouraging solution for future internet architecture. Managed, the centralized and controlled network has become more flexible and visible using SDN. On the other hand, these advantages bring us a more vulnerable environment and dangerous threats, causing network breakdowns, systems paralysis, online banking frauds and robberies. These issues have a significantly destructive impact on organizations, companies or even economies. Accuracy, high performance and real-time systems are essential to achieve this goal successfully. Extending intelligent machine learning algorithms in a network intrusion detection system (NIDS) through a software-defined network (SDN) has attracted considerable attention in the last decade. Big data availability, the diversity of data analysis techniques, and the massive improvement in the machine learning algorithms enable the building of an effective, reliable and dependable system for detecting different types of attacks that frequently target networks. This study demonstrates the use of machine learning algorithms for traffic monitoring to detect malicious behavior in the network as part of NIDS in the SDN controller. Different classical and advanced tree-based machine learning techniques, Decision Tree, Random Forest and XGBoost are chosen to demonstrate attack detection. The NSL-KDD dataset is used for training and testing the proposed methods; it is considered a benchmarking dataset for several state-of-the-art approaches in NIDS. Several advanced preprocessing techniques are performed on the dataset in order to extract the best form of the data, which produces outstanding results compared to other systems. Using just five out of 41 features of NSL-KDD, a multi-class classification task is conducted by detecting whether there is an attack and classifying the type of attack (DDoS, PROBE, R2L, and U2R), accomplishing an accuracy of 95.95%.