Boosted Tree Method Research Articles

Predicting and analysing air quality in smart cities using virtual sensors

Virtual sensors have become essential in generating synthetic data to support smart city initiatives, mainly when physical sensors are absent or inaccessible. They play a crucial role in monitoring air pollutants, benefiting those with health sensitivities, and aiding city planning in the context of smart cities. This research focuses on creating reliable victual sensors for air pollutant measurement through deep learning and machine learning models, providing a cost-effective solution to address the scarcity of physical sensors and data inaccuracies. The main contribution lies in improving smart cities by providing essential health-related data to mitigate the harmful effects of air pollution on residents. Continuous monitoring empowers officials and residents to reduce pollution exposure, ultimately improving public health. The study found that gradient-boosted trees are effective for daily and hourly predictions, emphasizing the vital role of virtual sensors in advancing data-driven decision-making and improving environmental quality in smart cities. The results showed that our model was better using the boosted trees method with an RMSE of approximately 10%.

A Comparison of LSTM, GRU, and XGBoost for forecasting Morocco's yield curve

The field of time series forecasting has grown significantly over the past several years and is now highly active. In numerous application domains, deep neural networks are exact and powerful. They are among the most popular machine learning techniques for resolving big data issues because of these factors. Historically, there have been numerous methods for accurately predicting the subsequent change in time series data. The time series forecasting problem and its mathematical underpinnings are first articulated in this study. Following that, a description of the most popular deep learning architectures used to date with success in time series forecasting is provided, emphasizing both their benefits and drawbacks. Feedforward networks, recurrent neural networks (such as Elman networks), long- and short-term memory (LSTM), and gated recurrent units (GRU) are given special consideration. Furthermore, the advantages of the XGBoost boosting tree method have shown its superiority in numerous data mining competitions in recent years. The high coefficients of the metric measures indicate that the proposed XGBoost model provides good predictive performance, according to the results.

Data-driven machine learning model of a Selective Catalytic Reduction on Filter (SCRF) in a heavy-duty diesel engine: A comparison of Artificial Neural Network with Tree-based algorithms

The Selective Catalytic Reduction on Filter (SCRF) system is yet to be deployed in current heavy-duty diesel engine aftertreatment system. Due to the thermal, space and cost benefits of the SCRF, it could become a useful component of the after-treatment system of a heavy-duty diesel engine, as regulators continue to demand an even cleaner environment. Ammonia cross-sensitivity of NOx sensors at the post-SCRF location poses challenges in measuring NOx emission accurately at this location and in turn affects the NOx conversion efficiency calculations. Developing a model that could replace the NOx sensor helps to mitigate the ammonia cross-sensitivity challenge as well as provides a medium to measure post-SCRF NOx concentration and NOx conversion efficiency while saving the cost on NOx sensors. This work focuses on a data-driven approach to developing a model for predicting NOx conversion efficiency across the SCRF using Artificial Neural Network, Bootstrap Forest, and Boosted Tree methods. Further, the three modeling techniques were also compared for accuracy and computation cost.

Robust Predictive Model for Different Cancers using Biomarker Proteins

Background: When analyzing multivariate data, it can be challenging to quantify and pinpoint relationships between a collection of consistent characteristics. Reliable computational prediction of cancer patient's response to treatment based on their clinical and molecular profiles is essential in this era of precision medicine. This is essential in helping doctors choose the least contaminated and most potent restorative therapies that are now available. Better patient monitoring and selection are now possible in clinical trials. Methods: This research proposes a novel robust model to aid in the diagnosis of various cancers induced by biomarker proteins (Protein Kinase B, MAPK, and mammalian Target of Rapamycin). Later, various medications (Perifosine, Wortmannin, and Rapamycin) were proposed to cure cancer. Various studies were carried out to obtain all of the results, which aid in the identification of various types of cancer. The drugs mentioned in this essay help to ward off cancer. Scaling and normalization were carried out using parallel coordinates plots and correlation tests, respectively. The boosted tree method and kNN with multiple distance approaches were also used to generate a solid model. The medical diagnosis system was enhanced by training the boosted tree technique to identify various tumors. A robust model was validated by predicting various values that were displayed against the observed value and agreed with the advised strategy to locate biomarkers to show the value of our method. Results: The results show that the predicted and observed values agree with each other, especially for MAPK pathways. The observed correlation coefficient (r2) is 0.9847 without intercept and 0.9849 with intercept. The precise computational prediction of the reaction of cancer patients to treatment based on the patient's clinical and molecular profiles is vital in the period of exactitude medicine. Conclusion: A robust model was validated by predicting the different values that were plotted with the observed value, which agrees with the results of the proposed technique to uncover biomarkers and shows the effectiveness of our technique.

Application of Data Mining Techniques to Predict Luminance of Pavement Aggregate

The primary purpose of the analysis presented here is to assess the feasibility of effectively predicting the aggregate luminance coefficient. Current road lighting standards and recommendations are based on assessing the level and distribution of luminance on the road surface. The brightness of a road surface depends on the amount of light falling on it, as well as the reflective properties of the road surface, which in turn depend on its physical condition, type and mineralogical composition. The complexity of the factors on which the value of the luminance coefficient depends it makes that data mining techniques the most appropriate tools for evaluation luminance coefficient phenomenon. This article uses five types of techniques: C&RT, boosted trees, random forest, neural network, and support vector machines. After a preliminary analysis, it was determined that the most effective technique was the boosted tree method. The results of the analysis indicated that the actual value of the luminance coefficient has multiple modal values within a single aggregate stockpile, depending on the mineralogical composition and grain size, and cannot be determined by a single central measure. The present model allowed us to determine the value of the luminance coefficient Qd with a mean error of 4.3 mcd-m−2·lx−1. In addition, it was found that the best aggregate for pavement brightening that allows high visibility during the day Qd and at night RL is a limestone aggregate. In the group of those that have the ability to potentially brighten the pavement were quartzite and granite aggregates.

Prediction of char yield and nitrogen fixation rate from pyrolysis of sewage sludge based on machine learning

Based on sewage sludge composition and pyrolysis processing conditions, machine learning was employed as a tool to predict the nitrogen fixation rate and char yield of sewage sludge pyrolysis. Multiple linear regression, decision tree, support vector machine, random forest, and gradient boosting tree methods were used in the research to model the collected data, and the model results were further discussed. The required data set is sorted by the existing articles, and the missing values are filled by k-Nearest Neighbors method. Predicting the char yield and nitrogen fixation rate by considering the ultimate and proximate composition of the sewage sludge can give good results. The results showed the gradient boosting tree method is the most accurate in predicting nitrogen fixation rate and char yield, with the coefficient of determination for char yield and nitrogen fixation rate reaching 0.864 and 0.860, respectively. Fusion of existing models by voting or stacking methods revealed promising further improvement in prediction accuracy, but it was limited. In further analysis, using the mean decrease impurity and SHapely Additive exPlanations methods to more accurately evaluate the feature importance. For example, HTT was the most important feature affecting char yield and nitrogen fixation rate, especially when predicting the nitrogen fixation rate, with a feature importance of 0.625. This study also quantified the effects of different features on char yield and nitrogen fixation rate using partial dependence and individual condition expectation plots to provide a reference for the utilization and research of sewage sludge pyrolysis.

Machine vision-based gradient-boosted tree and support vector regression for tool life prediction in turning

One of the essential elements of automated and intelligent machining processes is accurately predicting tool life. It also helps in achieving the goal of producing quality products with reduced production costs. This work proposes a computer vision-based tool wear monitoring and tool life prediction system using machine learning methods. Gradient-boosted trees and support vector machine (SVM) techniques are used to predict tool life. The experimental investigation on the CNC machine is conducted to study the applicability of the proposed tool wear monitoring system. Experiments are performed using workpiece material made of alloy steel and PVD-coated cutting inserts, and flank wear is monitored. An imaging system consisting of an industrial camera, lens, and LED ring light is mounted on the machine to capture tool wear zone images. Images are then processed by algorithms developed in MATLAB®. Boosted tree methods and the SVM methodology have 96% and 97% prediction accuracy, respectively. Validation tests are carried out to determine the accuracy of proposed models. It is observed that the prediction accuracy of boosted three and SVM is good, with a maximum error of 5.89% and 7.56%, respectively. The outcome of the study established that the developed system can monitor the tool wear with good accuracy and can be adopted in industries to optimize the utilization of tool inserts.

Combining discrete wavelet decomposition with soft computing techniques to predict monthly evapotranspiration in semi-arid Hakkâri province, Türkiye.

Accurate prediction of evapotranspiration values is important in planning agricultural irrigation, crop growth research, and hydrological modeling. This study is aimed at estimating monthly evapotranspiration (ET) values in Hakkâri province by combining support vector regression, bagged tree, and boosted tree methods with wavelet transform. For this purpose, precipitation, runoff, surface net solar radiation, air temperatures, and previous ET values were divided into sub-signals with various mother wavelets such as Daubechies 4, Meyer, and Symlet 2 and presented as input to machine learning (ML) algorithms. The study's main contribution to the literature is to reveal which wavelet-based machine learning model, mother wavelet type, and combination of meteorological data show the most realistic results in ET estimation. While establishing the models, the data were divided into 80% training and 20% testing. The models' performances were based on the widely used root mean square error, mean absolute error, determination coefficient, and Taylor diagrams. As a result of the study, it was revealed that the hybrid wavelet ML, which is established with input combinations separated into subcomponents by wavelet transform, generally produces more successful predictions than the stand-alone ML model. In addition, it was revealed that the optimum ET forecasting model was obtained with the wavelet bagged tree algorithm with Symlet 2 mother wavelet. Even though the best model established is based on the precipitation and temperature inputs, it was revealed that past ET, solar radiation, and runoff values are also effective inputs in ET prediction. The results can also be used in other regions of the world with semi-arid climates, such as Hakkâri. The study's outputs provide essential resources to decision-makers and planners to manage water resources and plan agricultural irrigation.

A multi-element non-intrusive Polynomial Chaos method using agglomerative clustering based on the derivatives to study irregular and discontinuous Quantities of Interest

A non-intrusive method to get a multi-element Polynomial Chaos model is developed. This method is called ME-ACD, for Multi-Element based on Agglomerative Clustering on Derivatives. It aims at approximating a Quantity of Interest which presents discontinuities or irregularities making it difficult to be accurately approximated by standard Polynomial Chaos models. The method permits to efficiently split the parameter space and to train local polynomial models of lower degrees on every element where the local pieces of the Quantity of Interest are smoother. The algorithm is based on agglomerative clustering of the observations in a well-chosen abstract space taking into account the value of the Quantity of Interest and of its derivatives with respect to the stochastic input parameters. The same observations are used for both partitioning the space and training the local models. Several partitions of the parameter space are tested, and the one leading to local models minimising a cross-validation error is selected. Once the training observations are labelled with a class number indicating the element they are located in, a neural network classifier is trained to determine which local model to use for further evaluations. The method has proven to efficiently split the parameter space for a set of applications of moderate dimension. The piecewise chaos model is compared with the ones of a standard Polynomial Chaos non-intrusive method and of a Gradient Boosted Trees method in terms of accuracy.

Physics-Guided Long Short-Term Memory Network for Streamflow and Flood Simulations in the Lancang–Mekong River Basin

A warming climate will intensify the water cycle, resulting in an exacerbation of water resources crises and flooding risks in the Lancang–Mekong River Basin (LMRB). The mitigation of these risks requires accurate streamflow and flood simulations. Process-based and data-driven hydrological models are the two major approaches for streamflow simulations, while a hybrid of these two methods promises advantageous prediction accuracy. In this study, we developed a hybrid physics-data (HPD) methodology for streamflow and flood prediction under the physics-guided neural network modeling framework. The HPD methodology leveraged simulation information from a process-based model (i.e., VIC-CaMa-Flood) along with the meteorological forcing information (precipitation, maximum temperature, minimum temperature, and wind speed) to simulate the daily streamflow series and flood events, using a long short-term memory (LSTM) neural network. This HPD methodology outperformed the pure process-based VIC-CaMa-Flood model or the pure observational data driven LSTM model by a large margin, suggesting the usefulness of introducing physical regularization in data-driven modeling, and the necessity of observation-informed bias correction for process-based models. We further developed a gradient boosting tree method to measure the information contribution from the process-based model simulation and the meteorological forcing data in our HPD methodology. The results show that the process-based model simulation contributes about 30% to the HPD outcome, outweighing the information contribution from each of the meteorological forcing variables (<20%). Our HPD methodology inherited the physical mechanisms of the process-based model, and the high predictability capability of the LSTM model, offering a novel way for making use of incomplete physical understanding, and insufficient data, to enhance streamflow and flood predictions.

Probing Diffusive Dynamics of Natural Tubule Nanoclays with Machine Learning

Reproducibility of the experimental results and object of study itself is one of the basic principles in science. But what if the object characterized by technologically important properties is natural and cannot be artificially reproduced one-to-one in the laboratory? The situation becomes even more complicated when we are interested in exploring stochastic properties of a natural system and only a limited set of noisy experimental data is available. In this paper we address these problems by exploring diffusive motion of some natural clays, halloysite and sepiolite, in a liquid environment. By using a combination of dark-field microscopy and machine learning algorithms, a quantitative theoretical characterization of the nanotubes' rotational diffusive dynamics is performed. Scanning the experimental video with the gradient boosting tree method, we can trace time dependence of the diffusion coefficient and probe different regimes of nonequilibrium rotational dynamics that are due to contacts with surfaces and other experimental imperfections. The method we propose is of general nature and can be applied to explore diffusive dynamics of various biological systems in real time.

Creation of mortality risk calculator using a I-123 mIBG-based machine learning model: differential prediction of arrhythmic death and heart-failure death

Abstract Funding Acknowledgements Type of funding sources: None. Background Although I-123 meta-iodobenzylguanidine (mIBG) has been applied to patients with chronic heart failure (CHF), a diagnostic tool for differential prediction of fatal arrhythmic events (ArE) and heart-failure death (HFD) has been pursued. Purpose The aim of this study was to create a calculator of mortality risk for differentiating mode of cardiac death using a machine learning (ML) method, and to test the accuracy in a new cohort of patients with CHF. Methods A total of 529 patients with CHF was used as the training database for ML. The ArE group consisted of patients with arrhythmic death, sudden cardiac death and appropriate therapy by implantable cardioverter defibrillator. A heart-to-mediastinum ratio (H/M) standardized to the medium-energy collimator condition was calculated with a planar anterior mIBG scintigram. The best classifier models for predicting HFD and ArE were determined by four-fold cross validation. Input variables included age, sex, New York Heart Association (NYHA) functional class, left ventricular ejection fraction, ischemic etiology, mIBG H/M and washout rate, and b-type natriuretic peptide (BNP) or NT Pro BNP, estimated glomerular filtration rate, hemoglobin, and complications such as diabetes and hypertension. After creating the ML-based model, the constructed classifier functions for ArE, HFD, and survival were exported for subsequent use. A new cohort of patients (n = 312, age 67 ± 13 years, 2015 or later) was used to test the ML-based model. Results The training database included 141 events (27%) with ArE (7%) and HFD (20%). Receiver-operating characteristic analysis by four-fold validation showed area under the curve value of 0.90 for HFD and 0.73 for ArE. Among various ML methods, the logistic regression method demonstrated the most stable calculation of the probability of ArE followed by random forest and gradient boosted tree methods. Therefore, the logistic-regression method was used for calculating both HFD and ArE probabilities. In the test cohort, patients with a high HFD probability >8% resulted in 6.3-fold higher HFD than those with low probability (≤ 8%). Patients with high ArE probability >8% showed 2.5-fold higher ArE than those with low probability (≤ 8%). Conclusion The ML-based mortality risk calculator could be used for stratifying patients at high and low risks, which might be useful for estimating appropriate treatment strategy.

Analysis of overweight vehicles on asphalt pavement performance using accelerated failure time models

ABSTRACT Vehicle overweighting has been one of the key driving forces of deteriorated pavements. The overweighting characteristics that dominate the pavement performance deterioration have been inadequately investigated. This study filled such a gap by analyzing the survival time of pavement under different overweighting scenarios. The survival model was based on the accelerated failure time (AFT) formulation. For improved predictivity, an implementation of the AFT using the gradient boosting tree method was employed. All data concerned were collected from the Long-Term Pavement Performance (LTPP) program. Fatigue cracking and rutting were selected as the pavement performance indicators. Each indicator had three levels of failure thresholds to investigate the impact of overweighting on short-term, mid-term, and long-term pavement performance. The predictive performance of AFT models showed that long-term pavement performance was more dependent on overweighting than short-term pavement performance. Compared with the volume and weight of overweight vehicles, the overweighting percentage described the overweighting situation the best. Fatigue cracking was more sensitive to the overweighting percentage than rutting. Without overweighting, the average pavement fatigue life and rutting life would be extended by about 43% and 29%, respectively.

Application of data driven machine learning approach for modelling of non-linear filtration through granular porous media

Modeling the relationship between volumetric flux (m3/s) and gradient of hydraulic head (m) is extremely challenging in case of non-linear filtration through porous packing. Due to the uncertainties associated with the definition and quantification of characteristic length and velocity, experimental, theoretical and numerical modelling approaches are not widely applicable. The machine learning algorithms have proven to be extremely useful for predicting similar situations when the physical process is too complex to understand. The performance of Artificial Neural Network (ANN), Random Forest (RF), and Boosted Tree methods have been investigated in the study for predicting the gradient of hydraulic head (target variable) in case of non-linear filtration through porous packing. Velocity of flow, media size, porosity, kinetic viscosity, and shape factor obtained from a wide range of reported data in the literature was used as input features. All three models were observed to predict the output values with significant accuracies (R2>0.90) for wide range of data obtained from different sources. A RF method based sensitivity analysis was performed to study the relative importance of different hydrological parameters over the target variable. These parameters in terms of a decreasing order was ranked as velocity, diameter, viscosity, porosity, and shape factor. These type of models can aid the researchers, planners, and designers to predict the hydraulic gradient or volumetric flux in multiple scenarios associated with non-linear filtration through porous media such as oil and gas exploration wells, water filters, rockfill dams etc.

Spatial prediction of spring locations in data poor region of Central Himalayas

Abstract This research explores the methods for understanding groundwater springs distribution and occurrence using Geographic Information System (GIS) and Machine Learning technique in data poor areas of the Central Himalayas. The objectives of this study are to analyse the distribution of natural springs, evaluate three random forest models for its predictability and establish a model for the prediction of occurrence of springs. This study evaluates the primary causal factors for occurrence of springs. The data used in this study consists of 20 parameters based on topography, geology, lithology, hydrology and land use as causal factors, whereas 621 spring location and discharge (n = 621) measured during 2014–2016 and 815 non-spring locations (generated by GIS tool) use as supporting evidence to train (80%) and test (20%) the prediction model. Results show that the Bootstrap method is comparatively reliable (92% accuracy) over Boosted tree (64% accuracy) and Decision tree (74% accuracy) methods to classify and predict the occurrence of springs in the watershed. Bootstrap Forest shows the high Prediction rate for True Positive (82% actual spring predicted as a spring) and True Negative (89% actual non-spring predicted as non-spring), and the model seems consistent in both responses. This model was then applied to an independent dataset to predict spring location estimates with 75% accuracy. Therefore, spatial statistical methods prove efficient at predicting spring occurrence in data poor regions.

Klasifikasi Data Aging Tunggakan Nasabah Menggunakan Metode Decision Tree Pada ULaMM Unit Kolaka

This study aims to classify aging of loan data using the decision tree method based on plafond, outstanding principal, and the amount of loan. The subjects in this study were the debtor of ULaMM (Unit Layanan Modal Mikro), unit of Kolaka, PT. PNM (Persero) Kendari Branch. The number of samples used is 100 data debtors. Based on the results of the research conducted, it was found that the classification analysis using the Decision Tree has an accuracy rate of 95.00%, while the classification analysis using the Gradient Boosted Tree has an accuracy level of 90.00%. From the results of the analysis that has been done, it can be concluded that for the data in this study, the classification method using the Decision Tree is better than the Gradient Boosted Tree method.

Predicting severe clinical events by learning about life-saving actions and outcomes using distant supervision.

Medical error is a leading cause of patient death in the United States. Among the different types of medical errors, harm to patients caused by doctors missing early signs of deterioration is especially challenging to address due to the heterogeneity of patients' physiological patterns. In this study, we implemented risk prediction models using the gradient boosted tree method to derive risk estimates for acute onset diseases in the near future. The prediction model uses physiological variables as input signals and the time of the administration of outcome-related interventions and discharge diagnoses as labels. We examine four categories of acute onset illness: acute heart failure (AHF), acute lung injury (ALI), acute kidney injury (AKI), and acute liver failure (ALF). To develop and test the model, we consider data from two sources: 23,578 admissions to the Intensive Care Unit (ICU) from the MIMIC-3 dataset (Beth-Israel Hospital) and 16,612 ICU admissions on hospitals affiliated with our institution (University of Washington Medical Center and Harborview Medical Center, the UW-CDR dataset). We systematically identify outcome-related interventions for each acute organ failure, then use them, along with discharge diagnoses, to label proxy events to train gradient boosted trees. The trained models achieve the highest F1 score with a value of 0.6018 when predicting the need for life-saving interventions for ALI within the next 24h in the MIMIC-3 dataset while showing a median F1 score of 0.3850 from all acute organ failures in both datasets. The approach also achieves the highest F1 score of 0.6301 when classifying a patient's ALI status at the time of discharge from the MIMIC-3 dataset, with a median F1 score of 0.4307 in both datasets. This study shows the potential for using the time of outcome-related intervention administrations and discharge diagnoses as labels to train supervised machine learning models that predict the risk of acute onset illnesses.

Soft Sensor Application in Identification of the Activated Sludge Bulking Considering the Technological and Economical Aspects of Smart Systems Functioning.

The paper presented the methodology for the construction of a soft sensor used for activated sludge bulking identification. Devising such solutions fits within the current trends and development of a smart system and infrastructure within smart cities. In order to optimize the selection of the data-mining method depending on the data collected within a wastewater treatment plant (WWTP), a number of methods were considered, including: artificial neural networks, support vector machines, random forests, boosted trees, and logistic regression. The analysis conducted sought the combinations of independent variables for which the devised soft sensor is characterized with high accuracy and at a relatively low cost of determination. With the measurement results pertaining to the quantity and quality of wastewater as well as the temperature in the activated sludge chambers, a good fit can be achieved with the boosted trees method. In order to simplify the selection of an optimal method for the identification of activated sludge bulking depending on the model requirements and the data collected within the WWTP, an original system of weight estimation was proposed, enabling a reduction in the number of independent variables in a model—quantity and quality of wastewater, operational parameters, and the cost of conducting measurements.

EEG Sleep Detection Algorithms a Comparative Study

In this paper it represents a comparison between some machine learning algorithm which is applied to solve the sleep monitoring issues. Sleep detection requires electroencephalogram signal for differentiating purpose. There are some methods & models that are already perused & built with training & testing datasets like- single layer perception, multilayer perception, SVM (support vector machine), boosted tree method. The difference between these models is measured using the Cohen’s index, the true positive & the true negative rate. Cross-validation technique is usually used to weigh the results of the models of monitoring sleep. The models successfully monitors sleep state reaching up to 94% and Cohen’s index successfully reaching up to 0.69. The success rate shows the considerable assurance for future expansion & practices.

Probabilistic deep neural network price forecasting based on residential load and wind speed predictions

Precise price forecasting can lessen the risk of participation in the deregulated electricity market. On account of a large amount of historical data, deep learning based methods can be a promising solution to achieve an accurate forecast. This study presents a deep neural network algorithm to estimate the probability density function of price, incorporating the prediction of wind speed and residential load as two other high volatile parameters. To this end, first, a combination of convolution neural network (CNN) and gated recurrent unit (GRU) is utilised to predict wind speed and residential load. Then, the results are integrated into historical price information to form the input dataset for price forecasting. The proposed price forecast procedure consists of CNN, GRU, and adaptive kernel density estimator (AKDE). AKDE is used as a numerical algorithm to capture probabilistic characterisation of real-time and day-ahead prices. Several deep and shallow networks and the proposed algorithm are implemented, and the results are compared. Furthermore, the effectiveness of AKDE in providing complete statistical information is verified through comparison with conventional and fixed smooth KDEs. In addition, the gradient boosting tree method is incorporated to verify the dependence of the price to the wind and the residential loads.