Gradient Boosting Tree Algorithm Research Articles

Building safety monitoring based on extreme gradient boosting in distributed optical fiber sensing

Abstract Aiming at the problem of the safety monitoring of the surrounding environment of the glass window on the building, a scheme of distributed optical fiber acoustic sensors based on the phase sensitive time-domain reflection is proposed. Firstly, Wigner bispectrum is used to analyze the time–frequency energy distribution of vibration signal, then the axially integrated bispectrum algorithm is used to extract the time–frequency characteristic components, and the method of sub-band division is used to avoid information loss and decrease the cost of computing. The eigenvectors of different vibration events are recognized by extreme gradient boosting tree algorithm. Based on the diversity of acoustic signal types in the environment, there are eight kinds of events including wind blowing, knocking, window opening, watering, hammering, dog barking, aircraft sound and no disturbance to be selected for vibration experiments. The experimental results show that the scheme can get better recognition effect, and the average recognition rate of eight kinds of events reaches 93.3%.

A Model Combining Seq2Seq Network and LightGBM Algorithm for Industrial Soft Sensor

As a key technology for industry 4.0, data-driven soft sensing plays an important role in the control and optimization of industrial processes. However, due to the large-scale, nonlinear and dynamic characteristics of industrial data, it is difficult to process industrial data. To solve these difficulties, a soft sensor modeling method based on a sequence to sequence model and a gradient boosting tree algorithm is developed. In this method, an unsupervised trained Seq2Seq model is used to extract dynamic features at first. Then a high-precision model based on LightGBM is constructed with the dynamic features and the original features as inputs. The developed method is validated on pulping data and compared with other machine learning methods such as RNN and SVR. The result shows the developed method has a better performance.

Comparative analysis of prediction algorithms for building energy usage prediction at an urban scale

Strategic planning for efficient and sustainable urban environments necessitates identification of scalable energy saving opportunities for the buildings sector. A possible resolution is the analysis of building energy use data at urban scale, although the available data is often sparse, inconsistent, diverse and heterogeneous in nature. Over the past decades, predictive modeling using sparse data has aided with the forecasting of building energy use. However, most studies of energy use prediction focus on individual buildings. This paper proposes the integration of building archetypes simulation, parametric analysis, and machine learning techniques as a solution to accurately predict individual building energy use at an urban level. The aim of the research described in this paper is to achieve accurate prediction of building energy performance, which will allow stakeholders, such as energy policymakers and urban planners, to make informed decisions when planning retrofit measures at large scale. The methodology generates synthetic building data for training the predictive model and predicts building energy use at urban scale with limited resources. The experimentation focuses on Dublin city through the development of synthetic building dataset using parametric analysis on previously identified key variables of two distinct building archetypes. Having compared different prediction algorithms, we show that the Gradient Boosted Trees algorithm gives a better prediction when compared to other algorithms.

Utilising artificial intelligence to determine patients at risk of a rare disease: idiopathic pulmonary arterial hypertension.

Idiopathic pulmonary arterial hypertension is a rare and life-shortening condition often diagnosed at an advanced stage. Despite increased awareness, the delay to diagnosis remains unchanged. This study explores whether a predictive model based on healthcare resource utilisation can be used to screen large populations to identify patients at high risk of idiopathic pulmonary arterial hypertension. Hospital Episode Statistics from the National Health Service in England, providing close to full national coverage, were used as a measure of healthcare resource utilisation. Data for patients with idiopathic pulmonary arterial hypertension from the National Pulmonary Hypertension Service in Sheffield were linked to pre-diagnosis Hospital Episode Statistics records. A non-idiopathic pulmonary arterial hypertension control cohort was selected from the Hospital Episode Statistics population. Patient history was limited to ≤5 years pre-diagnosis. Information on demographics, timing/frequency of diagnoses, medical specialities visited and procedures undertaken was captured. For modelling, a bagged gradient boosting trees algorithm was used to discriminate between cohorts. Between 2008 and 2016, 709 patients with idiopathic pulmonary arterial hypertension were identified and compared with a stratified cohort of 2,812,458 patients classified as non-idiopathic pulmonary arterial hypertension with ≥1 ICD-10 coded diagnosis of relevance to idiopathic pulmonary arterial hypertension. A predictive model was developed and validated using cross-validation. The timing and frequency of the clinical speciality seen, secondary diagnoses and age were key variables driving the algorithm's performance. To identify the 100 patients at highest risk of idiopathic pulmonary arterial hypertension, 969 patients would need to be screened with a specificity of 99.99% and sensitivity of 14.10% based on a prevalence of 5.5/million. The positive predictive and negative predictive values were 10.32% and 99.99%, respectively. This study highlights the potential application of artificial intelligence to readily available real-world data to screen for rare diseases such as idiopathic pulmonary arterial hypertension. This algorithm could provide low-cost screening at a population level, facilitating earlier diagnosis, improved diagnostic rates and patient outcomes. Studies to further validate this approach are warranted.

Classification of Postprandial Glycemic Status with Application to Insulin Dosing in Type 1 Diabetes-An In Silico Proof-of-Concept.

In the daily management of type 1 diabetes (T1D), determining the correct insulin dose to be injected at meal-time is fundamental to achieve optimal glycemic control. Wearable sensors, such as continuous glucose monitoring (CGM) devices, are instrumental to achieve this purpose. In this paper, we show how CGM data, together with commonly recorded inputs (carbohydrate intake and bolus insulin), can be used to develop an algorithm that allows classifying, at meal-time, the post-prandial glycemic status (i.e., blood glucose concentration being too low, too high, or within target range). Such an outcome can then be used to improve the efficacy of insulin therapy by reducing or increasing the corresponding meal bolus dose. A state-of-the-art T1D simulation environment, including intraday variability and a behavioral model, was used to generate a rich in silico dataset corresponding to 100 subjects over a two-month scenario. Then, an extreme gradient-boosted tree (XGB) algorithm was employed to classify the post-prandial glycemic status. Finally, we demonstrate how the XGB algorithm outcome can be exploited to improve glycemic control in T1D through real-time adjustment of the meal insulin bolus. The proposed XGB algorithm obtained good accuracy at classifying post-prandial glycemic status (AUROC = 0.84 [0.78, 0.87]). Consequently, when used to adjust, in real-time, meal insulin boluses obtained with a bolus calculator, the proposed approach improves glycemic control when compared to the baseline bolus calculator. In particular, percentage time in target [70, 180] mg/dL was improved from 61.98 (±13.89) to 67.00 (±11.54; p < 0.01) without increasing hypoglycemia.

Estimation of Interfacial Tension for Geological CO2 Storage

AbstractThe effective CO2 sequestration in saline aquifers as a climate change‐lessening solution is mainly governed by the interfacial tension (IFT) behavior between CO2 and brine. An innovative and competent decision tree‐based approach called stochastic gradient boosting (SGB) tree algorithm was applied to predict the CO2‐aquifer brine IFT as a function of temperature, pressure, and brine salinities. The produced results were compared with the previously reported outcomes of other machine learning models, namely, radial basis function networks, multilayer perceptron networks, least squares support vector machine, and adaptive neuro fuzzy inference system. Amongst all models, the developed SGB tree algorithm provided superior outputs and turned out to be the most accurate tool.

Leaves on trees: identifying halo stars with extreme gradient boosted trees

Context. Extended stellar haloes are a natural by-product of the hierarchical formation of massive galaxies like the Milky Way. If merging is a non-negligible factor in the growth of our Galaxy, evidence of such events should be encoded in its stellar halo. The reliable identification of genuine halo stars is a challenging task, however. Aims. With the advent of the Gaia space telescope, we are ushered into a new era of Galactic astronomy. The first Gaia data release contains the positions, parallaxes, and proper motions for over two million stars, mostly in the solar neighbourhood. The second Gaia data release will enlarge this sample to over 1.5 billion stars, the brightest ~ 5 million of which will have full phase-space information. Our aim for this paper is to develop a machine learning model for reliably identifying halo stars, even when their full phase-space information is not available. Methods. We use the Gradient Boosted Trees algorithm to build a supervised halo star classifier. The classifier is trained on a sample of stars extracted from the Gaia Universe Model Snapshot, which is also convolved with the errors of the public TGAS data, which is a subset of Gaia DR1, as well as with the expected uncertainties for the upcoming Gaia DR2 catalogue. We also trained our classifier on a dataset resulting from the cross-match between the TGAS and RAVE catalogues, where the halo stars are labelled in an entirely model-independent way. We then use this model to identify halo stars in TGAS. Results. When full phase-space information is available and for Gaia DR2-like uncertainties, our classifier is able to recover 90% of the halo stars with at most 30% distance errors, in a completely unseen test set and with negligible levels of contamination. When line-of-sight velocity is not available, we recover ~ 60% of such halo stars, with less than 10% contamination. When applied to the TGAS catalogue, our classifier detects 337 high confidence red giant branch halo stars. At first glance this number may seem small, however, it is consistent with the expectation from the models, given the uncertainties in the data. The large parallax errors are in fact the biggest limitation in our ability to identify a large number of halo stars in all the cases studied.

Optimisation algorithm for decision trees and the prediction of horizon displacement of landslides monitoring

In the era of big data, the data scale of landslide monitoring could reach above terabyte level, the traditional database and data mining technology could no longer meet the requirements of intelligent monitoring and early warning. To obtain early warning information with high reliability and real time by applying big data theory, mechanisms, models and methods as well as machine learning methods are the inevitable trends in the future. This study aimed to realise a real time and precise mid-long prediction of landslide displacement, proposed two distributed landslide displacement prediction models: DLDP-GBTs (distributed landslide displacement prediction with Gradient Boosted Trees algorithm) and DLDP-RF (distributed landslide displacement prediction with Random algorithm); the cross-validation method was also adopted to evaluate and adjust parameters to reduce the root mean squared error of the model predicted results. In addition, this study proposed the rapid selection of features by using XGboost model in distributed situations can improve the Model training efficiency under distributed condition. By comparing different regression algorithms models, it was found that the DLDP-GBTs model based on the gradient optimisation decision tree was better than the other two models in terms of accuracy and real-time performance, which meets the requirements under the big data background.

Comparative Analysis of Machine Learning Techniques to Identify Churn for Telecom Data

Big data analytics has been the focus for large scale data processing. Machine learning and Big data has great future in prediction. Churn prediction is one of the sub domain of big data. Preventing customer attrition especially in telecom is the advantage of churn prediction. Churn prediction is a day-to-day affair involving millions. So a solution to prevent customer attrition can save a lot. This paper propose to do comparison of three machine learning techniques Decision tree algorithm, Random Forest algorithm and Gradient Boosted tree algorithm using Apache Spark. Apache Spark is a data processing engine used in big data which provides in-memory processing so that the processing speed is higher. The analysis is made by extracting the features of the data set and training the model. Scala is a programming language that combines both object oriented and functional programming and so a powerful programming language. The analysis is implemented using Apache Spark and modelling is done using scala ML. The accuracy of Decision tree model came out as 86%, Random Forest model is 87% and Gradient Boosted tree is 85%.

Learning to predict soccer results from relational data with gradient boosted trees

We describe our winning solution to the 2017’s Soccer Prediction Challenge organized in conjunction with the MLJ’s special issue on Machine Learning for Soccer. The goal of the challenge was to predict outcomes of future matches within a selected time-frame from different leagues over the world. A dataset of over 200,000 past match outcomes was provided to the contestants. We experimented with both relational and feature-based methods to learn predictive models from the provided data. We employed relevant latent variables computable from the data, namely so called pi-ratings and also a rating based on the PageRank method. A method based on manually constructed features and the gradient boosted tree algorithm performed best on both the validation set and the challenge test set. We also discuss the validity of the assumption that probability predictions on the three ordinal match outcomes should be monotone, underlying the RPS measure of prediction quality.

Improving Renewable Energy Forecasting With a Grid of Numerical Weather Predictions

In the last two decades, renewable energy forecasting progressed toward the development of advanced physical and statistical algorithms aiming at improving point and probabilistic forecast skill. This paper describes a forecasting framework to explore information from a grid of numerical weather predictions (NWP) applied to both wind and solar energy. The methodology combines the gradient boosting trees algorithm with feature engineering techniques that extract the maximum information from the NWP grid. Compared to a model that only considers one NWP point for a specific location, the results show an average point forecast improvement (in terms of mean absolute error) of 16.09% and 12.85% for solar and wind power, respectively. The probabilistic forecast improvement, in terms of continuous ranked probabilistic score, was 13.11% and 12.06%, respectively.

Mapping the global depth to bedrock for land surface modeling

AbstractDepth to bedrock serves as the lower boundary of land surface models, which controls hydrologic and biogeochemical processes. This paper presents a framework for global estimation of depth to bedrock (DTB). Observations were extracted from a global compilation of soil profile data (ca. 1,30,000 locations) and borehole data (ca. 1.6 million locations). Additional pseudo‐observations generated by expert knowledge were added to fill in large sampling gaps. The model training points were then overlaid on a stack of 155 covariates including DEM‐based hydrological and morphological derivatives, lithologic units, MODIS surface reflectance bands and vegetation indices derived from the MODIS land products. Global spatial prediction models were developed using random forest and Gradient Boosting Tree algorithms. The final predictions were generated at the spatial resolution of 250 m as an ensemble prediction of the two independently fitted models. The 10–fold cross‐validation shows that the models explain 59% for absolute DTB and 34% for censored DTB (depths deep than 200 cm are predicted as 200 cm). The model for occurrence of R horizon (bedrock) within 200 cm does a good job. Visual comparisons of predictions in the study areas where more detailed maps of depth to bedrock exist show that there is a general match with spatial patterns from similar local studies. Limitation of the data set and extrapolation in data spare areas should not be ignored in applications. To improve accuracy of spatial prediction, more borehole drilling logs will need to be added to supplement the existing training points in under‐represented areas.

Smart City Mobility Application--Gradient Boosting Trees for Mobility Prediction and Analysis Based on Crowdsourced Data.

Mobility management represents one of the most important parts of the smart city concept. The way we travel, at what time of the day, for what purposes and with what transportation modes, have a pertinent impact on the overall quality of life in cities. To manage this process, detailed and comprehensive information on individuals’ behaviour is needed as well as effective feedback/communication channels. In this article, we explore the applicability of crowdsourced data for this purpose. We apply a gradient boosting trees algorithm to model individuals’ mobility decision making processes (particularly concerning what transportation mode they are likely to use). To accomplish this we rely on data collected from three sources: a dedicated smartphone application, a geographic information systems-based web interface and weather forecast data collected over a period of six months. The applicability of the developed model is seen as a potential platform for personalized mobility management in smart cities and a communication tool between the city (to steer the users towards more sustainable behaviour by additionally weighting preferred suggestions) and users (who can give feedback on the acceptability of the provided suggestions, by accepting or rejecting them, providing an additional input to the learning process).

New object detection features in the OpenCV library

In this work the object detection problem is considered. A short description of implementations of the object detection system with a discriminatively trained part based model and a gradient boosting trees algorithm (as part of OpenCV library) is given. Application of the gradient boosting trees learner to the object detection problem (in terms of the pedestrian detection problem) is explored.