Boosted Tree Models Research Articles

A Data-Driven Soft Sensor for Swarm Motion Speed Prediction Using Ensemble Learning Methods

Machine Learning (ML) for swarm motion prediction is a relatively unexplored area that could help sustain and monitor daily swarm robotics collective tasks. This paper focuses on a specific application of swarm robotics which is pattern formation, to demonstrate the ability of Ensemble Learning (EL) approaches to predict the motion speed of swarm robots. Specifically, the boosted trees (BST) and bagged trees (BT) algorithms are introduced to predict the motion speed of a swarm of miniature two-wheels differential driver mobile robots performing a circle-formation via the viscoelastic control model. This choice's motivation is due to EL-based models' ability to improve the performance of ML models by combining multiple learners versus single regressors. Both BST and BT algorithms' performances are compared to ten commonly known prediction models based on Support Vector Regressors (SVRs) and Gaussian Process Regressors (GPRs) with different kernel functions. Using simulated measurements recorded every 0.1 second from the robots' sensors, we demonstrate the effectiveness of the developed methods over conventional ML models (SVR and GPR) in a free/non-free obstacles environment. Results showed that the BST and BT regression models reached the highest prediction performance with fully and partially connected swarms and even when involving different swarm sizes.

A Prediction Method Based on Extreme Gradient Boosting Tree Model and its Application

Improving the accuracy of financing risk prediction is of great significance to the healthy development of grid enterprises. Taking a provincial-level power grid company as the research object, the financing risk index system is constructed by considering multiple dimensions, and the monthly financing risk index RI of power grid enterprises from 2015-2018 is determined based on entropy weight and comprehensive index method, while the financing risk prediction model is constructed with the help of extreme gradient boosting tree model. The empirical results show that compared with support vector regression and BP neural network models, the financing risk prediction model constructed based on the extreme gradient boosting model has an excellent performance in terms of prediction accuracy and stability.

Molecular Dynamics and Light Absorption Properties of Atmospheric Dissolved Organic Matter.

The light-absorbing organic aerosol referred to as brown carbon (BrC) affects the global radiative balance. The linkages between its molecular composition and light absorption properties and how environmental factors influence BrC composition are not well understood. In this study, atmospheric dissolved organic matter (ADOM) in 55 aerosol samples from Guangzhou was characterized using Fourier transform ion cyclotron resonance mass spectrometry and light absorption measurements. The abundant components in ADOM were aliphatics and peptide-likes (in structure), or nitrogen- and sulfur-containing compounds (in elemental composition). The light absorption properties of ADOM were positively correlated with the levels of unsaturated and aromatic structures. Particularly, 17 nitrogen-containing species, which are identified by a random forest, characterized the variation of BrC absorption well. Aggregated boosted tree model and nonmetric multidimensional scaling analysis show that the BrC composition was largely driven by meteorological conditions and anthropogenic activities, among which biomass burning (BB) and OH radical were the two important factors. BrC compounds often accumulate with elevated BB emissions and related secondary processes, whereas the photolysis/photooxidation of BrC usually occurs under high solar radiance/•OH concentration. This study first illuminated how environmental factors influence BrC at the molecular level and provided clues for the molecular-level research of BrC in the future.

Need for Emergent Intervention within 6 Hours: A Novel Prediction Model for Hospital Trauma Triage

Objective: A tiered trauma team activation system allocates resources proportional to patients’ needs based upon injury burden. Previous trauma hospital-triage models are limited to predicting Injury Severity Score which is based on > 10% all-cause in-hospital mortality, rather than need for emergent intervention within 6 hours (NEI-6). Our aim was to develop a novel prediction model for hospital-triage that utilizes criteria available to the EMS provider to predict NEI-6 and the need for a trauma team activation. Methods: A regional trauma quality collaborative was used to identify all trauma patients ≥ 16 years from the American College of Surgeons-Committee on Trauma verified Level 1 and 2 trauma centers. Logistic regression and random forest were used to construct two predictive models for NEI-6 based on clinically relevant variables. Restricted cubic splines were used to model nonlinear predictors. The accuracy of the prediction model was assessed in terms of discrimination. Results: Using data from 12,624 patients for the training dataset (62.6% male; median age 61 years; median ISS 9) and 9,445 patients for the validation dataset (62.6% male; median age 59 years; median ISS 9), the following significant predictors were selected for the prediction models: age, gender, field GCS, vital signs, intentionality, and mechanism of injury. The final boosted tree model showed an AUC of 0.85 in the validation cohort for predicting NEI-6. Conclusions: The NEI-6 trauma triage prediction model used prehospital metrics to predict need for highest level of trauma activation. Prehospital prediction of major trauma may reduce undertriage mortality and improve resource utilization.

Machine learning for initial insulin estimation in hospitalized patients.

ObjectiveThe study sought to determine whether machine learning can predict initial inpatient total daily dose (TDD) of insulin from electronic health records more accurately than existing guideline-based dosing recommendations.Materials and MethodsUsing electronic health records from a tertiary academic center between 2008 and 2020 of 16,848 inpatients receiving subcutaneous insulin who achieved target blood glucose control of 100-180 mg/dL on a calendar day, we trained an ensemble machine learning algorithm consisting of regularized regression, random forest, and gradient boosted tree models for 2-stage TDD prediction. We evaluated the ability to predict patients requiring more than 6 units TDD and their point-value TDDs to achieve target glucose control.ResultsThe method achieves an area under the receiver-operating characteristic curve of 0.85 (95% confidence interval [CI], 0.84-0.87) and area under the precision-recall curve of 0.65 (95% CI, 0.64-0.67) for classifying patients who require more than 6 units TDD. For patients requiring more than 6 units TDD, the mean absolute percent error in dose prediction based on standard clinical calculators using patient weight is in the range of 136%-329%, while the regression model based on weight improves to 60% (95% CI, 57%-63%), and the full ensemble model further improves to 51% (95% CI, 48%-54%).DiscussionOwingto the narrow therapeutic window and wide individual variability, insulin dosing requires adaptive and predictive approaches that can be supported through data-driven analytic tools.ConclusionsMachine learning approaches based on readily available electronic medical records can discriminate which inpatients will require more than 6 units TDD and estimate individual doses more accurately than standard guidelines and practices.

A note on real estate appraisal in Brazil

Abstract Brazilian banks commonly use linear regression to appraise real estate: they regress price on features like area, location, etc, and use the resulting model to estimate the market value of the target property. But Brazilian banks do not test the predictive performance of those models, which for all we know are no better than random guesses. That introduces huge inefficiencies in the real estate market. Here we propose a machine learning approach to the problem. We use real estate data scraped from 15 thousand online listings and use it to fit a boosted trees model. The resulting model has a median absolute error of 8.16%. We provide all data and source code.

Learning the Low Frequency Earthquake Activity on the Central San Andreas Fault

AbstractLow frequency earthquakes (LFEs) originating below the central San Andreas Fault are associated with slow‐slip beneath the seismogenic zone within the more ductile portion of the crust. Monitoring efforts over 15 years detected >1 million LFEs. We train a gradient boosted tree model using statistical features describing the seismic waveforms to estimate the hourly LFE event count. The burst‐like LFE behavior is reproduced, while lower amplitudes are predicted during the most active periods. The hourly event counts are up to 18% greater than the catalog. The ability to continuously monitor LFE activity provides insight to when geodetic measurements of slow slip are possible, without the need for developing a computational‐intensive template‐matching catalog. Similar waveform statistical features are found between detecting LFEs and tremors, which provides additional evidence tremors are composed of LFEs. The approach extracts information contained in continuous seismic waveforms that might benefit detecting precursory signals.

A New Immunohistochemical Algorithm Using Four Antibodies Has a High Concordance with Gene Expression Profiling Defined ABC and GCB Diffuse Large B-Cell Lymphoma

▪The new WHO 2016 classification of lymphoid neoplasms requires the identification of Activated B-cell like (ABC) and Germinal center B-cell like (GCB) Diffuse large B-cell lymphomas (DLBCL) NOS as oncogenetic pathways and probably sensitivity to specific therapies differ between these two subtypes. Whereas RNA based signatures from FFPE DLBCL seem promising in reproducibility, still 15% to 20% of DLBCL NOS are of the Unclassified subtype and there is variability between gene expression profiling (GEP) and RNA signatures considering this unclassified subtype. Therefore, the immunohistochemical tool, used during the diagnostic process of the lymphoma, might be useful to maintain and develop in addition to the molecular tools. Hans and coworkers previously developed an immunohistochemical algorithm with 86% concordance with GEP among patients treated with Rituximab with antracyclin based regimens. However, some antibodies such as BCL6 are difficult to evaluate and might hamper reproducibility of scoring. We propose an alternative to the Hans algorithm using 4 easy to use antibodies while keeping the classification process fairly simple.MethodsOne thousand eight hundred forty-three patients with previously untreated de novo CD20+ DLBCL were enrolled in the GELA (Groupe d'Etudes des Lymphomes de l'Adulte)/LYSA (Lymphoma Study Association) LNH01-5B and LNH03-B clinical trials. A Tissue microarray of 854 tumors were built and immunostaining with CD10, MUM1, FOXP1 (Barrans' threshold, 0 vs Variable/Strong), BCL6, IGM, MYC, BCL2 was performed. 761 cases of DLBCL NOS had immunostains data available. For 150 of these patients, frozen tissue was available and GEP has been analyzed using GeneChip Human Genome HGU133 Plus 2.0 arrays (Affymetrix, Santa Clara, CA, USA). A new immunohistochemical algorithm was then developped and validated in concordance with GEP using a training set and a validation set (100 cases for training, 50 cases for validation). The first step was to choose the best and most parsimonious machine learning algorithm capable of predicting DLBCL subtypes. Then, logical rules were created based on the predicted probabilities of class membership using a probability threshold that optimize concordance between machine learning and logical rules predictions. xgBoost algorithm (boosted trees models) with 4 covariables was chosen, using a cut-off of 0.39 for predicted class membership probabilities.ResultsA new algorithm (Table 1) using CD10, MUM1, FOXP1 and IGM was then derived with an accuracy of 93% (IC95 [0.80 ; 0.98]) on the validation set, compared to GEP classification. This algorithm has four arms taking into account the GCB marker CD10 and independently one of the three ABC (MUM1, FOXP1, IGM) markers to classify as GCB, ABC or non classified. Hans algorithm had an accuracy of 0.88 (IC95 [0.75 ; 0.96]) on this same dataset. On the training and validation set combined, our algorithm re-classified correctly 7 GCB subtypes compared to the Hans algorithm.Survival analysis on the remaining cases (611 patients) found an ABC vs GC hazard ratio (HR) of 2.0 (IC95 [1.47 ; 2.73]) for progression free survival (PFS) and 1.95 (IC95 [1.36-2.80]) for overall survival (OS), using ABC vs GCB classification of our new algorithm. In our series, Hans' algorithm found an ABC vs GCB HR of 1,74 IC95 [1,27-2,39] for PFS and 1,87 IC95 [1,29-2,71] for OS.ConclusionOur new algorithm shows excellent performance compared to the Gold Standard GEP classification, using only 4 immunostains, and allows for more accurate risk stratification both for PFS and OS than Hans'algorithm. Therefore, it can be used as a useful tool in addition to molecular signatures. [Display omitted] DisclosuresBruneau:Innate Pharma: Research Funding. Delarue:Gilead: Consultancy, Honoraria; BMS: Consultancy, Honoraria; Roche: Consultancy, Honoraria. Haioun:PFIZER: Consultancy, Honoraria; GILEAD: Consultancy, Honoraria; JANSSEN: Consultancy, Honoraria; Sandoz: Consultancy, Honoraria; Takeda: Consultancy, Honoraria; Amgen: Consultancy, Honoraria; Roche: Consultancy, Honoraria. Casasnovas:BMS: Consultancy, Honoraria; Abbvie: Consultancy, Honoraria; Sanofi: Consultancy, Honoraria; Takeda: Consultancy, Honoraria; Gilead: Consultancy, Honoraria, Research Funding; Roche: Consultancy, Honoraria, Research Funding. Morschhauser:Roche: Consultancy, Honoraria; Celgene: Consultancy, Honoraria; Janssen: Consultancy, Honoraria; Bristol-Myers Squibb: Consultancy, Honoraria; Gilead: Consultancy; Servier: Consultancy. Emile:Merck Serono: Honoraria. Tilly:Gilead: Honoraria; Immunogen: Honoraria; Karyopharm: Consultancy, Honoraria, Membership on an entity's Board of Directors or advisory committees; Celgene: Consultancy, Honoraria, Membership on an entity's Board of Directors or advisory committees, Research Funding; Roche: Consultancy, Honoraria, Membership on an entity's Board of Directors or advisory committees, Research Funding; Janssen: Consultancy, Honoraria, Membership on an entity's Board of Directors or advisory committees; Takeda: Consultancy, Honoraria. Salles:Celgene: Consultancy, Honoraria; Gilead: Consultancy, Honoraria; Janssen: Consultancy, Honoraria; Kite: Consultancy, Honoraria; MSD: Consultancy, Honoraria; Novartis: Consultancy, Honoraria; Servier: Consultancy, Honoraria; BMS: Consultancy; morphosys: Consultancy, Honoraria; Amgen: Consultancy, Honoraria; Roche: Consultancy, Honoraria, Research Funding. Jardin:Roche: Honoraria; celgen: Honoraria; Janssen: Honoraria. Molina:Merck Serono: Honoraria; Novartis: Honoraria; Takeda: Other: travel support to the ASH meeting.

Gaussian Mixture Models and Machine Learning Predict Megakaryocytic Growth and Differentiation Potential Ex Vivo

The ability to analyze single cells via flow cytometry has resulted in a wide range of biological and medical applications, such as monitoring populations of developing immune and hematopoietic cells over time. Currently, there is no established framework to compare and interpret time-series flow cytometry data for cellular engineering applications. Manual analysis of temporal trends is time-consuming and subjective for large-scale datasets. We resolved this bottleneck by developing TEmporal Gaussian Mixture models (TEGM), an unbiased computational strategy to quantify and predict temporal trends of developing cell subpopulations indicative of cellular phenotype. We validated its utility by accurately predicting peak CD41+/CD42+ % of megakaryocyte (MK) cultures derived from human CD34+-selected umbilical cord blood units (CB).TEGM applies Gaussian mixture modeling for feature extraction and gradient boosted trees for prediction. Our modeling strategy allows for data to be gated and clustered on multi-dimensional planes, enabling rapid processing of flow cytometry data. TEGM enables the extraction of subtle features, such as the dispersion and rate of change of surface marker expression for each subpopulation over time. These critical, yet hard-to-discern, features are fed into machine-learning algorithms that predict underlying cellular classes and phenotypes. Our framework can be flexibly applied to conventional flow cytometry sampling schemes, and allows for faster and more consistent processing of time-series flow cytometry data.As proof-of-concept, we applied our method to the analysis of ex vivo MK differentiation and maturation of hematopoietic cells from donors with varying potential to generate CD41+/CD42+ cells. Our computational approach consists of three major steps: preprocessing, feature extraction, and prediction (Figure 1). We illustrate the major steps of the computational approach by predicting peak CD41+/CD42+ % MK maturation of CD34+-selected CB cells from 16 independent donors. Cells were cultured over a 19-day multi-phase differentiation culture, consisting of a pre-expansion phase and a differentiation phase. The novel dataset comprised 720 measurements from 80 perturbations of 16 individual donors, with 9 time-point measurements sampled every 2-5 days for each donor. In the preprocessing step, we filtered out non-uniform events, margin events, and doublets. We then constructed an automated gating strategy to extract surface marker expression of various clusters of DAPIlow/CD41+ MK cells. Notably, we demonstrated that estimation of the CD34+ and CD42+ % was within 1% of manual gating estimates, thus illustrating the consistency and accuracy of the technique. We then performed feature extraction for each flow cytometry time-course dataset on several descriptors, such as growth rate, viability, production, percentage positivity of each surface marker, covariance of mean fluorescence intensity, rate of change, and bifurcation of each subpopulation. A gradient boosted tree model was trained using an explanatory matrix describing early characteristics (Day 0-Day 9) and tested to predict peak CD41+/CD42+ marker expression, which typically occurs on Day 14 to Day 17.Overall, we identified several influential early culture factors that are predictive of peak CD42+ % expression. We showed that CD41+ % on Day 5 and Day 7 is highly predictive of peak CD42+ % expression. Cell viability and CD34+ % were comparatively less predictive of peak CD42+ %. Based on our analysis, we were able to identify the best performing cultures with high sensitivity and specificity (AUROC = 0.92, where 1 denotes perfect accuracy). Predicted and actual CD41+/CD42+ responses were highly correlated (p=7.4e09, R = 0.87) using 3 independently selected partitions of test/training sets of our data. We demonstrated that the resulting model captured apparent cell development processes represented by cell surface markers including CD34, CD41, and CD42, and was able to predict MK differentiation and maturation potential of a given CB unit in culture. Identifying CB units with high and low MK potential early in the 19-day culture process can save expensive resources and time, and provides the potential to intervene during the culture process.Figure 1. Manual gating strategy (left) vs. TEGM gating strategy (right) for time-series flow cytometry data. [Display omitted] DisclosuresNo relevant conflicts of interest to declare.

Forecasting the hydroelectric power generation of GCMs using machine learning techniques and deep learning (Almus Dam, Turkey)

Renewable energy is one of the most important factors for developed and sustainable societies. However, its utilization in electrical power grid systems can be very challenging regarding rates predictably. Renewable energy depends mainly on environmental conditions such as rainfall-runoff ratios and temperature. Because of that, the expected power production heavily fluctuates, which makes the prediction and calculation of feed-in into the power grid very challenging. The accurate forecasting of energy production is a very crucial issue for power management process. This paper presents the results of deploying Machine Learning Techniques in short-term forecasting of the amount of energy produced of General Circulation Models (GCMs) Data by Almus Dam and Hydroelectric Power Plant in Tokat, Turkey. The study demonstrates the use of modeling techniques in hydropower forecasting process using the predicted monthly hydroelectric power generation data of GCMs from 2018 to 2080. Decision Tree, Deep Learning, Generalized Linear, Gradient Boosted Trees and Random Forest models are utilized to forecast the hydropower production. The results show that the correlation value of the gradient boosted trees model equals 0.717, which means that the gradient boosted trees model is the most successful model for the present data. The gradient boosted trees model used in the prediction process for each GCM in each scenario is 4.5 and 8.5. The results show that there are small differences between the models, which means that the predictions are going in similar directions for all these models.

Deep learning and Boosted trees for injuries prediction in power infrastructure projects

Electrical injury impacts are substantial and massive. Investments in electricity will continue to increase, leading to construction project complexities, which undoubtedly contribute to injuries and associated effects. Machine learning (ML) algorithms are used to quantify and model causes of injuries; however, conventional ML techniques do not produce optimal results since they require careful engineering to transform data into suitable forms. In this study, we develop and compare state-of-the-art ML algorithms (deep learning and boosted trees) with other conventional methods to overcome this problem by analyzing incident cases obtained from a leading UK power infrastructure company. The predictive performance of the developed models was benchmarked using a statistical comparison between observations and predicted values. Results showed that the implementation of deep feedforward neural networks achieved the best predictions (accuracy = 0.967 and Cohen Kappa measure = 0.964). Furthermore, we conduct a sensitivity analysis to determine the effect of input parameters and data sizes on the modes’ behavior. The sensitivity analysis results showed strong generalization abilities of the deep learning and boosted tree models and their effectiveness for safety risks management.

Artificial Flora Algorithm-Based Feature Selection with Gradient Boosted Tree Model for Diabetes Classification

PurposeClassification of medical data is essential to determine diabetic treatment options; therefore, the objective of the study was to develop a model to classify the three diabetes type diagnoses according to multiple patient attributes.MethodsThree different datasets are used to develop a novel medical data classification model. The proposed model involved preprocessing, artificial flora algorithm (AFA)-based feature selection, and gradient boosted tree (GBT)-based classification. Then, the processing occurred in two steps, namely, format conversion and data transformation. AFA was applied for selecting features, such as demographics, vital signs, laboratory tests, medications, from the patients’ electronic health records. Lastly, the GBT-based classification model was applied for classifying the patients’ cases to type I, type II, or gestational diabetes mellitus.ResultsThe effectiveness of the proposed AFA-GBT model was validated using three diabetes datasets to classify patient cases into one of the three different types of diabetes. The proposed model showed a maximum average precision of 91.64%, a recall of 97.46%, an accuracy of 99.93%, an F-score of 94.19%, and a kappa of 96.61%.ConclusionThe AFA-GBT model could classify patient diagnoses into the three diabetes types efficiently.

HAZARD ASSESSMENT OF DEBRIS FLOW ALONG ZIRE-BOMI SECTION OF SICHUAN-TIBET RAILWAY

Debris flow is one of the most developed disaster types in the Tibetan Plateau, and poses a certain threat to the construction of Sichuan-Tibet Railway and the safety of life and property due to its sudden outbreak, violent movement process, and strong destructive characteristics. Geological hazard assessment is part of the effective measures in the management and prevention of disaster prevention and mitigation. For reasonably quantifying the spatial distribution characteristics of debris flow hazard along the railway line, the paper takes the Zire-Bomi section of the Sichuan-Tibet Railway within Linzhi City as the experimental area. We apply the Bayesian optimization algorithm based random forest(TBOR) and gradient boosting tree model(TBOG)to quantify the debris flow hazard values of the route section and identify the hazard levels. The input information of the two models includes 172 historical hazard points and 11 feature parameters. The output information is the outbreak probability of debris flow for each prediction unit. Lastly, the ROC-AUC method is employed to test the evaluation outcomes of the two prediction algorithms. The findings reveal that in the TBOR and TBOG models, the overall debris flow hazard level of the Zire-Bomi line section of the Sichuan-Tibet Railway is relatively high. The proportion of hazard zones inside the high-higher hazard interval of the two models reaches 56.439% and 66.580% respectively, corresponding to the highest hazard point densities of 12.577/(102km2) and 12.940/(102km2)respectively. Compared to the ROC-AUC values derived from the TBOG model, the TBOR model computes 0.89, which is higher than that of TBOR at 0.83. Therefore, the TBOR model has better prediction accuracy. The research findings can provide necessary reference for the construction of disaster prevention and mitigation protection projects along Sichuan-Tibet Railway and the hazard evaluation of other line sections.

Predicting city-scale daily electricity consumption using data-driven models

• We studied how city electricity use is influenced by weather and COVID-19 pandemic. • Seven data-driven models were applied and evaluated for data of three cities. • Gradient boosting tree model delivers the most accurate prediction with CVRMSE of 4%−6%. • 1 °C increase of ambient temperature drives up the three cities electricity usage by around 5%. • COVID-19 curtailment reduced city-scale electricity usage by 2%−12%. Accurate electricity demand forecasts that account for impacts of extreme weather events are needed to inform electric grid operation and utility resource planning, as well as to enhance energy security and grid resilience. Three common data-driven models are used to predict city-scale daily electricity usage: linear regression models, machine learning models for time series data, and machine learning models for tabular data. In this study, we developed and compared seven data-driven models: (1) five-parameter change-point model, (2) Heating/Cooling Degree Hour model, (3) time series decomposed model implemented by Facebook Prophet, (4) Gradient Boosting Machine implemented by Microsoft lightGBM, and (5) three widely-used machine learning models (Random Forest, Support Vector Machine, Neural Network). Seven models are applied to the city-scale electricity usage data for three metropolitan areas in the United States: Sacramento, Los Angeles, and New York. Results show seven models can predict the metropolitan area's daily electricity use, with a coefficient of variation of the root mean square error (CVRMSE) less than 10%. The lightGBM provides the most accurate results, with CVRMSE on the test dataset of 6.5% for Los Angeles, 4.6% for Sacramento, and 4.1% for the New York metropolitan area. These models are further applied to explore how extreme weather events (e.g., heat waves) and unexpected public health events (e.g., COVID-19 pandemic) influence each city's electricity demand. Results show weather-sensitive component accounts for 30%–50% of the total daily electricity usage. Every degree Celsius ambient temperature increase in summer leads to about 5% (4.7% in Los Angeles, 6.2% in Sacramento, and 5.1% in New York) more daily electricity usage compared with the base load in the three metropolitan areas. The COVID-19 pandemic reduced city-scale electricity demand: compared with the pre-pandemic same months in 2019, daily electricity usage during the 2020 pandemic decreased by 10% in April and started to rebound in summer.

Logistic regression and machine learning predicted patient mortality from large sets of diagnosis codes comparably

ObjectiveThe objective of the study was to compare the performance of logistic regression and boosted trees for predicting patient mortality from large sets of diagnosis codes in electronic healthcare records. Study Design and SettingWe analyzed national hospital records and official death records for patients with myocardial infarction (n = 200,119), hip fracture (n = 169,646), or colorectal cancer surgery (n = 56,515) in England in 2015–2017. One-year mortality was predicted from patient age, sex, and socioeconomic status, and 202 to 257 International Classification of Diseases 10th Revision codes recorded in the preceding year or not (binary predictors). Performance measures included the c-statistic, scaled Brier score, and several measures of calibration. ResultsOne-year mortality was 17.2% (34,520) after myocardial infarction, 27.2% (46,115) after hip fracture, and 9.3% (5,273) after colorectal surgery. Optimism-adjusted c-statistics for the logistic regression models were 0.884 (95% confidence interval [CI]: 0.882, 0.886), 0.798 (0.796, 0.800), and 0.811 (0.805, 0.817). The equivalent c-statistics for the boosted tree models were 0.891 (95% CI: 0.889, 0.892), 0.804 (0.802, 0.806), and 0.803 (0.797, 0.809). Model performance was also similar when measured using scaled Brier scores. All models were well calibrated overall. ConclusionIn large datasets of electronic healthcare records, logistic regression and boosted tree models of numerous diagnosis codes predicted patient mortality comparably.

Exploration of text matching methods in Chinese disease Q&A systems: A method using ensemble based on BERT and boosted tree models

BackgroundText matching is one of the basic tasks in the field of natural language processing. Owing to the particularity of Chinese language and medical texts, text matching has greater application and research value in the medical field. In 2019, at the China Health Information Processing Conference (CHIP), 30,000 sets of real disease Q&A data in Chinese on diabetes, hypertension, hepatitis B, AIDS, and breast cancer were released for public evaluation. A total of 90 teams participated in the evaluation. PurposeTo explore the best method of text matching of Chinese medical Q&A data by participating in an evaluation competition. MethodAfter analyzing the Chinese medical Q&A data provided by the competition, we used the bidirectional encoder representations from transformers (BERT) model and a boosted tree model to compare the effects. At the same time, we analyzed the importance of the features extracted through feature engineering. Finally, we integrated the BERT and boosted tree models, and proved the effectiveness of the ensemble through a correlation analysis. ResultsThe final F1 score of the ensemble model is 0.90825, ranking first among the 90 participating teams. The highest F1 score of the single BERT model is 0.87443, whereas the highest F1 score of the boosted tree single model is only 0.86915. The F1 score of the BERT multi-model ensemble is 0.87473 (an average increase of 0.756% compared to the single model), and the F1 score of the boosted tree multi-model ensemble is 0.86720 (an average decrease of 0.03% compared to the single model). In the feature importance experiment, the out-degree and in-degree of the Q&A sentence are of utmost importance. In the correlation experiment, the correlation coefficients between models of the same type are all as high as 0.9, which shows a high similarity. The correlation coefficient between different types of models is approximately 0.7, which shows a certain degree of discrimination. With the ensemble of the two types of models, the F1 score reached 0.90825, which is 3.88% higher than that of the optimal single model. ConclusionIn our study, the proposed model ensemble method was shown to effectively improve the performance of a single model. It achieves good results in Chinese medical Q&A tasks and has a good generalization property.

Interpretable-Machine-Learning Evidence for Importance and Optimum of Learning Time

This study uses a machine learning technique, a boosted tree model, to relate the student cognitive achievement in the 2018 data from the Programme of International Student Assessment (PISA) to other features related to the student learning process, capturing the complex and nonlinear relationships in the data. The SHapley Additive exPlanations (SHAP) approach is subsequently used to explain the complexity of the model. It reveals the relative importance of each of the features in predicting cognitive achievement. We find that instruction time comes out as an important predictor, but with a nonlinear relationship between its value and the contribution to the prediction. We find that a large weekly learning time of more than 35 hours is associated with less positive or even negative effect on the predicted outcome. We discuss how this method can possibly be used to signal problems in the student population related to learning time or other features.

Comparison of multi-criteria-analytical hierarchy process and machine learning-boosted tree models for regional flood susceptibility mapping: a case study from Slovakia

Identification of areas susceptible to floods is an important issue which requires an increased attention due to the changing frequency and magnitude of floods, which is mainly a result of the ongoing climate change and increasing anthropic pressure on the landscape. In this study, the aim was to identify the areas susceptible to floods using and comparing two different approaches, namely the multi-criteria decision analysis-analytical hierarchy process (MCDA-AHP) and the machine learning-boosted classification (BCT) and boosted regression (BRT) tree. The study area was represented by the Topľa river basin, Slovakia. Altogether, seven relevant flood conditioning factors: elevation, slope, river network density, distance from river, flow accumulation, curve numbers and lithology as well as flood inventory database consisting of 107 flood locations were used. Based on the results, almost 40% of the study area is characterized by high to very high flood susceptibility using the MCDA-AHP. In case of the BCT and BRT models, the share of high and very high flood susceptibility class on the basin area is 45% and 38%, respectively. Validation of the performed flood susceptibility models confirmed generally higher accuracy of the machine learning models. The accuracy of the MCDA-AHP model was 81.33% while the accuracy of the boosted tree models was 87.70% and 91.42%, respectively, for classification and regression. The results of this study can enhance more effective preliminary flood risk assessment according to the EU Floods Directive.

Using machine learning to predict paediatric 30-day unplanned hospital readmissions: a case-control retrospective analysis of medical records, including written discharge documentation.

Objectives To assess whether adding clinical information and written discharge documentation variables improves prediction of paediatric 30-day same-hospital unplanned readmission compared with predictions based on administrative information alone. Methods A retrospective matched case-control study audited the medical records of patients discharged from a tertiary paediatric hospital in Western Australia (WA) between January 2010 and December 2014. A random selection of 470 patients with unplanned readmissions (out of 3330) were matched to 470 patients without readmissions based on age, sex, and principal diagnosis at the index admission. Prediction utility of three groups of variables (administrative, administrative and clinical, and administrative, clinical and written discharge documentation) were assessed using standard logistic regression and machine learning. Results Inclusion of written discharge documentation variables significantly improved prediction of readmission compared with models that used only administrative and/or clinical variables in standard logistic regression analysis (χ2 17=29.4, P=0.03). Highest prediction accuracy was obtained using a gradient boosted tree model (C-statistic=0.654), followed closely by random forest and elastic net modelling approaches. Variables highlighted as important for prediction included patients' social history (legal custody or patient was under the care of the Department for Child Protection), languages spoken other than English, completeness of nursing admission and discharge planning documentation, and timing of issuing discharge summary. Conclusions The variables of significant social history, low English language proficiency, incomplete discharge documentation, and delay in issuing the discharge summary add value to prediction models. What is known about the topic? Despite written discharge documentation playing a critical role in the continuity of care for paediatric patients, limited research has examined its association with, and ability to predict, unplanned hospital readmissions. Machine learning approaches have been applied to various health conditions and demonstrated improved predictive accuracy. However, few published studies have used machine learning to predict paediatric readmissions. What does this paper add? This paper presents the findings of the first known study in Australia to assess and report that written discharge documentation and clinical information improves unplanned rehospitalisation prediction accuracy in a paediatric cohort compared with administrative data alone. It is also the first known published study to use machine learning for the prediction of paediatric same-hospital unplanned readmission in Australia. The results show improved predictive performance of the machine learning approach compared with standard logistic regression. What are the implications for practitioners? The identified social and written discharge documentation predictors could be translated into clinical practice through improved discharge planning and processes, to prevent paediatric 30-day all-cause same-hospital unplanned readmission. The predictors identified in this study include significant social history, low English language proficiency, incomplete discharge documentation, and delay in issuing the discharge summary.

Prediction for the Risk of Multiple Chronic Conditions Among Working Population in the United States With Machine Learning Models.

Objective: Chronic diseases have become the most prevalent and costly health conditions in the healthcare industry, deteriorating the quality of life, adversely affecting the work productivity, and costing astounding medical resources. However, few studies have been conducted on the predictive analysis of multiple chronic conditions (MCC) based on the working population. Results: Seven machine learning algorithms are used to support the decision making of healthcare practitioner on the risk of MCC. The models were developed and validated using checkup data from 451,425 working population collected by the healthcare providers. Our result shows that all proposed models achieved satisfactory performance, with the AUC values ranging from 0.826 to 0.850. Among the seven predictive models, the gradient boosting tree model outperformed other models, achieving an AUC of 0.850. Conclusions: Our risk prediction model shows great promise in automating real-time diagnosis, supporting healthcare practitioners to target high-risk individuals efficiently, and helping healthcare practitioners tailor proactive strategies to prevent the onset or delay the progression of the chronic diseases.