Extremely Randomized Trees Research Articles

Research on imbalance machine learning methods for MRT_1WI soft tissue sarcoma data

BackgroundSoft tissue sarcoma is a rare and highly heterogeneous tumor in clinical practice. Pathological grading of the soft tissue sarcoma is a key factor in patient prognosis and treatment planning while the clinical data of soft tissue sarcoma are imbalanced. In this paper, we propose an effective solution to find the optimal imbalance machine learning model for predicting the classification of soft tissue sarcoma data.MethodsIn this paper, a large number of features are first obtained based on T_1WI images using the radiomics methods.Then, we explore the methods of feature selection, sampling and classification, get 17 imbalance machine learning models based on the above features and performed extensive experiments to classify imbalanced soft tissue sarcoma data. Meanwhile, we used another dataset splitting method as well, which could improve the classification performance and verify the validity of the models.ResultsThe experimental results show that the combination of extremely randomized trees (ERT) classification algorithm using SMOTETomek and the recursive feature elimination technique (RFE) performs best compared to other methods. The accuracy of RFE+STT+ERT is 81.57% , which is close to the accuracy of biopsy, and the accuracy is 95.69% when using another dataset splitting method.ConclusionPreoperative predicting pathological grade of soft tissue sarcoma in an accurate and noninvasive manner is essential. Our proposed machine learning method (RFE+STT+ERT) can make a positive contribution to solving the imbalanced data classification problem, which can favorably support the development of personalized treatment plans for soft tissue sarcoma patients.

Reconstruction of 0.05° all-sky daily maximum air temperature across Eurasia for 2003–2018 with multi-source satellite data and machine learning models

The Eurasian continent is highly vulnerable to climate change. However, there is a lack of high spatiotemporal resolution temperature datasets to support climate change analysis in this region. In this study, an all-sky daily maximum air temperature (Tmax) product at 0.05° spatial resolution across Eurasia for 2003–2018 was developed. This product is generated using a satellite-derived model, including parameters such as daytime and nighttime land surface temperature (LST), downward shortwave radiation, net radiation, leaf area index, enhanced vegetation index, and albedo. The study area, Eurasia, was divided into seven regions using a data-driven method. Four machine learning methods, histogram-based gradient boosting (HGB), extremely randomized trees (ET), random forest (RF), and deep belief network (DBN) were employed to train Tmax estimation models using 4476 stations from GHCN, GSOD, and CMDC. HGB was finally selected since it exhibited the highest estimation accuracy, the determination coefficient (R2) and root-mean-square-error (RMSE) of the HGB model are 0.984 and 1.736 °C with non-missing values in datasets and 0.985 and 1.812 °C with missing values respectively. The impact of the different situations of LST features on HGB models was tested. LST features containing interpolated LST and GLDAS Ta are considered the best. The spatial and temporal accuracy of HGB models was then examined across different land cover types, latitudes, elevations, and months. The permutation test was employed to examine the contributions of different features. Finally, HGB models trained using the best LST features were used to generate the Tmax products. In comparison with existing temperature products, the R2 and RMSE values were reported as 0.980 and 2.177 °C, indicating strong competition among existing Tmax products. In summary, our study provides a scheme for estimating parameters with missing feature values in a consistent manner and provides a solid foundation for the environmental and climate changes studies.

DBP-DeepCNN: Prediction of DNA-binding proteins using wavelet-based denoising and deep learning

DNA-binding proteins (DBPs) are highly concerned with several types of cancers (lung, breast, and liver), other fatal diseases (AIDS/HIV, asthma), and are used in the designing of drug. A series of predictors were constructed for identification of DBPs. However, a more accurate computational predictor is still essential for further performance improvement. In this work, a deep learning-based predictor (DBP-DeepCNN) is proposed for improving DBPs prediction. The salient features are derived by a novel method, namely, R-PSSM-DWT (Reduced position-specific scoring matrix-discrete wavelet transform) as well as Lead-BiPSSM (Lead-bigram-position specific scoring matrix), PSSM-DPC (Position specific scoring matrix-dipeptide composition), ED-PSSM (Evolutionary difference position specific scoring matrix), and F-PSSM (Filtered position specific scoring matrix). Further, the models are trained with 2D CNN (two-dimensional convolutional neural network), XGB (eXtreme gradient boosting), Adaboost, and ERT (extremely randomized trees). 2D CNN-based model with R-PSSM-DWT produced 6.92% and 1.32% higher accuracies than existing approach on training and independent datasets, respectively. These outcomes verified the superlative success rate of our novel predictor over the existing studies. In addition to being a promising method for large scale prediction of DBPs. DBP-DeepCNN would be fruitful for establishing more promising therapeutic strategies for chronic disease treatment.

Above-Ground Biomass Prediction for Croplands at a Sub-Meter Resolution Using UAV–LiDAR and Machine Learning Methods

Current endeavors to enhance the accuracy of in situ above-ground biomass (AGB) prediction for croplands rely on close-range monitoring surveys that use unstaffed aerial vehicles (UAVs) and mounted sensors. In precision agriculture, light detection and ranging (LiDAR) technologies are currently used to monitor crop growth, plant phenotyping, and biomass dynamics at the ecosystem scale. In this study, we utilized a UAV–LiDAR sensor to monitor two crop fields and a set of machine learning (ML) methods to predict real-time AGB over two consecutive years in the region of Mid-Jutland, Denmark. During each crop growing period, UAV surveys were conducted in parallel with AGB destructive sampling every 7–15 days, the AGB samples from which were used as the ground truth data. We evaluated the ability of the ML models to estimate the real-time values of AGB at a sub-meter resolution (0.17–0.52 m2). An extremely randomized trees (ERT) regressor was selected for the regression analysis, based on its predictive performance for the first year’s growing season. The model was retrained using previously identified hyperparameters to predict the AGB of the crops in the second year. The ERT performed AGB estimation using height and reflectance metrics from LiDAR-derived point cloud data and achieved a prediction performance of R2 = 0.48 at a spatial resolution of 0.35 m2. The prediction performance could be improved significantly by aggregating adjacent predictions (R2 = 0.71 and R2 = 0.93 at spatial resolutions of 1 m2 and 2 m2, respectively) as they ultimately converged to the reference biomass values because any individual errors averaged out. The AGB prediction results were examined as function of predictor type, training set size, sampling resolution, phenology, and canopy density. The results demonstrated that when combined with ML regression methods, the UAV–LiDAR method could be used to provide accurate real-time AGB prediction for crop fields at a high resolution, thereby providing a way to map their biochemical constituents.

Delineation of groundwater potential zones of upper Godavari sub-basin of India using bi-variate, MCDM and advanced machine learning algorithms

Sustainable management of groundwater resource is a most critical due to its over exploitation and ascending stress by industrial and socio-economic factors. It is utmost important to manage this precious resource by properly identifying the suitable Groundwater Potential Zones (GPZ). Therefore, the main aim of the present study is to delineate the GPZ in the upper Godavari sub-basin of India by employing different bi-variate, Multi Criteria Decision Making (MCDM), ensembled, and Machine Learning (ML) models. These models include Weight of Evidence (WoE) (bi-variate), Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) (MCDM), Fuzzified Functional Ratio (F-FR) (bi-variate), Extreme Gradient Boosting (XGB) (ML) and Extremely randomized Trees (ET) ML. The ensembled model featured different combination of WoE, TOPSIS and F-FR for mapping the enhanced accuracy in predicting the GPZ. A total of 15 groundwater factors were considered where 75% of the data were selected as training data and the rest 25% as validation data. These data were used to produce the ensembled ML models. The result of the model was plotted in terms of area under curve (AUC)-Receiver Operating Characteristics (ROC) curve and selected best model. The AUC-ROC result of the obtained model was found to be WoE oE models. The result of the model was plotted in termWoE_TOPSIS = 94%, WoE_F-FR = 93%, TOPSIS_F-FR = 94% and WoE_TOPSIS_F-FR = 95%. Results clearly indicate the improved accuracy of ensembled bi-variate and MCDM model over advanced ML model. The predicted statistical properties of the ensembled model also resembled ML models and a high correlation was observed. Thus, the ensembled model can be used over advanced ML models for delineating the GPZ mapping.

Hyperspectral estimation of petroleum hydrocarbon content in soil using ensemble learning method and LASSO feature extraction

ABSTRACT Rapid and accurate estimation of soil petroleum hydrocarbon content is crucial for analyzing the degree of soil pollution and evaluating pollution status. Surface soil samples and hyperspectral measurements in the reservoir area in Lenghu Town, Qinghai Province, China, were viewed as research objects, and the correlation between different spectral forms of original data and petroleum hydrocarbon content in soil was analyzed. To improve the estimation accuracy, we proposed a solution that introduces least absolute shrinkage and selection operator (LASSO) combined with extremely randomized trees (ERT) and gradient boosting decision tree (GBDT) ensemble learning for constructing hyperspectral estimation model. The results show: LASSO algorithm can not only solve the spectral multicollinearity problem effectively but also reduce the number and calculation complexity of soil hyperspectral variables to a great extent. Compared with traditional machine learning, ERT and GBDT perform superior. In particular, the estimation accuracy of the LASSO-GBDT model is the highest.

Application of Extremely Randomised Trees for exploring influential factors on variant crash severity data

Crash severity models play a crucial role in evaluating the influencing factors in the severity of traffic crashes. In this study, Extremely Randomised Tree (ERT) is used as a machine learning technique to analyse the severity of crashes. The crash data in the province of Khorasan Razavi, Iran, for a period of 5 years from 2013 to 2017, is used for crash severity model development. The dataset includes traffic-related variables, vehicle specifications, vehicle movement, land use characteristics, temporal characteristics, and environmental variables. In this paper, Feature Importance Analysis (FIA), Partial Dependence Plots (PDP), and Individual Conditional Expectation (ICE) plots are utilised to analyse and interpret the results. According to the results, the involvement of vulnerable road users such as motorcyclists and pedestrians alongside traffic-related variables are among the most significant variables in crash severity. Results show that the presence of motorcycles can increase the probability of injury crashes by around 30% and almost double the probability of fatal crashes. Analysing the interaction of PDPs shows that driving speeds above 60 km/h in residential areas raises the probability of injury crashes by about 10%. In addition, at speeds higher than 70 km/h, the presence of pedestrians approximately increases the probability of fatal crashes by 6%.

Data fusion and machine learning for ship fuel efficiency modeling: Part II – Voyage report data, AIS data and meteorological data

When voyage report data is utilized as the main data source for ship fuel efficiency analysis, its information on weather and sea conditions is often regarded as unreliable. To solve this issue, this study approaches AIS data to obtain the ship's actual detailed geographical positions along its sailing trajectory and then further retrieve the weather and sea condition information from publicly accessible meteorological data sources. These more reliable data about weather and sea conditions the ship sails through is fused into voyage report data in order to improve the accuracy of ship fuel consumption rate models. Eight 8100-TEU to 14,000-TEU containerships from a global shipping company were used in experiments. For each ship, nine datasets were constructed based on data fusion and eleven widely-adopted machine learning models were tested. Experimental results revealed the benefits of fusing voyage report data, AIS data, and meteorological data in improving the fit performances of machine learning models of forecasting ship fuel consumption rate. Over the best datasets, the performances of several decision tree-based models are promising, including Extremely randomized trees (ET), AdaBoost (AB), Gradient Tree Boosting (GB) and XGBoost (XG). With the best datasets, their R2 values over the training sets are all above 0.96 and mostly reach the level of 0.99–1.00, while their R2 values over the test sets are in the range from 0.75 to 0.90. Fit errors of ET, AB, GB, and XG on daily bunker fuel consumption, measured by RMSE and MAE, are usually between 0.8 and 4.5 ton/day. These results are slightly better than our previous study, which confirms the benefits of adopting the actual geographical positions of the ship recorded by AIS data, compared with the estimated geographical positions derived from the great circle route, in retrieving weather and sea conditions the ship sails through.

Data fusion and machine learning for ship fuel efficiency modeling: Part I – Voyage report data and meteorological data

The International Maritime Organization has been promoting energy-efficient operational measures to reduce ships' bunker fuel consumption and the accompanying emissions, including speed optimization, trim optimization, weather routing, and the virtual arrival policy. The theoretical foundation of these measures is a model that can accurately forecast a ship's bunker fuel consumption rate according to its sailing speed, displacement/draft, trim, weather conditions, and sea conditions. Voyage report is an important data source for ship fuel efficiency modeling but its information quality on weather and sea conditions is limited by a snapshotting practice with eye inspection. To overcome this issue, this study develops a solution to fuse voyage report data and publicly accessible meteorological data and constructs nine datasets based on this data fusion solution. Eleven widely-adopted machine learning models were tested over these datasets for eight 8100-TEU to 14,000-TEU containerships from a global shipping company. The best datasets found reveal the benefits of fusing voyage report data and meteorological data, as well as the practically acceptable quality of voyage report data. Extremely randomized trees (ET), AdaBoost (AB), Gradient Tree Boosting (GB) and XGBoost (XG) present the best fit and generalization performances. Their R2 values over the best datasets are all above 0.96 and even reach 0.99 to 1.00 for the training set, and 0.74 to 0.90 for the test set. Their fit errors on daily bunker fuel consumption are usually between 0.5 and 4.0 ton/day. These models have good interpretability in explaining the relative importance of different determinants to a ship's fuel consumption rate.

Data fusion and machine learning for ship fuel efficiency modeling: Part III – Sensor data and meteorological data

Sensors installed on a ship return high quality data that can be used for ship bunker fuel efficiency analysis. However, important information about weather and sea conditions the ship sails through, such as waves, sea currents, and sea water temperature, is often absent from sensor data. This study addresses this issue by fusing sensor data and publicly accessible meteorological data, constructing nine datasets accordingly, and experimenting with widely adopted machine learning (ML) models to quantify the relationship between a ship's fuel consumption rate (ton/day, or ton/h) and its voyage-based factors (sailing speed, draft, trim, weather conditions, and sea conditions). The best dataset found reveals the benefits of fusing sensor data and meteorological data for ship fuel consumption rate quantification. The best ML models found are consistent with our previous studies, including Extremely randomized trees (ET), Gradient Tree Boosting (GB) and XGBoost (XG). Given the best dataset from data fusion, their R2 values over the training set are 0.999 or 1.000, and their R2 values over the test set are all above 0.966. Their fit errors with RMSE values are below 0.75 ton/day, and with MAT below 0.52 ton/day. These promising results are well beyond the requirements of most industry applications for ship fuel efficiency analysis. The applicability of the selected datasets and ML models is also verified in a rolling horizon approach, resulting in a conjecture that a rolling horizon strategy of “5-month training + 1-month test/applicatoin” could work well in practice and sensor data of less than five months could be insufficient to train ML models.

Multi-omics to characterize the functional relationships of R-loops with epigenetic modifications, RNAPII transcription and gene expression.

Abnormal accumulation of R-loops results in replication stress, genome instability, chromatin alterations and gene silencing. Little research has been done to characterize functional relationships among R-loops, histone marks, RNA polymerase II (RNAPII) transcription and gene regulation. We built extremely randomized trees (ETs) models to predict the genome-wide R-loops using RNAPII and multiple histone modifications chromatin immunoprecipitation (ChIP)-seq, DNase-seq, Global Run-On sequencing (GRO-seq) and R-loop profiling data. We compared the performance of ET models to multiple machine learning approaches, and the proposed ET models achieved the best and extremely robust performances. Epigenetic profiles are highly predictive of R-loops genome-widely and they are strongly associated with R-loop formation. In addition, the presence of R-loops is significantly correlated with RNAPII transcription activity, H3K4me3 and open chromatin around the transcription start site, and H3K9me1 and H3K9me3 around the transcription termination site. RNAPII pausing defects were correlated with 5'R-loops accumulation, and transcriptional termination defects and read-throughs were correlated with 3'R-loops accumulation. Furthermore, we found driver genes with 5'R-loops and RNAPII pausing defects express significantly higher and genes with 3'R-loops and read-through transcription express significantly lower than genes without R-loops. These driver genes are enriched with chromosomal instability, Hippo-Merlin signaling Dysregulation, DNA damage response and TGF-β pathways, indicating R-loops accumulating at the 5' end of genes play oncogenic roles, whereas at the 3' end of genes play tumor-suppressive roles in tumorigenesis.

Spatial performance analysis in basketball with CART, random forest and extremely randomized trees

This paper proposes tools for spatial performance analysis in basketball. In detail, we aim at representing maps of the court visualizing areas with different levels of scoring probability of the analysed player or team. To do that, we propose the adoption of algorithmic modeling techniques. Firstly, following previous studies, we examine CART, highlighting strengths and weaknesses. With respect to what done in the past, here we propose the use of polar coordinates, which are more consistent with the basketball court geometry. In order to overcome CART’s drawbacks while maintaining its points of force, we propose to resort to CART-based ensemble learning algorithms, namely to Random Forest and Extremely Randomized Trees, which are shown to be able to give excellent results in terms of interpretation and robustness. Finally, an index is defined in order to measure the map’s graphical goodness, which can be used—jointly with measures of the out-of-sample error—to tune the algorithm’s parameters. The functioning of the proposed approaches is shown by the analysis of real data of the NBA regular season 2020/2021.

Combination of Hyperspectral and Machine Learning to Invert Soil Electrical Conductivity

An accurate estimation of soil electrical conductivity (EC) using hyperspectral techniques is of great significance for understanding the spatial distribution of solutes and soil salinization. Although spectral transformation has been widely used in data pre-processing, the performance of different pre-processing techniques (or combination methods) on different models of the same data set is still ambiguous. Moreover, extremely randomized trees (ERT) and light gradient boosting machine (LightGBM) models are new learning algorithms with good generalization performance (soil moisture and above-ground biomass), but are less studied in estimating soil salinity in the visible and near-infrared spectra. In this study, 130 soil EC data, soil measured hyperspectral data, topographic factors, conventional salinity indices such as Salinity Index 1, and two-band (2D) salinity indices such as ratio indices, were introduced. The five spectral pre-processing methods of standard normal variate (SNV), standard normal variate and detrend (SNV-DT), inverse (1/OR) (OR is original spectrum), inverse-log (Log(1/OR) and fractional order derivative (FOD) (range 0–2, with intervals of 0.25) were performed. A gradient boosting machine (GBM) was used to select sensitive spectral parameters. Models (extreme gradient boosting (XGBoost), LightGBM, random forest (RF), ERT, classification and regression tree (CART), and ridge regression (RR)) were used for inversion soil EC and model validation. The results reveal that the two-dimensional correlation coefficient highlighted EC more effectively than the one-dimensional. Under SNV and the second order derivative, the two-dimensional correlation coefficient increased by 0.286 and 0.258 compared to the one-dimension, respectively. The 13 characteristic factors of slope, NDI, SI-T, RI, profile curvature, DOA, plane curvature, SI (conventional), elevation, Int2, aspect, S1 and TWI provided 90% of the cumulative importance for EC using GBM. Among the six machine models, the ERT model performed the best for simulation (R2 = 0.98) and validation (R2 = 0.96). The ERT model showed the best performance among the EC estimation models from the reference data. The kriging map based on the ERT simulation showed a close relationship with the measured data. Our study selected the effective pre-processing methods (SNV and the 2 order derivative) using one- and two-dimensional correlation, 13 important factors and the ERT model for EC hyperspectral inversion. This provides a theoretical support for the quantitative monitoring of soil salinization on a larger scale using remote sensing techniques.

Explainable step-wise binary classification for the susceptibility assessment of geo-hydrological hazards

This research proposes a novel step-wise binary prediction framework for the susceptibility assessment of geo-hydrological hazards specific to floods and landslides. The framework of the study comprises two major steps: prediction of geo-hydrological hazard-prone locations (Step-1: hazard/non-hazard), and classification of geo-hydrological hazards by identifying the locations of floods and landslides separately (Step-2: floods/landslides). We used 1326 historically experienced hazard locations (i.e., 726 for floods and 690 for landslides) in the Kentucky River basin, United States, along with the 13 hazard conditioning factors. Extremely randomized trees (ERT) coupled with the particle swarm optimization (PSO) was adopted to provide an effective classification scheme. Based on the predictions of the ERT-PSO in the first step, correctly classified hazard instances were used in the second step of the prediction task to further deepen the machine learning application. The results revealed a strong agreement between the predicted and observed hazard locations with an AUROC of 0.8032 and 0.8845 for geo-hydrological hazard (Step-1) and flood/landslide classifications (Step-2), respectively. The proposed hybrid prediction framework introduced considerably accurate performance as 73.78% and 72.91% of the hazard and non-hazard classes were correctly identified at Step-1, respectively, while at Step-2, 72.31% of the flooding points and 84.85% of the landslide points were ascertained accurately. Overall findings emerged from Step-1 illustrated that nearly 10% of the entire basin is susceptible to geo-hydrological hazards with very high probability, whereas very low susceptible areas cover only 20% of the basin. A model-agnostic game-theory based SHapley Additive exPlanations (SHAP) algorithm was employed to anatomize the contribution of hazard conditioning factors on the incident outcome predictions aiding to increase the interpretability of the adopted methodology. The holistic approach adopted in the present research has significant potential in providing insights into the practical and theoretical grounds of the literature.

Using computational-intelligence algorithms and remote sensing data to optimize the locations of check dams to control sediment and runoff in Kandolus watershed, Mazandaran, Iran

Construction of check dams is one of the important measures in controlling soil erosion and sediment entering the reservoirs of dams. These structures, which provide water reduction and sedimentation by modifying the slope of the canal, play a significant role in preventing the transfer of sediment and pollutants to dam reservoirs, and water bodies. The most challenging task, however, is to identify suitable sites for check dams. In the present study, four machine learning algorithms (namely K-nearest neighbour (KNN), extreme gradient boosting (XGBoost), extremely randomized tree (ERT), and random forest (RF)) and high-resolution remote sensing data were used to find the optimal locations of check dams in the Kandolus watershed in Mazandaran Province (Iran). Sixteen topographical, hydrological, and geomorphic factors were used as inputs in the abovementioned machine learning approaches to identify appropriate locations of check dams. The models were evaluated using receiver operating characteristics (ROC) statistical analyses. The results showed that the RF, ERT, XGB, and KNN models could accurately identify the suitable locations of check dams with the area under curve (AUC) values of 0.93, 0.92, 0.83, and 0.82, respectively. According to the AUC values, RF had the highest accuracy to identify the suitable locations for check dams. This study demonstrate that the utilized artificial intelligence methods and remote sensing data can help land-use planners and water resource managers identify optimal locations of check dams to more efficiency control floods, reduce erosion and land degradation, and enhance groundwater recharge.

Assessment of Landslide Susceptibility of the Wiśnickie Foothills Mts. (The Flysch Carpathians, Poland) Using Selected Machine Learning Algorithms

Landslides are well-known phenomena that cause significant changes to the relief of an area’s terrain, often causing damage to technical infrastructure and loss of life. One of the possible means of reducing the negative impact of landslides on people’s lives or property is to recognize areas that are prone to their occurrence. The most common approach to this problem is preparing landslide susceptibility maps. These can factor in the actual location of landslides or the causal relationship between landslides and selected environmental factors. Creating a classification of landslide-prone areas is a challenging task when landslide density is not high and the area of analysis is large. We prepared shallow 10 m × 10 m resolution landslide susceptibility maps of the Wiśnickie Foothills (Western Carpathians, Poland) using eleven different machine learning algorithms derived from the Python libraries Scikit-learn and Imbalanced-Learn. The analyzed area is characterized by a mean density of 3.4 surficial landslides (composed of soils and rocks) per km2. We also compared different approaches to imbalanced sets of data: Logistic Regression, Naive Bayes, Random Forest, AdaBoost, Bagging, ExtraTrees (Extremely Randomized Trees), Easy Ensemble, Balanced Bagging, Balanced Random Forest, RUSBoost and a hybrid model combining Random Under Sampler and Multi-layer Perceptron algorithms. The environmental factors (slope inclination and aspect, distance from rivers, lithology, soil type and permeability, groundwater table depth, profile and plan curvature, mean annual rainfall) were categorized and divided into training (70%) and testing (30%) sets. Accuracy, recall, G-mean and area under receiver operating curve (AUC) were used to validate the quality of the models. The results confirmed that algorithms based on decision tree classifiers are suitable for preparing landslide susceptibility maps. We also found that methods that generate random undersampling subsets (Easy Ensemble, Balanced Bagging, RUSBoost) and ensemble methods (Bagging, AdaBoost, Extra-Trees) both yield very similar test results to those that use full sets of data for training. Relatively high-quality results can also be obtained by integrating the Random Under Sampler algorithm with the Multi-layer Perceptron algorithm.

Multi-epoch machine learning 1: Unravelling nature versus nurture for galaxy formation

Abstract We present a novel machine learning method for predicting the baryonic properties of dark matter only subhalos from N-body simulations. Our model is built using the extremely randomized tree (ERT) algorithm and takes subhalo properties over a wide range of redshifts as its input features. We train our model using the IllustrisTNG simulations to predict blackhole mass, gas mass, magnitudes, star formation rate, stellar mass, and metallicity. We compare the results of our method with a baseline model from previous works, and against a model that only considers the mass history of the subhalo. We find that our new model significantly outperforms both of the other models. We then investigate the predictive power of each input by looking at feature importance scores from the ERT algorithm. We produce feature importance plots for each baryonic property, and find that they differ significantly. We identify low redshifts as being most important for predicting star formation rate and gas mass, with high redshifts being most important for predicting stellar mass and metallicity, and consider what this implies for nature versus nurture. We find that the physical properties of galaxies investigated in this study are all driven by nurture and not nature. The only property showing a somewhat stronger impact of nature is the present-day star formation rate of galaxies. Finally we verify that the feature importance plots are discovering physical patterns, and that the trends shown are not an artefact of the ERT algorithm.

Evaluation of Various Tree-Based Ensemble Models for Estimating Solar Energy Resource Potential in Different Climatic Zones of China

Solar photovoltaic (PV) electricity generation is growing rapidly in China. Accurate estimation of solar energy resource potential (Rs) is crucial for siting, designing, evaluating and optimizing PV systems. Seven types of tree-based ensemble models, including classification and regression trees (CART), extremely randomized trees (ET), random forest (RF), gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost), gradient boosting with categorical features support (CatBoost) and light gradient boosting method (LightGBM), as well as the multi-layer perceotron (MLP) and support vector machine (SVM), were applied to estimate Rs using a k-fold cross-validation method. The three newly developed models (CatBoost, LighGBM, XGBoost) and GBDT model generally outperformed the other five models with satisfactory accuracy (R2 ranging from 0.893–0.916, RMSE ranging from 1.943–2.195 MJm−2d−1, and MAE ranging from 1.457–1.646 MJm−2d−1 on average) and provided acceptable model stability (increasing the percentage in testing RMSE over training RMSE from 8.3% to 31.9%) under seven input combinations. In addition, the CatBoost (12.3 s), LightGBM (13.9 s), XGBoost (20.5 s) and GBDT (16.8 s) exhibited satisfactory computational efficiency compared with the MLP (132.1 s) and SVM (256.8 s). Comprehensively considering the model accuracy, stability and computational time, the newly developed tree-based models (CatBoost, LighGBM, XGBoost) and commonly used GBDT model were recommended for modeling Rs in contrasting climates of China and possibly similar climatic zones elsewhere around the world. This study evaluated three newly developed tree-based ensemble models of estimating Rs in various climates of China, from model accuracy, model stability and computational efficiency, which provides a new look at indicators of evaluating machine learning methods.

BIM and machine learning in seismic damage prediction for non-structural exterior infill walls

Despite the seismic vulnerability of non-structural Exterior Infill Walls (EIWs), their resilient design has received minimal attention. This study addresses the issue by proposing a novel framework for predicting possible damage states of EIWs. The framework benefits from an automated combination of Building Information Modeling as a visualized 3D database of the building's components and the Machine Learning classification as the prediction engine. The framework's applicability is studied in a Proof of Concept example of the exterior walls of the buildings damaged in the 2017 earthquake in Kermanshah, Iran. The Extremely Randomized Trees classifier produced the best results for predicting new cases with an overall accuracy of %86. The trained model is used to develop a system for predicting the damage states of EIWs of a new building. The proposed framework works as a complementary tool in buildings' design and operation phases to enhance EIWs' seismic resilience.

Decreasing False-alarm Rates in CNN-based Solar Flare Prediction Using SDO/HMI Data

A hybrid two-stage machine-learning architecture that addresses the problem of excessive false positives (false alarms) in solar flare prediction systems is investigated. The first stage is a convolutional neural network (CNN) model based on the VGG-16 architecture that extracts features from a temporal stack of consecutive Solar Dynamics Observatory Helioseismic and Magnetic Imager magnetogram images to produce a flaring probability. The probability of flaring is added to a feature vector derived from the magnetograms to train an extremely randomized trees (ERT) model in the second stage to produce a binary deterministic prediction (flare/no-flare) in a 12 hr forecast window. To tune the hyperparameters of the architecture, a new evaluation metric is introduced: the “scaled True Skill Statistic.” It specifically addresses the large discrepancy between the true positive rate and the false positive rate in the highly unbalanced solar flare event training data sets. Through hyperparameter tuning to maximize this new metric, our two-stage architecture drastically reduces false positives by ≈48% without significantly affecting the true positives (reduction by ≈12%), when compared with predictions from the first-stage CNN alone. This, in turn, improves various traditional binary classification metrics sensitive to false positives, such as the precision, F1, and the Heidke Skill Score. The end result is a more robust 12 hr flare prediction system that could be combined with current operational flare-forecasting methods. Additionally, using the ERT-based feature-ranking mechanism, we show that the CNN output probability is highly ranked in terms of flare prediction relevance.