Extremely Randomized Trees Research Articles

Modelling of daily lake surface water temperature from air temperature: Extremely randomized trees (ERT) versus Air2Water, MARS, M5Tree, RF and MLPNN

Prediction of rivers and lakes water temperature plays an important role in hydrology, ecology, and water resources planning and management. Recently, machines learning approaches have been widely used for modelling water temperature, and the obtained results vary depending on the kind of models and the selections of the appropriates predictors. In the present paper, a new family of machines learning are proposed and compared to the famous air2stream model, using a large data set collected at 25 lakes in the northern part of Poland. The proposed models were: (i) the extremely randomized trees (ERT), (ii) the multivariate adaptive regression splines (MARS), (iii) the M5 Model tree (M5Tree), (iv) the random forest (RF), and (v) the multilayer perceptron neural network (MLPNN). The models were developed using the air temperature as input variables and the component of the Gregorian calendar (year, month and day) number. Results obtained were evaluated using several statistical indices: the root mean square error (RMSE), the mean absolute error (MAE), correlation coefficient (R) and Nash-Sutcliffe efficiency coefficient (NSE). Obtained results reveals that the air2stream model outperformed all other machines learning models and worked best with high accuracy at all the 25 lakes, and none of the ERT, MARS, M5Tree, RF and MLPNN models was able to provides an improvement of the water temperature prediction compared to the air2stream.

Implementation of ensemble methods on QSAR Study of NS3 inhibitor activity as anti-dengue agent

ABSTRACT Dengue fever is a disease transmitted by infected mosquitoes. This disease spreads in several countries, especially those with a tropical climate. To date, there is no specific drug that can be used to treat dengue. Use of clinically investigated drugs, such as Balapiravir, is still not effective in inhibiting the activity of virus replication. The design of a drug candidate can be performed by using the non-structural protein 3 (NS3) as target. This study aimed to develop QSAR models to predict the inhibitory activity class of NS3 inhibitors. The classification was performed by using feature importance analysis for selecting the descriptors and three ensemble methods, i.e. random forest (RF), adaptive boosting (AdaBoost), and extremely randomized trees (ERT), for model design and prediction. Hyperparameter tuning was performed to improve the performance of the models. Based on the results, we found that model 9, developed from ERT produced the best performance with values of accuracy and AUC equal to 0.73 and 0.82, respectively. Use of y-scrambling method allowed us to confirm that the model was not related to the chance correlation.

A Novel Random Split Point Procedure Using Extremely Randomized (Extra) Trees Ensemble Method for Human Activity Recognition

INTRODUCTION: Automatic detection and recognition of various human physical movements while performing daily life activities such as walking, jogging, running, sitting, standing etc. are usually considered as Activity Recognition (AR). AR is a prominent research area in many applications, such as elderly care, security and surveillance, smart homes, health and fitness. Extremely Randomized Trees Classifier (ET Classifier) is a type of ensemble learning technique used in Activity Recognition, which clusters several different decision trees into a forest from a single learning set and gives the classification result. But it suffers from high variance and over-fitting problem due to high inter-dependency among hyperparameters during model building.OBJECTIVES: The primary objective of this paper is to propose a novel RandomSplitPoint procedure for Extra tree classifier to make the existing approach more robust, less variance, less computational time in obtaining optimal split points and faster in model building. This approach generates K random split points from all the candidate features of the dataset and selects the best split point based on the maximum score obtained by information gain measure.METHODS: In the proposed method to improve the randomization and accuracy of AR system, a novel random split-point procedure for ET classifier is proposed. This approach reduces the bias-variance problem induced due to the three hyperparameters such as K, nmin and M used in split-point procedure of existing ET classifier (K : number of randomly selected attributes at each node, nmin : minimum sample size for splitting a node, M : number of decision trees for ensemble). This approach generates K random split points from all the candidate features of the dataset and selects the best split point based on the maximum score obtained by information gain measure.RESULTS: The proposed approach is experimented with two public AR datasets HAR and HAPT (UCI Machine Learning Repository) containing 6 and 12 activities respectively. In HAR dataset, smartphone sensed sensor signals of 3 static and 3 dynamic human daily activities are there, where as in HAPT dataset apart from these 6 daily activities, 6 postural transitions data is available. Experimental results and comparative analysis show that the proposed method outperforms over other existing techniques with an accuracy of 94.16% for HAR dataset and 92.63% for HAPT dataset. It also takes less computational time in finding optimal split-points and less model building time.CONCLUSION: AR systems can be used as an intelligent system in healthcare to monitor the behaviour of healthy people by recognizing their daily activities. These systems also help in early detection of some chronic diseases and improve the quality of life. In this paper, an attempt is made to improve the accuracy of Activity Recognition over some existing methods.

Heat load prediction of residential buildings based on discrete wavelet transform and tree-based ensemble learning

Currently, the high energy consumption of district heating system represents a problem. Thus, accurate prediction of future heat load is the key to ensure energy saving and improve system operation efficiency. However, the accuracy of the prediction model based on single machine learning algorithms need to be improved, and the hybrid prediction model suffers from redundant inputs. The objective of this study was to obtain the most accurate prediction of the short-term heat load consumption. The discrete wavelet transform, two tree-based ensemble learning algorithms, extremely randomized trees regression and gradient boosting decision tree were used to establish hybrid models for predicting the heat load 1 h in advance. Pearson and least absolute shrinkage and selection operator (LASSO) methods were used to optimize the feature set, including system parameters, meteorological parameters and time steps. This resulted in four feature sets. In addition, the effect of the historical heat load on the performance of the prediction model was analysed. Actual operation data of an established heating system were collected to test the performance of the models. The results were then compared with those of the models established by artificial neural network and support vector regression. The results show that the historical heat load had a significant impact on the prediction accuracy of the heat load of the upcoming hour, and the extreme random tree model based on discrete wavelet transform showed a better prediction performance and lower model complexity in case of using the feature set selected by the LASSO method.

A frequency-domain machine learning method for dual-calibrated fMRI mapping of oxygen extraction fraction (OEF) and cerebral metabolic rate of oxygen consumption (CMRO2).

Magnetic resonance imaging (MRI) offers the possibility to non-invasively map the brain's metabolic oxygen consumption (CMRO2), which is essential for understanding and monitoring neural function in both health and disease. However, in depth study of oxygen metabolism with MRI has so far been hindered by the lack of robust methods. One MRI method of mapping CMRO2 is based on the simultaneous acquisition of cerebral blood flow (CBF) and blood oxygen level dependent (BOLD) weighted images during respiratory modulation of both oxygen and carbon dioxide. Although this dual-calibrated methodology has shown promise in the research setting, current analysis methods are unstable in the presence of noise and/or are computationally demanding. In this paper, we present a machine learning implementation for the multi-parametric assessment of dual-calibrated fMRI data. The proposed method aims to address the issues of stability, accuracy, and computational overhead, removing significant barriers to the investigation of oxygen metabolism with MRI. The method utilizes a time-frequency transformation of the acquired perfusion and BOLD-weighted data, from which appropriate feature vectors are selected for training of machine learning regressors. The implemented machine learning methods are chosen for their robustness to noise and their ability to map complex non-linear relationships (such as those that exist between BOLD signal weighting and blood oxygenation). An extremely randomized trees (ET) regressor is used to estimate resting blood flow and a multi-layer perceptron (MLP) is used to estimate CMRO2 and the oxygen extraction fraction (OEF). Synthetic data with additive noise are used to train the regressors, with data simulated to cover a wide range of physiologically plausible parameters. The performance of the implemented analysis method is compared to published methods both in simulation and with in-vivo data (n = 30). The proposed method is demonstrated to significantly reduce computation time, error, and proportional bias in both CMRO2 and OEF estimates. The introduction of the proposed analysis pipeline has the potential to not only increase the detectability of metabolic difference between groups of subjects, but may also allow for single subject examinations within a clinical context.

Object-based landfast sea ice detection over West Antarctica using time series ALOS PALSAR data

Landfast sea ice (fast ice) is an important feature prevalent around the Antarctic coast, which is affected by climate change and energy exchanges with the atmosphere and ocean. This study proposed a method for detection of the West Antarctic fast ice using the Advanced Land Observing Satellite Phased Array L-band SAR (ALOS PALSAR) images. The algorithm has combined image segmentation, image correlation analysis, and machine learning techniques (i.e., random forest (RF), extremely randomized trees (ERT), and logistic regression (LR)). We used SAR images with a baseline of 5 days that are not in the same orbit but overlap each other as overlaps between swaths in adjacent orbits are often available in the polar regions. The underlying assumption for the proposed fast ice detection algorithm is that fast ice regions in SAR images with a time interval of 5 days are highly correlated. The object-based approach proposed in this study was well suited to high-resolution SAR images in deriving spatially homogeneous fast ice regions. The image segmentation results using the optimized parameters showed a distinct difference in the backscatter temporal evolution between fast ice and pack ice regions. Correlation and STD of backscattering coefficients were found to be the most significant variables for the object-based fast ice detection from two temporally separated images. In overall, the quantitative and qualitative evaluation demonstrated that the algorithm was an effective approach to detect fast ice with high accuracies. The models well detected various fast ice regions in the West Antarctica but misclassified some objects. The misclassifications occurred toward the edge of fast ice regions with relatively rapid changes in backscattering between both data acquisitions. On the other hand, few fast ice objects were misclassified as uniform backscattering over time occurred by chance on very small objects far from the coast. Very old multi-year fast ice regions with high backscattered signals were also a source for some misclassifications. This may be due to the sensitivity of L-band to snow structure to some extent and a thinner ice over the region with either ice growth (no deformation) or closing (slight deformation) between both images. Heavy snow load on the ice could be another error source for some misclassification as well. The approach allowed for the reliable detection of fast ice regions by using L-band SAR images with a small local incidence angle difference.

Fusion of Five Satellite-Derived Products Using Extremely Randomized Trees to Estimate Terrestrial Latent Heat Flux over Europe

An accurate estimation of spatially and temporally continuous latent heat flux (LE) is essential in the assessment of surface water and energy balance. Various satellite-derived LE products have been generated to enhance the simulation of terrestrial LE, yet each individual LE product shows large discrepancies and uncertainties. Our study used Extremely Randomized Trees (ETR) to fuse five satellite-derived terrestrial LE products to reduce uncertainties from the individual products and improve terrestrial LE estimations over Europe. The validation results demonstrated that the estimation using the ETR fusion method increased the R2 of five individual LE products (ranging from 0.53 to 0.61) to 0.97 and decreased the RMSE (ranging from 26.37 to 33.17 W/m2) to 5.85 W/m2. Compared with three other machine learning fusion models, Gradient Boosting Regression Tree (GBRT), Random Forest (RF), and Gaussian Process Regression (GPR), ETR exhibited the best performance in terms of both training and validation accuracy. We also applied the ETR fusion method to implement the mapping of average annual terrestrial LE over Europe at a resolution of 0.05 ◦ in the period from 2002 to 2005. When compared with global LE products such as the Global Land Surface Satellite (GLASS) and the Moderate Resolution Imaging Spectroradiometer (MODIS), the fusion LE using ETR exhibited a relatively small gap, which confirmed that it is reasonable and reliable for the estimation of the terrestrial LE over Europe.

REAL-Time Smartphone Activity Classification Using Inertial Sensors-Recognition of Scrolling, Typing, and Watching Videos While Sitting or Walking.

By developing awareness of smartphone activities that the user is performing on their smartphone, such as scrolling feeds, typing and watching videos, we can develop application features that are beneficial to the users, such as personalization. It is currently not possible to access real-time smartphone activities directly, due to standard smartphone privileges and if internal movement sensors can detect them, there may be implications for access policies. Our research seeks to understand whether the sensor data from existing smartphone inertial measurement unit (IMU) sensors (triaxial accelerometers, gyroscopes and magnetometers) can be used to classify typical human smartphone activities. We designed and conducted a study with human participants which uses an Android app to collect motion data during scrolling, typing and watching videos, while walking or seated and the baseline of smartphone non-use, while sitting and walking. We then trained a machine learning (ML) model to perform real-time activity recognition of those eight states. We investigated various algorithms and parameters for the best accuracy. Our optimal solution achieved an accuracy of 78.6% with the Extremely Randomized Trees algorithm, data sampled at 50 Hz and 5-s windows. We conclude by discussing the viability of using IMU sensors to recognize common smartphone activities.

An improved Stacking framework for stock index prediction by leveraging tree-based ensemble models and deep learning algorithms

Stock price index is an essential component of financial systems and indicates the economic performance in the national level. Even if a small improvement in its forecasting performance will be highly profitable and meaningful. This manuscript input technical features together with macroeconomic indicators into an improved Stacking framework for predicting the direction of the stock price index in respect of the price prevailing some time earlier, if necessary, a month. Random forest (RF), extremely randomized trees (ERT), extreme gradient boosting (XGBoost) and light gradient boosting machine (LightGBM), which pertain to the tree-based algorithms, and recurrent neural networks (RNN), bidirectional RNN, RNN with long short-term memory (LSTM) and gated recurrent unit (GRU) layer, which pertain to the deep learning algorithms, are stacked as base classifiers in the first layer. Cross-validation method is then implemented to iteratively generate the input for the second level classifier in order to prevent overfitting. In the second layer, logistic regression, as well as its regularized version, are employed as meta-classifiers to identify the unique learning pattern of the base classifiers. Empirical results over three major U.S. stock indices indicate that our improved Stacking method outperforms state-of-the-art ensemble learning algorithms and deep learning models, achieving a higher level of accuracy, F-score and AUC value. Besides, another contribution in our research paper is the design of a Lasso (least absolute shrinkage and selection operator) based meta-classifier that is capable of automatically weighting/selecting the optimal base learners for the forecasting task. Our findings provide an integrated Stacking framework in the financial area.

Crime risk analysis through big data algorithm with urban metrics

Crime is pervasive all around the world. Understanding the influence of social features on crime occurrences of a city is a hot topic among researchers. Correlations between crime and other social characteristics have been studied by large amounts of statistical models, including Ordinary Least Square (OLS) linear regression model, Random Forest (RF) regression model, Artificial Neural Network (ANN) model and so on. However, results of these studies, such as the prediction accuracy, are not satisfying and many contradictory conclusions are achieved in previous research works. These controversies are triggered by several factors, including the non-Gaussian distributions and multicollinearity of urban social data, inaccuracy and inadequacy of the processed data, etc. To fill these gaps, we analyzed the influence of 18 urban indicators within 6 categories including geography, economy, education, housing, urbanization and population structure on crime risk in China’s major prefecture-level cities by year. We used the big data algorithm, Least Absolute Shrinkage and Selection Operator (LASSO) and Extremely-randomized Trees (Extra-Trees), to predict the crime risk and quantify the influence of urban parameters on crime. 83% of accuracy on crime risk prediction can be obtained from our fitted model and the importance of urban indicators is ranked. Results show that area of land used for living, number of subscribers of mobile telephone, employed population are the three main factors on the crime occurrences in China. Our research makes contributions to better understanding of the effects of urban indicators on crime in a socialist nation, and providing instructions and strategies for crime prediction and crime rate control with governments, in this big-data era.

Network inference with ensembles of bi-clustering trees

BackgroundNetwork inference is crucial for biomedicine and systems biology. Biological entities and their associations are often modeled as interaction networks. Examples include drug protein interaction or gene regulatory networks. Studying and elucidating such networks can lead to the comprehension of complex biological processes. However, usually we have only partial knowledge of those networks and the experimental identification of all the existing associations between biological entities is very time consuming and particularly expensive. Many computational approaches have been proposed over the years for network inference, nonetheless, efficiency and accuracy are still persisting open problems. Here, we propose bi-clustering tree ensembles as a new machine learning method for network inference, extending the traditional tree-ensemble models to the global network setting. The proposed approach addresses the network inference problem as a multi-label classification task. More specifically, the nodes of a network (e.g., drugs or proteins in a drug-protein interaction network) are modelled as samples described by features (e.g., chemical structure similarities or protein sequence similarities). The labels in our setting represent the presence or absence of links connecting the nodes of the interaction network (e.g., drug-protein interactions in a drug-protein interaction network).ResultsWe extended traditional tree-ensemble methods, such as extremely randomized trees (ERT) and random forests (RF) to ensembles of bi-clustering trees, integrating background information from both node sets of a heterogeneous network into the same learning framework. We performed an empirical evaluation, comparing the proposed approach to currently used tree-ensemble based approaches as well as other approaches from the literature. We demonstrated the effectiveness of our approach in different interaction prediction (network inference) settings. For evaluation purposes, we used several benchmark datasets that represent drug-protein and gene regulatory networks. We also applied our proposed method to two versions of a chemical-protein association network extracted from the STITCH database, demonstrating the potential of our model in predicting non-reported interactions.ConclusionsBi-clustering trees outperform existing tree-based strategies as well as machine learning methods based on other algorithms. Since our approach is based on tree-ensembles it inherits the advantages of tree-ensemble learning, such as handling of missing values, scalability and interpretability.

Identification of discriminatory antibiotic resistance genes among environmental resistomes using extremely randomized tree algorithm

BackgroundThe interconnectivities of built and natural environments can serve as conduits for the proliferation and dissemination of antibiotic resistance genes (ARGs). Several studies have compared the broad spectrum of ARGs (i.e., “resistomes”) in various environmental compartments, but there is a need to identify unique ARG occurrence patterns (i.e., “discriminatory ARGs”), characteristic of each environment. Such an approach will help to identify factors influencing ARG proliferation, facilitate development of relative comparisons of the ARGs distinguishing various environments, and help pave the way towards ranking environments based on their likelihood of contributing to the spread of clinically relevant antibiotic resistance. Here we formulate and demonstrate an approach using an extremely randomized tree (ERT) algorithm combined with a Bayesian optimization technique to capture ARG variability in environmental samples and identify the discriminatory ARGs. The potential of ERT for identifying discriminatory ARGs was first evaluated using in silico metagenomic datasets (simulated metagenomic Illumina sequencing data) with known variability. The application of ERT was then demonstrated through analyses using publicly available and in-house metagenomic datasets associated with (1) different aquatic habitats (e.g., river, wastewater influent, hospital effluent, and dairy farm effluent) to compare resistomes between distinct environments and (2) different river samples (i.e., Amazon, Kalamas, and Cam Rivers) to compare resistome characteristics of similar environments.ResultsThe approach was found to readily identify discriminatory ARGs in the in silico datasets. Also, it was not found to be biased towards ARGs with high relative abundance, which is a common limitation of feature projection methods, and instead only captured those ARGs that elicited significant profiles. Analyses of publicly available metagenomic datasets further demonstrated that the ERT approach can effectively differentiate real-world environmental samples and identify discriminatory ARGs based on pre-defined categorizing schemes.ConclusionsHere a new methodology was formulated to characterize and compare variances in ARG profiles between metagenomic data sets derived from similar/dissimilar environments. Specifically, identification of discriminatory ARGs among samples representing various environments can be identified based on factors of interest. The methodology could prove to be a particularly useful tool for ARG surveillance and the assessment of the effectiveness of strategies for mitigating the spread of antibiotic resistance. The python package is hosted in the Git repository: https://github.com/gaarangoa/ExtrARG

Machine-assisted Semi-Simulation Model (MSSM): Estimating Galactic Baryonic Properties from Their Dark Matter Using A Machine Trained on Hydrodynamic Simulations

Abstract We present a pipeline to estimate baryonic properties of a galaxy inside a dark matter (DM) halo in DM-only simulations using a machine trained on high-resolution hydrodynamic simulations. As an example, we use the IllustrisTNG hydrodynamic simulation of a (75 h−1Mpc)3 volume to train our machine to predict e.g., stellar mass and star formation rate in a galaxy-sized halo based purely on its DM content. An extremely randomized tree (ERT) algorithm is used together with multiple novel improvements we introduce here such as a refined error function in machine training and two-stage learning. Aided by these improvements, our model demonstrates a significantly increased accuracy in predicting baryonic properties compared to prior attempts — in other words, the machine better mimics IllustrisTNG’s galaxy-halo correlation. By applying our machine to the MultiDark-Planck DM-only simulation of a large (1 h−1Gpc)3 volume, we then validate the pipeline that rapidly generates a galaxy catalogue from a DM halo catalogue using the correlations the machine found in IllustrisTNG. We also compare our galaxy catalogue with the ones produced by popular semi-analytic models (SAMs). Our so-called machine-assisted semi-simulation model (MSSM) is shown to be largely compatible with SAMs, and may become a promising method to transplant the baryon physics of galaxy-scale hydrodynamic calculations onto a larger-volume DM-only run. We discuss the benefits that machine-based approaches like this entail, as well as suggestions to raise the scientific potential of such approaches.

Improving plankton image classification using context metadata

AbstractAdvances in both hardware and software are enabling rapid proliferation of in situ plankton imaging methods, requiring more effective machine learning approaches to image classification. Deep Learning methods, such as convolutional neural networks (CNNs), show marked improvement over traditional feature‐based supervised machine learning algorithms, but require careful optimization of hyperparameters and adequate training sets. Here, we document some best practices in applying CNNs to zooplankton and marine snow images and note where our results differ from contemporary Deep Learning findings in other domains. We boost the performance of CNN classifiers by incorporating metadata of different types and illustrate how to assimilate metadata beyond simple concatenation. We utilize both geotemporal (e.g., sample depth, location, time of day) and hydrographic (e.g., temperature, salinity, chlorophyll a) metadata and show that either type by itself, or both combined, can substantially reduce error rates. Incorporation of context metadata also boosts performance of the feature‐based classifiers we evaluated: Random Forest, Extremely Randomized Trees, Gradient Boosted Classifier, Support Vector Machines, and Multilayer Perceptron. For our assessments, we use an original data set of 350,000 in situ images (roughly 50% marine snow and 50% non‐snow sorted into 26 categories) from a novel in situ Zooglider. We document asymptotically increasing performance with more computationally intensive techniques, such as substantially deeper networks and artificially augmented data sets. Our best model achieves 92.3% accuracy with our 27‐class data set. We provide guidance for further refinements that are likely to provide additional gains in classifier accuracy.

Reconstruction of Ocean Color Data Using Machine Learning Techniques in Polar Regions: Focusing on Off Cape Hallett, Ross Sea

The most problematic issue in the ocean color application is the presence of heavy clouds, especially in polar regions. For that reason, the demand for the ocean color application in polar regions is increased. As a way to overcome such issues, we conducted the reconstruction of the chlorophyll-a concentration (CHL) data using the machine learning-based models to raise the usability of CHL data. This analysis was first conducted on a regional scale and focused on the biologically-valued Cape Hallett, Ross Sea, Antarctica. Environmental factors and geographical information associated with phytoplankton dynamics were considered as predictors for the CHL reconstruction, which were obtained from cloud-free microwave and reanalysis data. As the machine learning models used in the present study, the ensemble-based models such as Random forest (RF) and Extremely randomized tree (ET) were selected with 10-fold cross-validation. As a result, both CHL reconstructions from the two models showed significant agreement with the standard satellite-derived CHL data. In addition, the reconstructed CHLs were close to the actual CHL value even where it was not observed by the satellites. However, there is a slight difference between the CHL reconstruction results from the RF and the ET, which is likely caused by the difference in the contribution of each predictor. In addition, we examined the variable importance for the CHL reconstruction quantitatively. As such, the sea surface and atmospheric temperature, and the photosynthetically available radiation have high contributions to the model developments. Mostly, geographic information appears to have a lower contribution relative to environmental predictors. Lastly, we estimated the partial dependences for the predictors for further study on the variable contribution and investigated the contributions to the CHL reconstruction with changes in the predictors.

A Cascade Ensemble Learning Model for Human Activity Recognition with Smartphones.

Human activity recognition (HAR) has gained lots of attention in recent years due to its high demand in different domains. In this paper, a novel HAR system based on a cascade ensemble learning (CELearning) model is proposed. Each layer of the proposed model is comprised of Extremely Gradient Boosting Trees (XGBoost), Random Forest, Extremely Randomized Trees (ExtraTrees) and Softmax Regression, and the model goes deeper layer by layer. The initial input vectors sampled from smartphone accelerometer and gyroscope sensor are trained separately by four different classifiers in the first layer, and the probability vectors representing different classes to which each sample belongs are obtained. Both the initial input data and the probability vectors are concatenated together and considered as input to the next layer’s classifiers, and eventually the final prediction is obtained according to the classifiers of the last layer. This system achieved satisfying classification accuracy on two public datasets of HAR based on smartphone accelerometer and gyroscope sensor. The experimental results show that the proposed approach has gained better classification accuracy for HAR compared to existing state-of-the-art methods, and the training process of the model is simple and efficient.

Ensemble learning algorithm based on multi-parameters for sleep staging.

The aim of this study is to propose a high-accuracy and high-efficiency sleep staging algorithm using single-channel electroencephalograms (EEGs). The process consists four parts: signal preprocessing, feature extraction, feature selection, and classification algorithms. In the preconditioning of EEG, wavelet function and IIR filter are used for noise reduction. In feature selection, 15 feature algorithms in time domain, time-frequency domain, and nonlinearity are selected to obtain 30 feature parameters. Feature selection is very important for eliminating irrelevant and redundant features. Feature selection algorithms as Fisher score, Sequential Forward Selection (SFS), Sequential Floating Forward Selection (SFFS), and Fast Correlation-Based Filter Solution (FCBF) were used. The paper establishes a new ensemble learning algorithm based on stacking model. The basic layers are k-Nearest Neighbor (KNN), Random Forest (RF), Extremely Randomized Trees (ERT), Multi-layer Perceptron (MLP), and Extreme Gradient Boosting (XGBoost) and the second layer is a Logistic regression. Comparing classification of RF, Gradient Boosting Decision Tree (GBDT), and XGBoost, the accuracies and kappa coefficients are 96.67% and 0.96 using the proposed method. It is higher than other classification algorithms.The results show that the proposed method can accurately sleep staging using single-channel EEG and has a high ability to predict sleep staging. Graphical abstract.

An Ensemble Cascading Extremely Randomized Trees Framework for Short-Term Traffic Flow Prediction

An Ensemble Cascading Extremely Randomized Trees Framework for Short-Term Traffic Flow Prediction

Gaia GraL: Gaia DR2 Gravitational Lens Systems

Aims. In this work, we aim to provide a reliable list of gravitational lens candidates based on a search performed over the entire Gaia Data Release 2 (Gaia DR2). We also aim to show that the astrometric and photometric information coming from the Gaia satellite yield sufficient insights for supervised learning methods to automatically identify strong gravitational lens candidates with an efficiency that is comparable to methods based on image processing. Methods. We simulated 106 623 188 lens systems composed of more than two images, based on a regular grid of parameters characterizing a non-singular isothermal ellipsoid lens model in the presence of an external shear. These simulations are used as an input for training and testing our supervised learning models consisting of extremely randomized trees (ERTs). These trees are finally used to assign to each of the 2 129 659 clusters of celestial objects extracted from the Gaia DR2 a discriminant value that reflects the ability of our simulations to match the observed relative positions and fluxes from each cluster. Once complemented with additional constraints, these discriminant values allow us to identify strong gravitational lens candidates out of the list of clusters. Results. We report the discovery of 15 new quadruply-imaged lens candidates with angular separations of less than 6″ and assess the performance of our approach by recovering 12 of the 13 known quadruply-imaged systems with all their components detected in Gaia DR2 with a misclassification rate of fortuitous clusters of stars as lens systems that is below 1%. Similarly, the identification capability of our method regarding quadruply-imaged systems where three images are detected in Gaia DR2 is assessed by recovering 10 of the 13 known quadruply-imaged systems having one of their constituting images discarded. The associated misclassification rate varies between 5.83% and 20%, depending on the image we decided to remove.

Prediction of Material Removal Rate for Chemical Mechanical Planarization Using Decision Tree-Based Ensemble Learning

Chemical mechanical planarization (CMP) has been widely used in the semiconductor industry to create planar surfaces with a combination of chemical and mechanical forces. A CMP process is very complex because several chemical and mechanical phenomena (e.g., surface kinetics, electrochemical interfaces, contact mechanics, stress mechanics, hydrodynamics, and tribochemistry) are involved. Predicting the material removal rate (MRR) in a CMP process with sufficient accuracy is essential to achieving uniform surface finish. While physics-based methods have been introduced to predict MRRs, little research has been reported on monitoring and predictive modeling of the MRR in CMP. This paper presents a novel decision tree-based ensemble learning algorithm that can train the predictive model of the MRR. The stacking technique is used to combine three decision tree-based learning algorithms, including the random forests (RF), gradient boosting trees (GBT), and extremely randomized trees (ERT), via a meta-regressor. The proposed method is demonstrated on the data collected from a CMP tool that removes material from the surface of wafers. Experimental results have shown that the decision tree-based ensemble learning algorithm using stacking can predict the MRR in the CMP process with very high accuracy.