Rotation Forest Ensemble Research Articles

Predicting sewer structural condition using hybrid machine learning algorithms

ABSTRACT Predicting the structural condition of sewer pipes plays a vital role in the predictive maintenance of sewer pipes and renewal plans of many water utilities. This study explores the simultaneous utilization of physical and environmental features of sewer pipes in sewer structural condition prediction. Three (3) hybrid machine learning models which are the combination of Bagging (BG), Dagging (DG), and Rotation Forest (RotF) ensembles with a J48 Decision Tree (J48DT) based classifier were used to predict sewer pipe conditions in Ålesund city, Norway. The classification performance of the machine learning models was evaluated using the area under the receiver operating characteristic (AUC-ROC) and the area under the precision-recall (AUC-PRC) curves. The RotF-J48DT model had the highest (AUC-ROC = 0.857, AUC-PRC = 0.918) values, followed by the BG-J48DT, and the base classifier J48DT. The RotF-J48DT hybrid model should be considered when predicting the condition of sewer pipes in the study area.

An Efficient Prediction System for Coronary Heart Disease Risk Using Selected Principal Components and Hyperparameter Optimization

Medical science-related studies have reinforced that the prevalence of coronary heart disease which is associated with the heart and blood vessels has been the most significant cause of health loss and death globally. Recently, data mining and machine learning have been used to detect diseases based on the unique characteristics of a person. However, these techniques have often posed challenges due to the complexity in understanding the objective of the datasets, the existence of too many factors to analyze as well as lack of performance accuracy. This research work is of two-fold effort: firstly, feature extraction and selection. This entails extraction of the principal components, and consequently, the Correlation-based Feature Selection (CFS) method was applied to select the finest principal components of the combined (Cleveland and Statlog) heart dataset. Secondly, by applying datasets to three single and three ensemble classifiers, the best hyperparameters that reflect the pre-eminent predictive outcomes were investigated. The experimental result reveals that hyperparameter optimization has improved the accuracy of all the models. In the comparative studies, the proposed work outperformed related works with an accuracy of 97.91%, and an AUC of 0.996 by employing six optimal principal components selected from the CFS method and optimizing parameters of the Rotation Forest ensemble classifier.

Ensemble hybrid machine learning methods for gully erosion susceptibility mapping: K-fold cross validation approach

Gully erosion is one of the important problems creating barrier to agricultural development. The present research used the radial basis function neural network (RBFnn) and its ensemble with random sub-space (RSS) and rotation forest (RTF) ensemble Meta classifiers for the spatial mapping of gully erosion susceptibility (GES) in Hinglo river basin. 120 gullies were marked and grouped into four-fold. A total of 23 factors including topographical, hydrological, lithological, and soil physio-chemical properties were effectively used. GES maps were built by RBFnn, RSS-RBFnn, and RTF-RBFnn models. The very high susceptibility zone of RBFnn, RTF-RBFnn and RSS-RBFnn models covered 6.75%, 6.72% and 6.57% in Fold-1, 6.21%, 6.10% and 6.09% in Fold-2, 6.26%, 6.13% and 6.05% in Fold-3 and 7%, 6.975% and 6.42% in Fold-4 of the basin. Receiver operating characteristics (ROC) curve and statistical techniques such as mean-absolute-error (MAE), root-mean-absolute-error (RMSE) and relative gully density area (R-index) methods were used for evaluating the GES maps. The results of the ROC, MAE, RMSE and R-index methods showed that the models of susceptibility to gully erosion have excellent predictive efficiency. The simulation results based on machine learning are satisfactory and outstanding and could be used to forecast the areas vulnerable to gully erosion.

Classification of epileptic seizure using rotation forest ensemble method with 1D-LBP feature extraction

In this article, an approach for extracting and choosing aspects of unprocessed EEG signals using the one-dimensional local binary pattern (1D-LBP) is presented. In view of making the right decision on different issues, notably in the medical applications like epilepsy diagnosis, a functional approach to extracting the best aspects of EEG signals has been developed. There are two major steps in the suggested method: First, features are extracted and selected using enhanced 1D-LBP model that combines 1D-LBP adjacent models with PCA (1D-LBPc2p). The second step involves classification using random rotating forestensemble classification algorithm. A comparison of the proposed methodwith 13 different reported methodologies, comprising uniform and non-uniform 1D-LBP, is carried out. In comparison to other models and classifiers, the results show that thetechnique described in proposed method has high detection rate as well as competence by giving high accuracy. The method provided in this paper can be used to extract features and pick EEG signals and data sets from a variety of sources.

Rotation Forest for multi-target regression

The prediction of multiple numeric outputs at the same time is called multi-target regression (MTR), and it has gained attention during the last decades. This task is a challenging research topic in supervised learning because it poses additional difficulties to traditional single-target regression (STR), and many real-world problems involve the prediction of multiple targets at once. One of the most successful approaches to deal with MTR, although not the only one, consists in transforming the problem in several STR problems, whose outputs will be combined building up the MTR output. In this paper, the Rotation Forest ensemble method, previously proposed for single-label classification and single-target regression, is adapted to MTR tasks and tested with several regressors and data sets. Our proposal rotates the input space in an efficient and novel fashion, avoiding extra rotations forced by MTR problem decomposition. Four approaches for MTR are used: single-target (ST), stacked-single target (SST), Ensembles of Regressor Chains (ERC), and Multi-target Regression via Quantization (MRQ). For assessing the benefits of the proposal, a thorough experimentation with 28 MTR data sets and statistical tests are used, concluding that Rotation Forest, adapted by means of these approaches, outperforms other popular ensembles, such as Bagging and Random Forest.

EEG-based emotion recognition using tunable Q wavelet transform and rotation forest ensemble classifier

Emotion recognition by artificial intelligence (AI) is a challenging task. A wide variety of research has been done, which demonstrated the utility of audio, imagery, and electroencephalography (EEG) data for automatic emotion recognition. This paper presents a new automated emotion recognition framework, which utilizes electroencephalography (EEG) signals. The proposed method is lightweight, and it consists of four major phases, which include: a reprocessing phase, a feature extraction phase, a feature dimension reduction phase, and a classification phase. A discrete wavelet transforms (DWT) based noise reduction method, which is hereby named multi scale principal component analysis (MSPCA), is utilized during the pre-processing phase, where a Symlets-4 filter is utilized for noise reduction. A tunable Q wavelet transform (TQWT) is utilized as feature extractor. Six different statistical methods are used for dimension reduction. In the classification step, rotation forest ensemble (RFE) classifier is utilized with different classification algorithms such as k-Nearest Neighbor (k-NN), support vector machine (SVM), artificial neural network (ANN), random forest (RF), and four different types of the decision tree (DT) algorithms. The proposed framework achieves over 93 % classification accuracy with RFE + SVM. The results clearly show that the proposed TQWT and RFE based emotion recognition framework is an effective approach for emotion recognition using EEG signals.

Landslide susceptibility mapping using state-of-the-art machine learning ensembles

This study propose a new approach through which the landslide susceptibility in Quang Nam (Vietnam) will be estimated using the best model among the following algorithms: Decision Table (DT), Naïve Bayes (NB), Decision Table - Naïve Bayes (DTNB), Bagging Ensemble, Cascade Generalization Ensemble, Dagging Ensemble, Decorate Ensemble, MultiBoost Ensemble, MultiScheme Ensemble, Real Ada Boost Ensemble, Rotation Forest Ensemble, Random Sub Space Ensemble. In this regard, a map with 1130 landslide, was created and further partitioned into training (70%) and testing (30%) locations. The correlation-based features selections (CFS) method was used to select a number of 15 landslide influencing factors. Landslide locations, included in the training sample, and the landslide predictors were used as input data in order to run the above mentioned models. Kappa index, Accuracy (%) and ROC curve were employed to estimate the model’s performance and to test the outcomes provided by the models. Among the eleven machine learning algorithms, Random Sub Space Decision Table Naïve Bayes (RSSDTNB) was the most performant model with an AUC = 0.839, Accuracy = 76.55% and Kappa Index = 0.531. Therefore, this algorithm was involved in the estimation of landslide susceptibility. The Success Rate (AUC = 0.815) and Prediction Rate (AUC = 0.826) revealed the achievement of high-quality results.

Groundwater Potential Mapping Using GIS-Based Hybrid Artificial Intelligence Methods.

Groundwater is one of the major valuable water resources for the use of communities, agriculture, and industries. In the present study, we have developed three novel hybrid artificial intelligence (AI) models which is a combination of modified RealAdaBoost (MRAB), bagging (BA), and rotation forest (RF) ensembles with functional tree (FT) base classifier for the groundwater potential mapping (GPM) in the basaltic terrain at DakLak province, Highland Centre, Vietnam. Based on the literature survey, these proposed hybrid AI models are new and have not been used in the GPM of an area. Geospatial techniques were used and geo-hydrological data of 130 groundwater wells and 12 topographical and geo-environmental factors were used in the model studies. One-R Attribute Evaluation feature selection method was used for the selection of relevant input parameters for the development of AI models. The performance of these models was evaluated using various statistical measures including area under the receiver operation curve (AUC). Results indicated that though all the hybrid models developed in this study enhanced the goodness-of-fit and prediction accuracy, but MRAB-FT (AUC=0.742) model outperformed RF-FT (AUC=0.736), BA-FT (AUC=0.714), and single FT (AUC=0.674) models. Therefore, the MRAB-FT model can be considered as a promising AI hybrid technique for the accurate GPM. Accurate mapping of the groundwater potential zones will help in adequately recharging the aquifer for optimum use of groundwater resources by maintaining the balance between consumption and exploitation.

Locally weighted learning based hybrid intelligence models for groundwater potential mapping and modeling: A case study at Gia Lai province, Vietnam

The groundwater potential map is an important tool for a sustainable water management and land use planning, particularly for agricultural countries like Vietnam. In this article, we proposed new machine learning ensemble techniques namely AdaBoost ensemble (ABLWL), Bagging ensemble (BLWL), Multi Boost ensemble (MBLWL), Rotation Forest ensemble (RFLWL) with Locally Weighted Learning (LWL) algorithm as a base classifier to build the groundwater potential map of Gia Lai province in Vietnam. For this study, eleven conditioning factors (aspect, altitude, curvature, slope, Stream Transport Index (STI), Topographic Wetness Index (TWI), soil, geology, river density, rainfall, land-use) and 134 wells yield data was used to create training (70%) and testing (30%) datasets for the development and validation of the models. Several statistical indices were used namely Positive Predictive Value (PPV), Negative Predictive Value (NPV), Sensitivity (SST), Specificity (SPF), Accuracy (ACC), Kappa, and Receiver Operating Characteristics (ROC) curve to validate and compare performance of models. Results show that performance of all the models is good to very good (AUC: 0.75 to 0.829) but the ABLWL model with AUC = 0.89 is the best. All the models applied in this study can support decision-makers to streamline the management of the groundwater and to develop economy not only of specific territories but also in other regions across the world with minor changes of the input parameters.

Landslide Susceptibility Mapping Using Rotation Forest Ensemble Technique with Different Decision Trees in the Three Gorges Reservoir Area, China

This study presents a new ensemble framework to predict landslide susceptibility by integrating decision trees (DTs) with the rotation forest (RF) ensemble technique. The proposed framework mainly includes four steps. First, training and validation sets are randomly selected according to historical landslide locations. Then, landslide conditioning factors are selected and screened by the gain ratio method. Next, several training subsets are produced from the training set and a series of trained DTs are obtained by using a DT as a base classifier couple with different training subsets. Finally, the resultant landslide susceptibility map is produced by combining all the DT classification results using the RF ensemble technique. Experimental results demonstrate that the performance of all the DTs can be effectively improved by integrating them with the RF ensemble technique. Specifically, the proposed ensemble methods achieved the predictive values of 0.012–0.121 higher than the DTs in terms of area under the curve (AUC). Furthermore, the proposed ensemble methods are better than the most popular ensemble methods with the predictive values of 0.005–0.083 in terms of AUC. Therefore, the proposed ensemble framework is effective to further improve the spatial prediction of landslides.

Performance of rotation forest ensemble classifier and feature extractor in predicting protein interactions using amino acid sequences

BackgroundThere are two significant problems associated with predicting protein-protein interactions using the sequences of amino acids. The first problem is representing each sequence as a feature vector, and the second is designing a model that can identify the protein interactions. Thus, effective feature extraction methods can lead to improved model performance. In this study, we used two types of feature extraction methods—global encoding and pseudo-substitution matrix representation (PseudoSMR)—to represent the sequences of amino acids in human proteins and Human Immunodeficiency Virus type 1 (HIV-1) to address the classification problem of predicting protein-protein interactions. We also compared principal component analysis (PCA) with independent principal component analysis (IPCA) as methods for transforming Rotation Forest.ResultsThe results show that using global encoding and PseudoSMR as a feature extraction method successfully represents the amino acid sequence for the Rotation Forest classifier with PCA or with IPCA. This can be seen from the comparison of the results of evaluation metrics, which were >73% across the six different parameters. The accuracy of both methods was >74%. The results for the other model performance criteria, such as sensitivity, specificity, precision, and F1-score, were all >73%. The data used in this study can be accessed using the following link: https://www.dsc.ui.ac.id/research/amino-acid-pred/.ConclusionsBoth global encoding and PseudoSMR can successfully represent the sequences of amino acids. Rotation Forest (PCA) performed better than Rotation Forest (IPCA) in terms of predicting protein-protein interactions between HIV-1 and human proteins. Both the Rotation Forest (PCA) classifier and the Rotation Forest IPCA classifier performed better than other classifiers, such as Gradient Boosting, K-Nearest Neighbor, Logistic Regression, Random Forest, and Support Vector Machine (SVM). Rotation Forest (PCA) and Rotation Forest (IPCA) have accuracy, sensitivity, specificity, precision, and F1-score values >70% while the other classifiers have values <70%.

Two hidden layer neural network-based rotation forest ensemble for hyperspectral image classification

Decision tree-based Rotation Forest could generate satisfactory but lower classification accuracy for a given training sample set and image data, owing to the inherent disadvantages in decision trees, namely myopic, replication and fragmentation problem. To improve performance of Rotation Forest technique, we propose to utilize two-hidden-layered-feedforward neural network as base classifier instead of decision tree. We examine the classification performance of proposed model under two situations, namely when free network parameters are maintained the same across all ensemble components and otherwise. The proposed model, where each component is initialized with different pair of initial weights and bias, performs better than decision tree-based Rotation Forest on three different Hyperspectral sensor datasets – AVIRIS, ROSIS and Hyperion. Improvements in classification accuracy are above 2% and up to 3% depending upon dataset. Also, the proposed model achieves improvement in accuracy over Random Forest in the range 4.2–8.8%.

A novel hybrid approach of landslide susceptibility modelling using rotation forest ensemble and different base classifiers

In the present study, Rotation Forest ensemble was integrated with different base classifiers to develop different hybrid models namely Rotation Forest based Support Vector Machines (RFSVM), Rotation Forest based Artificial Neural Networks (RFANN), Rotation Forest based Decision Trees (RFDT), and Rotation Forest based Naïve Bayes (RFNB) for landslide susceptibility modelling. The validity of these models was evaluated using statistical methods such as Root Mean Square Error (RMSE), Kappa index, accuracy, and the area under the success rate and predictive rate curves (AUC). Part of the landslide prone area of Pithoragarh district, Uttarakhand, Himalaya, India was selected as the study area. Results indicate that the RFDT is the best model showing the highest predictive capability (AUC = 0.741) in comparison to RFANN (AUC = 0.710), RFSVM (AUC = 0.701), and RFNB (AUC = 0.640) models. The present study would be helpful in the selection of best model for landslide susceptibility mapping.

Diagnosis of Neuromuscular Disorders Using DT-CWT and Rotation Forest Ensemble Classifier

Electromyographic (EMG) signals are utilized to analyze the neuromuscular disorders. Machine learning algorithms have been employed as a decision support system to detect neuromuscular disorders. EMG signals contain noise from different sources, such as electrical and electronic instruments and movement artifacts. In this paper, the multiscale principal component analysis (MSPCA) has been used to remove the impulsive noise from the EMG signals. Then, the dual-tree complex wavelet transform (DT-CWT) is utilized for feature extraction, and the rotation forest ensemble classifier is employed for the recognition of EMG signals. In addition, the performance of several classifiers with rotation forest has been studied. An efficient combination of DT-CWT and rotation forest achieved good performance, using tenfold cross validation regarding the total classification accuracy. Results are promising and showed that the rotation forest achieved an accuracy of 99.7% with clinical EMG signals using support vector machine and 96.6% with simulated EMG signals using the artificial neural network (ANN).

Towards Mobile Cognitive Fatigue Assessment as Indicated by Physical, Social, Environmental, and Emotional Factors

This research sought to establish which in-situ measures of cognitive fatigue, physical activity, social interaction, location, emotional state and facial landmarks, made using a smartphone application, could be used to indicate episodes of cognitive fatigue. This assessment was realised using cognitive tests (assessing memory, attention, reaction time, information processing speed and executive function), self-assessment, contextual factors and facial feature analysis. This study also investigated the use of an ensemble algorithm for the classification of cognitive fatigue utilising facial features and a Rotation Forest approach. Self-assessment of cognitive fatigue was shown to directly correlate with reaction time through a Psychomotor Vigilance Task (r = .643, p = .001), and self-reported increases in the level of social activity (r = .377, p = .001). Facial feature analysis revealed dominant emotions of sadness and anger when participants were cognitively fatigued. It also revealed underlying facial cues that indicated higher levels of cognitive fatigue including expressions of negative valence, and Facial Action Coding System units of increased brow furrow, eyelid tightening and lip suck. In addition, a Principle Component Analysis based Rotation Forest ensemble with a ternary output demonstrated a cognitive fatigue classification accuracy of 82.17%. The findings presented indicate that the inclusion of data relating to surrounding cognitive, social, physical and emotional factors can improve the accuracy of mobile in-situ cognitive fatigue assessment using our previously validated smartphone-based cognitive fatigue assessment approach. The findings further suggest gross-level fatigue status may be potentially classified to a reasonable degree of accuracy using facial features, which may give rise to personalised in-situ fatigue detection.

Landslide susceptibility mapping using J48 Decision Tree with AdaBoost, Bagging and Rotation Forest ensembles in the Guangchang area (China)

Landslides are a manifestation of slope instability causing different kinds of damage affecting life and property. Therefore, high-performance-based landslide prediction models are useful to government institutions for developing strategies for landslide hazard prevention and mitigation. Development of data mining based algorithms shows that high-performance models can be obtained using ensemble frameworks. The primary objective of this study is to investigate and compare the use of current state-of-the-art ensemble techniques, such as AdaBoost, Bagging, and Rotation Forest, for landslide susceptibility assessment with the base classifier of J48 Decision Tree (JDT). The Guangchang district (Jiangxi province, China) was selected as the case study. Firstly, a landslide inventory map with 237 landslide locations was constructed; the landslide locations were then randomly divided into a ratio of 70/30 for the training and validating models. Secondly, fifteen landslide conditioning factors were prepared, such as slope, aspect, altitude, topographic wetness index (TWI), stream power index (SPI), sediment transport index (STI), plan curvature, profile curvature, lithology, distance to faults, distance to rivers, distance to roads, land use, normalized difference vegetation index (NDVI), and rainfall. Relief-F with the 10-fold cross-validation method was applied to quantify the predictive ability of the conditioning factors and for feature selection. Using the JDT and its three ensemble techniques, a total of four landslide susceptibility models were constructed. Finally, the overall performance of the resulting models was assessed and compared using area under the receiver operating characteristic (ROC) curve (AUC) and statistical indexes. The result showed that all landslide models have high performance (AUC>0.8). However, the JDT with the Rotation Forest model presents the highest prediction capability (AUC=0.855), followed by the JDT with the AdaBoost (0.850), the Bagging (0.839), and the JDT (0.814), respectively. Therefore, the result demonstrates that the JDT with Rotation Forest is the best optimized model in this study and it can be considered as a promising method for landslide susceptibility mapping in similar cases for better accuracy.

A novel hybrid artificial intelligence approach based on the rotation forest ensemble and naïve Bayes tree classifiers for a landslide susceptibility assessment in Langao County, China

ABSTRACTThe main objective of this study was to produce landslide susceptibility maps for Langao County, China, using a novel hybrid artificial intelligence method based on rotation forest ensembles (RFEs) and naïve Bayes tree (NBT) classifiers labeled the RF-NBT model. The spatial database consisted of eighteen conditioning factors that were selected using the information gain ratio (IGR) method. The model was evaluated using quantitative statistical criteria, including the sensitivity, specificity, accuracy, root mean squared error (RMSE), and area under the receiver operating characteristic curve (AUC). Furthermore, the new model was compared with the NBT, functional tree (FT), logistic model tree (LMT) and reduced-error pruning tree (REPTree) soft computing benchmark models. The findings indicated that the RF-NBT model showed an increased prediction accuracy relative to the NBT model using both the training and validation datasets, and the RF-NBT model exhibited a greater capability for landslide susceptibility mapping. The new RF-NBT model also showed the most preferable results compared with the FT, LMT and REPTree models. Finally, an analysis of the landslide density (LD) using the RF-NBT model demonstrated that the very high susceptibility (VHS) class had the highest LD (3.552) among the landslide susceptibility maps. These results can be used for the planning and management of areas vulnerable to landslides in order to prevent damages caused by such natural disasters.

A hybrid machine learning ensemble approach based on a Radial Basis Function neural network and Rotation Forest for landslide susceptibility modeling: A case study in the Himalayan area, India

In this paper, a hybrid machine learning ensemble approach namely the Rotation Forest based Radial Basis Function (RFRBF) neural network is proposed for spatial prediction of landslides in part of the Himalayan area (India). The proposed approach is an integration of the Radial Basis Function (RBF) neural network classifier and Rotation Forest ensemble, which are state-of-the art machine learning algorithms for classification problems. For this purpose, a spatial database of the study area was established that consists of 930 landslide locations and fifteen influencing parameters (slope angle, road density, curvature, land use, distance to road, plan curvature, lineament density, distance to lineaments, rainfall, distance to river, profile curvature, elevation, slope aspect, river density, and soil type). Using the database, training and validation datasets were generated for constructing and validating the model. Performance of the model was assessed using the Receiver Operating Characteristic (ROC) curve, area under the ROC curve (AUC), statistical analysis methods, and the Chi square test. In addition, Logistic Regression (LR), Multi-layer Perceptron Neural Networks (MLP Neural Nets), Naïve Bayes (NB), and the hybrid model of Rotation Forest and Decision Trees (RFDT) were selected for comparison. The results show that the proposed RFRBF model has the highest prediction capability in comparison to the other models (LR, MLP Neural Nets, NB, and RFDT); therefore, the proposed RFRBF model is promising and should be used as an alternative technique for landslide susceptibility modeling.

Aggressive and agitated behavior recognition from accelerometer data using non-negative matrix factorization

This paper presents a novel approach for aggressive and agitated behavior recognition using accelerometer data. Our approach applies first a noise reduction technique using the moving average filter method. Then, multiple features such as mean, variance, entropy, correlation and covariance are extracted from the filtered acceleration data using a sliding window. Non-negative matrix factorization is then used in order to project the data into a new reduced space that captures the significant structure of the data. The recognition is performed using the rotation forest ensemble method. The proposed approach is validated using extensive experiments on a real dataset collected at Toronto Rehabilitation Institute. We empirically demonstrate that our proposed approach accurately discriminates between behaviors and performs better than several state-of-the-art approaches.

Ensembles of classifiers based on dimensionality reduction

We present a novel approach for the construction of ensemble classifiers based on dimensionality reduction. Dimensionality reduction methods represent datasets using a small number of attributes while preserving the information conveyed by the original dataset. The ensemble members are trained based on dimension-reduced versions of the training set. These versions are obtained by applying dimensionality reduction to the original training set using different values of the input parameters. This construction meets both the diversity and accuracy criteria which are required to construct an ensemble classifier where the former criterion is obtained by the various input parameter values and the latter is achieved due to the decorrelation and noise reduction properties of dimensionality reduction. In order to classify a test sample, it is first embedded into the dimension reduced space of each individual classifier by using an out-of-sample extension algorithm. Each classifier is then applied to the embedded sample and the classification is obtained via a voting scheme. We present three variations of the proposed approach based on the Random Projections, the Diffusion Maps and the Random Subspaces dimensionality reduction algorithms. We also present a multi-strategy ensemble which combines AdaBoost and Diffusion Maps. A comparison is made with the Bagging, AdaBoost, Rotation Forest ensemble classifiers and also with the base classifier which does not incorporate dimensionality reduction. Our experiments used seventeen benchmark datasets from the UCI repository. The results obtained by the proposed algorithms were superior in many cases to other algorithms.