Implementation Of Random Forests Research Articles

Efficient Feature Selection for Classification of Immunotherapy and Medical Treatments Utilising Random Forest and Decision Trees

Immunotherapy is an important topic in healthcare as it affects patients’ treatments for breast cancer, diabetes, and immunotherapy. However, immunotherapy for warts is less representative because of the lack of data. Machine learning is frequently utilised for treatment diagnosis by converting raw immunotherapy data into useful insights. Efficient classification of immunotherapy treatments is crucial for a productive diagnosis. This study considers immunotherapy with a data-driven and ’less is more perspective’. Despite using a portion of the available imbalance and complex data, the process of diagnosis of immunotherapy treatment is made reasonably precise by considering the parameters of accuracy, sensitivity, and specificity. The contribution of this study is focused on ”more is less” feature selection, which states that approximately 80% of the effects or results of a system are caused by 20% of the inputs. The features that contribute most to the classification of immunotherapy treatments are prioritised. This study proposes the implementation of Random Forest and Decision Trees for the classification of immunotherapy treatments. The relevant experimental medical data are explored as a case study. The experiments are conducted using Weka and Python data analysis tools, performing data preprocessing, class balancing, and feature selection. Random Forest performed better than the Decision Trees. By Applying Random Forest and utilising only one feature (time) as an input variable, a classification accuracy of 88.88%, sensitivity of 95.45%, and specificity of 60% are attained. By using 12.5% of the dataset, when implementing Random Forest together with ordinary feature selection, the diagnosis of immunotherapy treatments is become more efficient, despite using a portion of data features reasonable results are obtained.

Implementation of random forest, adaptive boosting, and gradient boosting decision trees algorithms for gully erosion susceptibility mapping using remote sensing and GIS

Implementation of random forest, adaptive boosting, and gradient boosting decision trees algorithms for gully erosion susceptibility mapping using remote sensing and GIS

Implementation of Random Forest for Vehicle Type Classification using Gamma Correction Algorithm

Traffic control systems can be a valuable tool for monitoring road traffic by counting and tracking vehicles in real time. This research is the initial research into the development of methods to improve the accuracy of vehicle detection and recognition. The purpose of the study was to analyze the performance of gamma correction and random forest performance to improve the accuracy of vehicle detection and recognition. Performance measures used in the study were confusion matrix, accuracy, precision, recall, F1-score. Based on experimental results, random forest with gamma=1.5 got the best accuracy of 85.00%, while random forest with gamma=0.5 got accuracy of 81.30%, random forest with gamma=1.0 got accuracy of 84.00%, random forest with gamma=2.0 got accuracy of 84.00%.

Implementation of Random Forest and Extreme Gradient Boosting in the Classification of Heart Disease using Particle Swarm Optimization Feature Selection

Heart disease is a condition that ranks as the primary cause of death worldwide. Based on available data, over 36 million people have succumbed to non-communicable diseases, and heart disease falls within the category of non-communicable diseases. This research employs a heart disease dataset from the UCI Repository, consisting of 303 instances and 14 categorical features. In this research, the data were analyzed using the classification methods XGBoost (Extreme Gradient Boosting) and Random Forest, which can be applied with PSO (Particle Swarm Optimization) as a feature selection technique to address the issue of irrelevant features. This issue can impact prediction performance on the heart disease dataset. From the results of the conducted research, the obtained values for the XGBoost (Extreme Gradient Boosting) model were 0.877, and for the Random Forest model, it was 0.874. On the other hand, in the model utilizing Particle Swarm Optimization (PSO), the obtained AUC values are 0.913 for XGBoost (Extreme Gradient Boosting) and 0.918 for Random Forest. These research results demonstrate that PSO (Particle Swarm Optimization) can enhance the AUC of heart disease prediction performance. Therefore, this research contributes to enhancing the precision and efficiency of heart disease patient data processing, which benefits heart disease diagnosis in terms of speed and accuracy.

Defending the Defender: Adversarial Learning Based Defending Strategy for Learning Based Security Methods in Cyber-Physical Systems (CPS)

Cyber-Physical Systems (CPS) are prone to many security exploitations due to a greater attack surface being introduced by their cyber component by the nature of their remote accessibility or non-isolated capability. Security exploitations, on the other hand, rise in complexities, aiming for more powerful attacks and evasion from detections. The real-world applicability of CPS thus poses a question mark due to security infringements. Researchers have been developing new and robust techniques to enhance the security of these systems. Many techniques and security aspects are being considered to build robust security systems; these include attack prevention, attack detection, and attack mitigation as security development techniques with consideration of confidentiality, integrity, and availability as some of the important security aspects. In this paper, we have proposed machine learning-based intelligent attack detection strategies which have evolved as a result of failures in traditional signature-based techniques to detect zero-day attacks and attacks of a complex nature. Many researchers have evaluated the feasibility of learning models in the security domain and pointed out their capability to detect known as well as unknown attacks (zero-day attacks). However, these learning models are also vulnerable to adversarial attacks like poisoning attacks, evasion attacks, and exploration attacks. To make use of a robust-cum-intelligent security mechanism, we have proposed an adversarial learning-based defense strategy for the security of CPS to ensure CPS security and invoke resilience against adversarial attacks. We have evaluated the proposed strategy through the implementation of Random Forest (RF), Artificial Neural Network (ANN), and Long Short-Term Memory (LSTM) on the ToN_IoT Network dataset and an adversarial dataset generated through the Generative Adversarial Network (GAN) model.

Identification of representative trees in random forests based on a new tree-based distance measure

In life sciences, random forests are often used to train predictive models. However, gaining any explanatory insight into the mechanics leading to a specific outcome is rather complex, which impedes the implementation of random forests into clinical practice. By simplifying a complex ensemble of decision trees to a single most representative tree, it is assumed to be possible to observe common tree structures, the importance of specific features and variable interactions. Thus, representative trees could also help to understand interactions between genetic variants. Intuitively, representative trees are those with the minimal distance to all other trees, which requires a proper definition of the distance between two trees. Thus, we developed a new tree-based distance measure, which incorporates more of the underlying tree structure than other metrics. We compared our new method with the existing metrics in an extensive simulation study and applied it to predict the age at onset based on a set of genetic risk factors in a clinical data set. In our simulation study we were able to show the advantages of our weighted splitting variable approach. Our real data application revealed that representative trees are not only able to replicate the results from a recent genome-wide association study, but also can give additional explanations of the genetic mechanisms. Finally, we implemented all compared distance measures in R and made them publicly available in the R package timbR (https://github.com/imbs-hl/timbR).

Enhanced Fuzzy Random Forest by Using Doubly Randomness and Copying From Dynamic Dictionary Attributes

While fuzzy random forest (FRF) as a fuzzy implementation of random forest has earned its strong ambiguity/uncertainty handling capability on a rich variety of considerably low dimensional datasets, this article revisits FRF and attempts to enhance its generalization capability and computational speed on high dimensional datasets. For the first issue, in addition to the original use of randomness in FRF, a doubly randomness is newly introduced into the generation of both the candidate attributes and the best splitting attributes in FRF. For the computational speed issue, while the proposed new fuzzy information gain (NFG) measure does not apply to all candidate attributes, the remaining NFG values can be quickly retrieved by copying from the dynamically generated dictionary. As a result, a new method called enhanced fuzzy random forest (E-FRF) is proposed and justified theoretically from the consistency perspective. Our extensive experimental results indicate that the proposed method E-FRF has at least comparable performance to the comparative methods, and is an advantageous alternative to FRF in terms of both testing accuracy and running speed in most of the adopted high dimensional datasets.

Prediction Map of Rainfall Classification Using Random Forest and Inverse Distance Weighted (IDW)

The amount of rainfall that occurs can affect natural disasters and even food production to economic activities. the factor of the area where the rain occurs is one of the main parameters for how the change occurs. So, it is necessary to have a rainfall prediction approach that aims to find out when and what type of rain will occur. Spatial classification and interpolation are two methods used to make predictions. Random Forest is a classification method that can be used to predict rainfall. and Inverse Distance Weighted is one of the stochastic interpolation techniques to calculate the estimated rainfall from the data points of rainfall that occur so that the distribution can be visualized. In the implementation of random forest, the model that is built on a daily basis gets the best level of accuracy in the 5D model sub model C with an accuracy of 0.8238 while the monthly model gets the best level of accuracy in the sub-model B 4M 0.9362. and the results of predictions and mapping using IDW show that daily predictions from June 1-4 2022 show that Most of Java Island will experience light rain, June 5-7 2022 most of Java Island will experience sunny cloudy days. And for monthly predictions, August and June 2022 show the distribution of monthly rainfall with predictions that most of Java is cloudy, while May, July, October, September have light rainfall in most of Java

Airborne HySpex Hyperspectral Versus Multitemporal Sentinel-2 Images for Mountain Plant Communities Mapping

Climate change and anthropopression significantly impact plant communities by leading to the spread of expansive and alien invasive plants, thus reducing their biodiversity. Due to significant elevation gradients, high-mountain plant communities in a small area allow for the monitoring of the most important environmental changes. Additionally, being a tourist attraction, they are exposed to direct human influence (e.g., trampling). Airborne hyperspectral remote sensing is one of the best data sources for vegetation mapping, but flight campaign costs limit the repeatability of surveys. A possible alternative approach is to use satellite data from the Copernicus Earth observation program. In our study, we compared multitemporal Sentinel-2 data with HySpex airborne hyperspectral images to map the plant communities on Tatra Mountains based on open-source R programing implementation of Random Forest and Support Vector Machine classifiers. As high-mountain ecosystems are adapted to topographic conditions, the input of Digital Elevation Model (DEM) derivatives on the classification accuracy was analyzed and the effect of the number of training pixels was tested to procure practical information for field campaign planning. For 13 classes (from rock scree communities and alpine grasslands to montane conifer and deciduous forests), we achieved results in the range of 76–90% F1-score depending on the data set. Topographic features: digital terrain model (DTM), normalized digital surface model (nDSM), and aspect and slope maps improved the accuracy of HySpex spectral images, transforming their minimum noise fraction (MNF) bands and Sentinel-2 data sets by 5–15% of the F1-score. Maps obtained on the basis of HySpex imagery (2 m; 430 bands) had a high similarity to maps obtained on the basis of multitemporal Sentinel-2 data (10 m; 132 bands; 11 acquisition dates), which was less than one percentage point for classifications based on 500–1000 pixels; for sets consisting of 50–100 pixels, Random Forest (RF) offered better accuracy.

A machine learning-based approach to directly compare the diagnostic accuracy of myocardial perfusion imaging by conventional and cadmium-zinc telluride SPECT

BackgroundWe evaluated the performance of conventional (C) single-photon emission computed tomography (SPECT) and cadmium-zinc-telluride (CZT)-SPECT in a large cohort of patients with suspected or known coronary artery disease (CAD) and compared the diagnostic accuracy of the two systems using machine learning (ML) algorithms. Methods and ResultsA total of 517 consecutive patients underwent stress myocardial perfusion imaging (MPI) by both C-SPECT and CZT-SPECT. In the overall population, an excellent correlation between stress MPI data and left ventricular (LV) functional parameters measured by C-SPECT and by CZT-SPECT was observed (all P < .001). ML analysis performed through the implementation of random forest (RF) and k-nearest neighbors (NN) algorithms proved that CZT-SPECT has greater accuracy than C-SPECT in detecting CAD. For both algorithms, the sensitivity of CZT-SPECT (96% for RF and 60% for k-NN) was greater than that of C-SPECT (88% for RF and 53% for k-NN). ConclusionsMPI data and LV functional parameters obtained by CZT-SPECT are highly reproducible and provide good correlation with those obtained by C-SPECT. ML approach showed that the accuracy and sensitivity of CZT-SPECT is greater than C-SPECT in detecting CAD.

Comparing the Prognostic Value of Stress Myocardial Perfusion Imaging by Conventional and Cadmium-Zinc Telluride Single-Photon Emission Computed Tomography through a Machine Learning Approach.

We compared the prognostic value of myocardial perfusion imaging (MPI) by conventional- (C-) single-photon emission computed tomography (SPECT) and cadmium-zinc-telluride- (CZT-) SPECT in a cohort of patients with suspected or known coronary artery disease (CAD) using machine learning (ML) algorithms. A total of 453 consecutive patients underwent stress MPI by both C-SPECT and CZT-SPECT. The outcome was a composite end point of all-cause death, cardiac death, nonfatal myocardial infarction, or coronary revascularization procedures whichever occurred first. ML analysis performed through the implementation of random forest (RF) and k-nearest neighbors (KNN) algorithms proved that CZT-SPECT has greater accuracy than C-SPECT in detecting CAD. For both algorithms, the sensitivity of CZT-SPECT (96% for RF and 60% for KNN) was greater than that of C-SPECT (88% for RF and 53% for KNN). A preliminary univariate analysis was performed through Mann-Whitney tests separately on the features of each camera in order to understand which ones could distinguish patients who will experience an adverse event from those who will not. Then, a machine learning analysis was performed by using Matlab (v. 2019b). Tree, KNN, support vector machine (SVM), Naïve Bayes, and RF were implemented twice: first, the analysis was performed on the as-is dataset; then, since the dataset was imbalanced (patients experiencing an adverse event were lower than the others), the analysis was performed again after balancing the classes through the Synthetic Minority Oversampling Technique. According to KNN and SVM with and without balancing the classes, the accuracy (p value = 0.02 and p value = 0.01) and recall (p value = 0.001 and p value = 0.03) of the CZT-SPECT were greater than those obtained by C-SPECT in a statistically significant way. ML approach showed that although the prognostic value of stress MPI by C-SPECT and CZT-SPECT is comparable, CZT-SPECT seems to have higher accuracy and recall.

Informative Regions In Viral Genomes.

Viruses, far from being just parasites affecting hosts’ fitness, are major players in any microbial ecosystem. In spite of their broad abundance, viruses, in particular bacteriophages, remain largely unknown since only about 20% of sequences obtained from viral community DNA surveys could be annotated by comparison with public databases. In order to shed some light into this genetic dark matter we expanded the search of orthologous groups as potential markers to viral taxonomy from bacteriophages and included eukaryotic viruses, establishing a set of 31,150 ViPhOGs (Eukaryotic Viruses and Phages Orthologous Groups). To do this, we examine the non-redundant viral diversity stored in public databases, predict proteins in genomes lacking such information, and used all annotated and predicted proteins to identify potential protein domains. The clustering of domains and unannotated regions into orthologous groups was done using cogSoft. Finally, we employed a random forest implementation to classify genomes into their taxonomy and found that the presence or absence of ViPhOGs is significantly associated with their taxonomy. Furthermore, we established a set of 1457 ViPhOGs that given their importance for the classification could be considered as markers or signatures for the different taxonomic groups defined by the ICTV at the order, family, and genus levels.

Evaluation of Four Multiple Imputation Methods for Handling Missing Binary Outcome Data in the Presence of an Interaction between a Dummy and a Continuous Variable

Multiple imputation by chained equations (MICE) is the most common method for imputing missing data. In the MICE algorithm, imputation can be performed using a variety of parametric and nonparametric methods. The default setting in the implementation of MICE is for imputation models to include variables as linear terms only with no interactions, but omission of interaction terms may lead to biased results. It is investigated, using simulated and real datasets, whether recursive partitioning creates appropriate variability between imputations and unbiased parameter estimates with appropriate confidence intervals. We compared four multiple imputation (MI) methods on a real and a simulated dataset. MI methods included using predictive mean matching with an interaction term in the imputation model in MICE (MICE-interaction), classification and regression tree (CART) for specifying the imputation model in MICE (MICE-CART), the implementation of random forest (RF) in MICE (MICE-RF), and MICE-Stratified method. We first selected secondary data and devised an experimental design that consisted of 40 scenarios (2 × 5 × 4), which differed by the rate of simulated missing data (10%, 20%, 30%, 40%, and 50%), the missing mechanism (MAR and MCAR), and imputation method (MICE-Interaction, MICE-CART, MICE-RF, and MICE-Stratified). First, we randomly drew 700 observations with replacement 300 times, and then the missing data were created. The evaluation was based on raw bias (RB) as well as five other measurements that were averaged over the repetitions. Next, in a simulation study, we generated data 1000 times with a sample size of 700. Then, we created missing data for each dataset once. For all scenarios, the same criteria were used as for real data to evaluate the performance of methods in the simulation study. It is concluded that, when there is an interaction effect between a dummy and a continuous predictor, substantial gains are possible by using recursive partitioning for imputation compared to parametric methods, and also, the MICE-Interaction method is always more efficient and convenient to preserve interaction effects than the other methods.

Assessment of Landsat Based Deep-Learning Membership Analysis for Development of from–to Change Time Series in the Prairie Region of Canada from 1984 to 2018

The prairie region of Canada is a dynamically changing landscape in relation to past and present anthropogenic activities and recent climate change. Improving our understanding of the rate, timing, and distribution of landscape change is needed to determine the impact on wildlife populations and biodiversity, ultimately leading to better-informed management regarding requirements for habitat amount and its connectedness. In this research, we assessed the viability of an approach to detect from–to class changes designed to be scalable to the prairie region with the capacity for local refinement. It employed a deep-learning convolutional neural network to model general land covers and examined class memberships to identify land-cover conversions. For this implementation, eight land-cover categories were derived from the Agriculture and Agri-Food Canada Annual Space-Based Crop Inventory. Change was assessed in three study areas that contained different mixes of grassland, pasture, and forest cover. Results showed that the deep-learning method produced the highest accuracy across all classes relative to an implementation of random forest that included some first-order texture measures. Overall accuracy was 4% greater with the deep-learning classifier and class accuracies were more balanced. Evaluation of change accuracy suggested good performance for many conversions such as grassland to crop, forest to crop, water to dryland covers, and most bare/developed-related changes. Changes involving pasture with grassland or cropland were more difficult to detect due to spectral confusion among classes. Similarly, conversion to forests in some cases was poorly detected due to gradual and subtle change characteristics combined with confusion between forest, shrub, and croplands. The proposed framework involved several processing steps that can be explored to enhance the thematic content and accuracy for large regional implementation. Evaluation for understanding connectivity in natural land covers and related declines in species at risk is planned for future research.

Feature Extraction for Heroin-Use Classification Using Imbalanced Random Forest Methods

Background and aims: The National Survey on Drug Use and Health (NSDUH) contains a large number of responses and many features. This study aims to identify features from within NSDUH that are important in classifying heroin use. Proper implementation of random forest (RF) techniques copes with the highly imbalanced nature of heroin usage among respondents to identify features that are prominent in classification models involving nonlinear combinations of predictive variables. To date, methods for the proper application of RF to imbalanced medical datasets have not been defined. Methods: Three different RF classification techniques are applied to the 2016 NSDUH. The techniques are compared using scoring criteria, including area under the precision recall curve (AUPRC), to identify the best model. Variable importance scores (VIS) are checked for stability across the three models and the VIS from the best model are used to highlight features and categories of features that most influence the classification of heroin users. Findings: The best performing method was RF with random oversampling (AUPRC = 0.5437). The category of features regarding other drug use was most important (average z-scored VIS = 1.66) followed by age-of-first-use features (0.32). The most important individual feature was cocaine usage (z-scored VIS = 11.05), followed by crack usage (6.51). The most important individual feature other than specific drug use flags was the use of marijuana under the age of 18 (3.11). This study demonstrates a method for the use of RF in feature extraction from imbalanced medical datasets with many predictors.

Nearest Neighbor Gaussian Process for Quantitative Structure-Activity Relationships.

While Gaussian process models are typically restricted to smaller data sets, we propose a variation which extends its applicability to the larger data sets common in the industrial drug discovery space, making it relatively novel in the quantitative structure-activity relationship (QSAR) field. By incorporating locality-sensitive hashing for fast nearest neighbor searches, the nearest neighbor Gaussian process model makes predictions with time complexity that is sub-linear with the sample size. The model can be efficiently built, permitting rapid updates to prevent degradation as new data is collected. Given its small number of hyperparameters, it is robust against overfitting and generalizes about as well as other common QSAR models. Like the usual Gaussian process model, it natively produces principled and well-calibrated uncertainty estimates on its predictions. We compare this new model with implementations of random forest, light gradient boosting, and k-nearest neighbors to highlight these promising advantages. The code for the nearest neighbor Gaussian process is available at https://github.com/Merck/nngp.

Tree aggregation for random forest class probability estimation

AbstractIn random forest methodology, an overall prediction or estimate is made by aggregating predictions made by individual decision trees. Popular implementations of random forests rely on different methods for aggregating predictions. In this study, we provide an empirical analysis of the performance of aggregation approaches available for classification and regression problems. We show that while the choice of aggregation scheme usually has little impact in regression, it can have a profound effect on probability estimation in classification problems. Our study illustrates the causes of calibration issues that arise from two popular aggregation approaches and highlights the important role that terminal nodesize plays in the aggregation of tree predictions. We show that optimal choices for random forest tuning parameters depend heavily on the manner in which tree predictions are aggregated.

FPGA-Based Network Traffic Classification Using Machine Learning

Real-time classification of internet traffic is critical for the efficient management of networks. Classification approaches based on machine learning techniques have shown promising results with high levels of accuracy. In this article, the suitability of packet-level and flow-level features is validated using stepwise regression and random forest feature selection. Moreover, the optimal percentage of packets considered within a flow while extracting flow-level features is determined. Several experiments are conducted using naive Bayes, support vector machine, $k$ -nearest neighbor, random forest, and artificial neural networks on the University of Brescia (UNIBS) and the University of New Brunswick (UNB) datasets, which are both publicly available. The performed experiments show that 60% of flow packets are a good compromise that ensures high performance in the least processing time. The results of the conducted experiments indicate that random forest outperforms other algorithms achieving a maximum accuracy of 98.5% and an F-score of 0.932. Further, and since software-based classifiers cannot meet the anticipated real-time requirements, we propose a Field-Programmable Gate Array (FPGA) based random forest implementation that utilizes a highly pipelined architecture to accelerate such a time-consuming task. The proposed design achieves an average throughput of 163.24 Gbps, exceeding throughputs of reported hardware-based classifiers that use comparable approaches, which in turn ensures the continuity of real-time traffic classification at congested data centers.

A Hybrid Set of Handwriting Features for Handwritten Recognition

Handwriting of each person is unique since each person has their own unique and different style of handwriting. Handwriting verification can be performed in two ways, dynamic and static. The dynamic verification process is the writer dependent whereas the static verification process is the writer independent procedure. The features can be spatial, structural, statistical, geometrical, graphological, and from other feature extraction techniques. In this work, we are considering the combination of multilevel feature set for writer recognition and identification purpose. A dataset of different handwriting samples collected from 100 different writers is used for this experiment. A decision tree classifier with random forest implementation is used for recognition and identification of writer with 98.2% accuracy.

Random forest implementation and optimization for Big Data analytics on LexisNexis\u2019s high performance computing cluster platform

In this paper, we comprehensively explain how we built a novel implementation of the Random Forest algorithm on the High Performance Computing Cluster (HPCC) Systems Platform from LexisNexis. The algorithm was previously unavailable on that platform. Random Forest’s learning process is based on the principle of recursive partitioning and although recursion per se is not allowed in ECL (HPCC’s programming language), we were able to implement the recursive partition algorithm as an iterative split/partition process. In addition, we analyze the flaws found in our initial implementation and we thoroughly describe all the modifications required to overcome the bottleneck within the iterative split/partition process, i.e., the optimization of the data gathering of selected independent variables which are used for the node’s best-split analysis. Essentially, we describe how our initial Random Forest implementation has been optimized and has become an efficient distributed machine learning implementation for Big Data. By taking full advantage of the HPCC Systems Platform’s Big Data processing and analytics capabilities, we succeed in enhancing the data gathering method from an inefficient Pass them All and Filter approach into an effective and completely parallelized Fetching on Demand approach. Finally, based upon the results of our learning process runtime comparison between these two approaches, we confirm the speed up of our optimized Random Forest implementation.