Random Forest-based Classifier Research Articles

COMPARATIVE ANALYSIS OF RANDOM FOREST AND ADABOOST LEARNING MODELS FOR THE CLASSIFICATION OF ATTACKS IN INTERNET OF THINGS

Attacks are actions that attempt to break one of the following properties of the computer system: confidentiality, integrity, and availability. The immense increment in the amount of internet applications and the appearance of modern networks has created the need for improved security mechanisms. Internet of Things (IoT) is a system that uses the Internet to facilitate communication between sensors and devices. Several approaches have been used to build attacks detection system in the past. This study built two ensemble models for the classification of attacks using Random Forest and Adaboost algorithms respectively. Feature importance was used for selecting promising attributes from the IoT intrusion dataset. Thereafter, the results of the classification models were evaluated and compared. The models were evaluated based on when feature selection technique was applied and without respectively. For Random Forest-based classification model with feature selection, 99.0% ,0.95,0.88,0.82, were obtained for accuracy, recall, f1-score, and precision respectively while without feature selection 69.0%,0.86,0.76,0.64 were obtained respectively. For Adaboost-based classification model with feature selection 99.0%.0.69,0.61,0.66 were obtained for accuracy, recall, f1-score and precision respectively. Without feature selection the Adaboost model recorded 58.0%,0.58,0.48,0.50 respectively. The results showed that both models achieved high rates with feature selection technique used, with Random Forest performing slightly better, both learning models showed promised performances in classifying attacks in IoT environments. This study concluded that the use of the chosen feature selection method helped improve the performances of the two ensembles in the classification of attacks in the IoT dataset.

Spatial modelling and classification of altered coal using random forest-based methods at Moatize Basin, Mozambique

Coal basins affected by intrusion pose distinctive challenges, such as the complexity of determining the extent of impacted coal from borehole cuttings and the potential for inaccuracies in the spatial modelling of coal quality parameters. It is more challenging in complex thinly bedded coal seams, characterised by multiple sub-layers of varying thickness and coal quality. This study employs machine learning techniques in the highly complex thinly bedded Permian-aged Chipanga coal seam in the Moatize Basin, Mozambique. The study aims to assess the efficacy of these techniques for the spatial modelling and classification of altered coals. The results demonstrate that the random forest-based classifier can discriminate between altered and fresh coal with a range of accuracy of 97%–100%. Furthermore, among the five spatial methods assessed, geographical quantile regression forest emerges as one of the top-performing approaches for assessing the impact of magmatic intrusions on coal properties within the Moatize Basin. Although previous studies have shown the ability of these methods for spatial modelling and coal classification on single coal seam, these techniques have not been used in complex thin-bedded coal seams. Therefore, the findings of this study highlight the implications of these methods in navigating the complexities of thin-bedded coal seams.

Lesion attention guided neural network for contrast-enhanced mammography-based biomarker status prediction in breast cancer

Background and objectiveAccurate identification of molecular biomarker statuses is crucial in cancer diagnosis, treatment, and prognosis. Studies have demonstrated that medical images could be utilized for non-invasive prediction of biomarker statues. The biomarker status-associated features extracted from medical images are essential in developing medical image-based non-invasive prediction models. Contrast-enhanced mammography (CEM) is a promising imaging technique for breast cancer diagnosis. This study aims to develop a neural network-based method to extract biomarker-related image features from CEM images and evaluate the potential of CEM in non-invasive biomarker status prediction. MethodsAn end-to-end learning convolutional neural network with the whole breast images as inputs was proposed to extract CEM features for biomarker status prediction in breast cancer. The network focused on lesion regions and flexibly extracted image features from lesion and peri‑tumor regions by employing supervised learning with a smooth L1-based consistency constraint. An image-level weakly supervised segmentation network based on Vision Transformer with cross attention to contrast images of breasts with lesions against the contralateral breast images was developed for automatic lesion segmentation. Finally, prediction models were developed following further selection of significant features and the implementation of random forest-based classification. Results were reported using the area under the curve (AUC), accuracy, sensitivity, and specificity. ResultsA dataset from 1203 breast cancer patients was utilized to develop and evaluate the proposed method. Compared to the method without lesion attention and with only lesion regions as inputs, the proposed method performed better at biomarker status prediction. Specifically, it achieved an AUC of 0.71 (95 % confidence interval [CI]: 0.65, 0.77) for Ki-67 and 0.73 (95 % CI: 0.65, 0.80) for human epidermal growth factor receptor 2 (HER2). ConclusionsA lesion attention-guided neural network was proposed in this work to extract CEM image features for biomarker status prediction in breast cancer. The promising results demonstrated the potential of CEM in non-invasively predicting the biomarker statuses in breast cancer.

Exploring factors associated with higher education students’ learning outcomes in emergency remote teaching environments during the COVID-19 pandemic: General patterns and individual differences

In this study, we aimed to explore the factors associated with higher education students’ learning outcomes in emergency remote teaching environments (ERTEs) during the COVID-19 pandemic at both the population and individual levels, given the limited understanding in previous research. 9418 students from 41 countries were selected for analysis from a survey-based dataset that was collected with the aim of understanding the self-perceived impacts of the first-wave COVID-19 pandemic on higher education students. We conducted structural equation modeling to explore associated factors and latent profile analysis to identify student profiles based on these factors. Utilizing the identified profiles, we developed a random forest-based classifier to identify the membership of students’ profiles. The results showed that six environmental and individual factors—partially mediated by academic emotions—were significantly associated with learning outcomes. The positively associated factors, ranked by path coefficient, were satisfaction with synchronous course organization (SCO), computer skills (CS), satisfaction with asynchronous course organization (ACO), satisfaction with administration support (AS), and perceived teacher support (TS). The negatively associated factor was worry about life. Based on these factors, eight profiles were identified with varying learning outcomes and student characteristics. The classifier achieved a testing accuracy of 0.904. By integrating variable-centered and person-centered approaches, this study bridges the gap in understandings of general patterns and individual differences regarding key factors associated with higher education students’ learning outcomes. The findings have implications for designing individualized interventions and support strategies to enhance student learning outcomes and mitigate educational disparities in ERTEs during crisis situations.

A Data-Driven Approach to Estimate Incident-Induced Delays Using Incomplete Probe Vehicle Data: Application to Safety Service Patrol Program Evaluation

This paper presents a data-driven approach to estimate incident-induced delays (IIDs) using probe vehicle data while accounting for missing data. The proposed approach is applied to evaluate the effectiveness of a safety service patrol (SSP) program. Existing data-driven methods for IID estimation usually rely on complete data sets. The proposed approach employs a random forest-based classification model and an interpolation method to estimate IIDs when real-time data are completely or partially missing during the incident-impacted time period. It also identifies reference profiles from the closest spatial-temporal road segments to improve data availability. The case study shows that the SSP program in the Quad Cities area of Iowa reduces IIDs associated with various incidents by 15%–91%. This data-driven evaluation framework can be applied to other traffic incident management programs, allowing more accurate and objective evaluations of their effectiveness.

METB-10. EXPLAINABLE ARTIFICIAL INTELLIGENCE REVEALS DNA METHYLATION PATTERNS UNDERLYING BRAIN TUMOR CLASSIFICATION

Abstract Over 100 different molecular classes of brain tumors are recognized across histopathological grades and age groups. Their precise diagnosis is crucial for prognostication and appropriate treatment decisions, particularly in pediatric patients. Recently, a random forest-based classifier trained on genome-wide DNA methylation profiles has been developed to guide the accurate classification of these tumors. Over recent years, this classifier has been applied to tens-of-thousands of patient tumors all over the world. Clinical and basic neuro-oncology research would benefit greatly from a detailed understanding of the underlying artificial intelligence decision process. However, why the algorithm arrives at a certain prediction remains unclear. Here, we develop an interpretable framework to explain the model’s decisions and to uncover DNA methylation patterns distinguishing different tumor classes. Using this approach, we demonstrate both global and local differences across epigenetic landscapes. For instance, the model frequently uses DNA methylation probes in CpG islands to distinguish IDH mutant gliomas from other tumor classes. Tumor classes associated with global hypomethylation are distinguished by probes located in broad heterochromatic domains. At the level of individual genes, we find hundreds of promoter and gene body regions specifically methylated or unmethylated in specific classes. In general, we detect a high degree of redundant information in the dataset, with multiple genes across the genome distinguishing individual tumor classes, and many CpG probes representing each gene. These findings explain the technical robustness of the array-based classifier, and provide a starting point for the design of more compact point of care assays. The results of our approach are provided to the research community via an interactive web interface. We hope that this resource will help to build up trust in the machine learning-based classifier in clinical settings, foster biomarker discovery, and enable cancer epigenome research in the context of the central nervous system tumors.

Active deep learning for segmentation of industrial CT data

Abstract This contribution proposes an approach and the respective tool that uses Active Deep Learning (ADL) to segment industrial three-dimensional computed tomography (3D CT) data. The general approach is application-independent and includes an iterative human-in-the-loop Active Learning (AL) process that produces labeled training data and a trained Deep Learning (DL) model for semantic segmentation. The model is continuously improved during iterations such that manual labeling effort is reduced. In addition, the user can minimize user interaction with the aid of a random forest-based classifier and focus on unclear or invalid segmentation results. The complete workflow is implemented within one single Python tool. The approach is demonstrated in detail for two industrial use cases: Single fiber analysis and plant segmentation. For plant segmentation, the method is compared to a baseline and a classic image processing algorithm.

DEFEG: Deep Ensemble with Weighted Feature Generation

With the significant breakthrough of Deep Neural Networks in recent years, multi-layer architecture has influenced other sub-fields of machine learning including ensemble learning. In 2017, Zhou and Feng introduced a deep random forest called gcForest that involves several layers of Random Forest-based classifiers. Although gcForest has outperformed several benchmark algorithms on specific datasets in terms of classification accuracy and model complexity, its input features do not ensure better performance when going deeply through layer-by-layer architecture. We address this limitation by introducing a deep ensemble model with a novel feature generation module. Unlike gcForest where the original features are concatenated to the outputs of classifiers to generate the input features for the subsequent layer, we integrate weights on the classifiers’ outputs as augmented features to grow the deep model. The usage of weights in the feature generation process can adjust the input data of each layer, leading the better results for the deep model. We encode the weights using variable-length encoding and develop a variable-length Particle Swarm Optimization method to search for the optimal values of the weights by maximizing the classification accuracy on the validation data. Experiments on a number of UCI datasets confirm the benefit of the proposed method compared to some well-known benchmark algorithms.

Research on fatigue identification methods based on low-load wearable ECG monitoring devices

The identification of fatigue in personal workers in particular environments can be achieved through early warning techniques. In order to prevent excessive fatigue of medical workers staying in infected areas in the early phase of the coronavirus disease pandemic, a system of low-load wearable electrocardiogram (ECG) devices was used as intelligent acquisition terminals to perform a continuous measurement ECG collection. While machine learning (ML) algorithms and heart rate variability (HRV) offer the promise of fatigue detection for many, there is a demand for ever-increasing reliability in this area, especially in real-life activities. This study proposes a random forest-based classification ML model to identify the four categories of fatigue levels in frontline medical workers using HRV. Based on the wavelet transform in ECG signal processing, stationary wavelet transform was applied to eliminate the main perturbation of ECG in the motion state. Feature selection was performed using ReliefF weighting analysis in combination with redundancy analysis to optimize modeling accuracy. The experimental results of the overall fatigue identification achieved an accuracy of 97.9% with an AUC value of 0.99. With the four-category identification model, the accuracy is 85.6%. These results proved that fatigue analysis based on low-load wearable ECG monitoring at low exertion can accurately determine the level of fatigue of caregivers and provide further ideas for researchers working on fatigue identification in special environments.

PRR-HyPred: A two-layer hybrid framework to predict pattern recognition receptors and their families by employing sequence encoded optimal features

Pattern recognition receptors (PRRs) recognize distinct features on the surface of pathogens or damaged cells and play key roles in the innate immune system. PRRs are divided into various families, including Toll-like receptors, retinoic acid-inducible gene-I-like receptors, nucleotide oligomerization domain-like receptors, and C-type lectin receptors. As these are implicated in host health and several diseases, their accurate identification is indispensable for their functional characterization and targeted therapeutic approaches. Here, we construct PRR-HyPred, a novel two-layer hybrid framework in which the first layer predicts whether a given sequence is PRR or non-PRR using a support vector machine, and in the second, the predicted PRR sequence is assigned to a specific family using a random forest-based classifier. Based on a 10-fold cross-validation test, PRR-HyPred achieved 83.4 % accuracy in the first layer and 95 % in the second, with Matthew's correlation coefficient values of 0.639 and 0.816, respectively. This is the first study that can simultaneously predict and classify PRRs into specific families. PRR-HyPred is available as a web portal at https://procarb.org/PRRHyPred/. We hope that it could be a valuable tool for the large-scale prediction and classification of PRRs and subsequently facilitate future studies.

Computational Prediction of the Phenotypic Effect of Flavonoids on Adiponectin Biosynthesis.

In silico machine learning applications for phenotype-based screening have primarily been limited due to the lack of machine-readable data related to disease phenotypes. Adiponectin, a nuclear receptor (NR)-regulated adipocytokine, is relatively downregulated in human metabolic diseases. Here, we present a machine-learning model to predict the adiponectin-secretion-promoting activity of flavonoid-associated phytochemicals (FAPs). We modeled a structure-activity relationship between the chemical similarity of FAPs and their bioactivities using a random forest-based classifier, which provided the NR activity of each FAP as a probability. To link the classifier-predicted NR activity to the phenotype, we next designed a single-cell transcriptomics-based multiple linear regression model to generate the relative adiponectin score (RAS) of FAPs. In experimental validation, estimated RAS values of FAPs isolated from Scutellaria baicalensis exhibited a significant correlation with their adiponectin-secretion-promoting activity. The combined cheminformatics and bioinformatics approach enables the computational reconstruction of phenotype-based screening systems.

Classification Approach for Attention Assessment via Singular Spectrum Analysis Based on Single-Channel Electroencephalograms

Attention refers to the human psychological ability to focus on doing an activity. The attention assessment plays an important role in diagnosing attention deficit hyperactivity disorder (ADHD). In this paper, the attention assessment is performed via a classification approach. First, the single-channel electroencephalograms (EEGs) are acquired from various participants when they perform various activities. Then, fast Fourier transform (FFT) is applied to the acquired EEGs, and the high-frequency components are discarded for performing denoising. Next, empirical mode decomposition (EMD) is applied to remove the underlying trend of the signals. In order to extract more features, singular spectrum analysis (SSA) is employed to increase the total number of the components. Finally, some typical models such as the random forest-based classifier, the support vector machine (SVM)-based classifier, and the back-propagation (BP) neural network-based classifier are used for performing the classifications. Here, the percentages of the classification accuracies are employed as the attention scores. The computer numerical simulation results show that our proposed method yields a higher classification performance compared to the traditional methods without performing the EMD and SSA.

A comparative assessment of the statistical methods based on urban population density estimation

Population density is important spatial information for addressing the use and access to land resources in cities under the Sustainable Development Goals. This is because the spatial data support appropriate spatial policies at the spatial scale and predicts how much land will be consumed in the future. The study aims to compare and evaluate the regression tools in the context of estimating the population density difference. The three analysis tools used are Random Forest-Based Classification, Multiple Linear Regression, and Geographically Weighted Regression. The sampling area covers cities around Türkiye. Comparative results showed that the two most important descriptive variables in the Random Forest-Based Classification model are the density difference of the new developed area and the connectivity. The three main explanatory variables of the Multiple Linear Regression model are centrality, vehicle ownership, and accessibility. The results of the Multiple Linear Regression model (a non-spatial model) and the Geographically Weighted Regression model (a spatial model), were found to be quite similar. The importance of accessibility and connectivity is more evident in the Multiple Linear Regression model when the Random Forest-Based Classification model highlights the density values in the new development areas.

Predicting the predisposition to colorectal cancer based on SNP profiles of immune phenotypes using supervised learning models.

This study explores the machine learning-based assessment of predisposition to colorectal cancer based on single nucleotide polymorphisms (SNP). Such a computational approach may be used as a risk indicator and an auxiliary diagnosis method that complements the traditional methods such as biopsy and CT scan. Moreover, it may be used to develop a low-cost screening test for the early detection of colorectal cancers to improve public health. We employ several supervised classification algorithms. Besides, we apply data imputation to fill in the missing genotype values. The employed dataset includes SNPs observed in particular colorectal cancer-associated genomic loci that are located within DNA regions of 11 selected genes obtained from 115 individuals. We make the following observations: (i) random forest-based classifier using one-hot encoding and K-nearest neighbor (KNN)-based imputation performs the best among the studied classifiers with an F1 score of 89% and area under the curve (AUC) score of 0.96. (ii) One-hot encoding together with K-nearest neighbor-based data imputation increases the F1 scores by around 26% in comparison to the baseline approach which does not employ them. (iii) The proposed model outperforms a commonly employed state-of-the-art approach, ColonFlag, under all evaluated settings by up to 24% in terms of the AUC score. Based on the high accuracy of the constructed predictive models, the studied 11 genes may be considered a gene panel candidate for colon cancer risk screening.

Photometric redshift-aided classification using ensemble learning

We present SHEEP, a new machine learning approach to the classic problem of astronomical source classification, which combines the outputs from the XGBoost, LightGBM, and CatBoost learning algorithms to create stronger classifiers. A novel step in our pipeline is that prior to performing the classification, SHEEP first estimates photometric redshifts, which are then placed into the data set as an additional feature for classification model training; this results in significant improvements in the subsequent classification performance. SHEEP contains two distinct classification methodologies: (i) Multi-class and (ii) one versus all with correction by a meta-learner. We demonstrate the performance of SHEEP for the classification of stars, galaxies, and quasars using a data set composed of SDSS and WISE photometry of 3.5 million astronomical sources. The resulting F1 -scores are as follows: 0.992 for galaxies; 0.967 for quasars; and 0.985 for stars. In terms of the F1-scores for the three classes, SHEEP is found to outperform a recent RandomForest-based classification approach using an essentially identical data set. Our methodology also facilitates model and data set explainability via feature importances; it also allows the selection of sources whose uncertain classifications may make them interesting sources for follow-up observations.

The Application of Deep Learning Imputation and Other Advanced Methods for Handling Missing Values in Network Intrusion Detection

In intelligent information systems data play a critical role. The issue of missing data is one of the commonplace problems occurring in data collected in the real world. The problem stems directly from the very nature of data collection. In this paper, the notion of handling missing values in a real-world application of computational intelligence is considered. Two experimental campaigns were conducted, evaluating different approaches to the missing values imputation on Random Forest-based classifiers, trained using modern cybersecurity benchmarks datasets: CICIDS2017 and IoT-23. In result of the experiments it transpired that the chosen algorithm for data imputation has a severe impact on the results of the classifier used for network intrusion detection. It also comes to light that one of the most popular approaches to handling missing data — complete case analysis — should never be used in cybersecurity.

Intrusion Detection Using Combination of GA Based Feature Selection and Random Forest Machine Learning Supervised Approach

As of late, the fast advancement of web innovation brings numerous serious organization security issues connected to vindictive interruptions. Interruption Detection System is viewed as one of the huge procedures to defend the organization from both outer and inward assaults. In any case, with the quick development of the IoT organization, cyberattacks are additionally evolving rapidly, and numerous obscure sorts are appearing in the contemporary organization climate. Thusly, the productivity of conventional mark based and oddity based Intrusion Detection System is inadequate. We propose a clever Intrusion Detection System, which utilizes a developmental strategy based include choice methodology and a Random Forest-based classifier. The development based include selector utilizes an imaginative Fitness Function to choose the significant elements and decreases aspects of the information, which raise the Ture Positive Rate and lessen the False Positive Rate simultaneously. With extraordinary high precision in multi-order errands and remarkable abilities of taking care of commotion in gigantic information situations, the Random Forest strategy is broadly utilized in peculiarity identification. This examination proposes a structure that can choose all the more consistent highlights and further develop the order results as contrasted and different innovations. The proposed structure is tried and investigated UNSW-NB15 datasets and NSL-KDD datasets. Different measurable outcomes and itemized correlation with different strategies are introduced inside this article.

A Heartbeat Classifier for Continuous Prediction Using a Wearable Device

Heartbeat monitoring may play an essential role in the early detection of cardiovascular disease. When using a traditional monitoring system, an abnormal heartbeat may not appear during a recording in a healthcare facility due to the limited time. Thus, continuous and long-term monitoring is needed. Moreover, the conventional equipment may not be portable and cannot be used at arbitrary times and locations. A wearable sensor device such as Polar H10 offers the same capability as an alternative. It has gold-standard heartbeat recording and communication ability but still lacks analytical processing of the recorded data. An automatic heartbeat classification system can play as an analyzer and is still an open problem in the development stage. This paper proposes a heartbeat classifier based on RR interval data for real-time and continuous heartbeat monitoring using the Polar H10 wearable device. Several machine learning and deep learning methods were used to train the classifier. In the training process, we also compare intra-patient and inter-patient paradigms on the original and oversampling datasets to achieve higher classification accuracy and the fastest computation speed. As a result, with a constrain in RR interval data as the feature, the random forest-based classifier implemented in the system achieved up to 99.67% for accuracy, precision, recall, and F1-score. We are also conducting experiments involving healthy people to evaluate the classifier in a real-time monitoring system.

RAEN: Rate Adaptation for Effective Nodes in Backscatter Networks.

A backscatter network, as a key enabling technology for interconnecting plentiful IoT sensing devices, can be applicable to a variety of interesting applications, e.g., wireless sensing and motion tracking. In these scenarios, the vital information-carrying effective nodes always suffer from an extremely low individual reading rate, which results from both unpredictable channel conditions and intense competition from other nodes. In this paper, we propose a rate-adaptation algorithm for effective nodes (RAEN), to improve the throughput of effective nodes, by allowing them to transmit exclusively and work in an appropriate data rate. RAEN works in two stages: (1) RAEN exclusively extracts effective nodes with an identification module and selection module; (2) then, RAEN leverages a trigger mechanism, combined with a random forest-based classifier, to predict the overall optimal rate. As RAEN is fully compatible with the EPC C1G2 standard, we implement the experiment through a commercial reader and multiple RFID tags. Comprehensive experiments show that RAEN improves the throughput of effective nodes by 3×, when 1/6 of the nodes are effective, compared with normal reading. What is more, the throughput of RAEN is better than traditional rate-adaptation methods.

Immunological Predictors of Post Infectious Inflammatory Response Syndrome in HIV-Negative Immunocompetent Cryptococcal Meningitis.

ObjectiveThis research aims to study the correlation between serum immune factors and post-infectious inflammatory response syndrome (PIIRS) in immunocompetent cryptococcal meningitis (CM), and explore whether serum immune factors could be used to predict the development of PIIRS.MethodsA cohort of 30 patients with PIIRS and 87 patients without PIIRS was selected from 347 CM patients. We analyzed the general clinical information and immunological indexes (cytokines, complement, immunoglobulin, inflammation, related cytological and biochemical indexes). Spearman correlation analysis and principal component analysis were used to explore the effects of the variables on PIIRS. Additionally, the variables were identified by a random forest-based classifier for predicting the development of PIIRS. The clinical value of predictors was verified by survival analysis.ResultsCompared with patients without PIIRS, patients with PIIRS had lower baseline serum interleukin-6 (IL-6, P = 0.006), immunoglobulin M (IgM, P = 0.004), and a higher baseline neutrophil ratio (P <0.001). The baseline neutrophil ratio (r = 0.359, P = 0.001), IgM (r = −0.272, P = 0.025), and IL-6 (r = −0.259, P = 0.027) were significantly correlated with PIIRS. Combining principal component analysis and random forest results, neutrophil ratio, neutrophil count, IgM, IL-6, and D-dimer were useful predictors. The accuracy of random forest prediction was 75.00%, AUC, and sensitivity were 0.76 and 70%, respectively. Further survival analysis of the time from treatment to PIIRS revealed that the development of PIIRS was associated with IgM (more than 98 days of treatment) and neutrophil ratio/count.ConclusionBaseline neutrophils ratio, neutrophil count, IgM, IL-6, and D-dimer may be clinically useful predictors of PIIRS in HIV-negative immunocompetent CM patients.