Online Random Forest Research Articles

Research on Match Trends in Tennis Competitions Based on Bayesian Online Change Point Detection and Random Forest Prediction Model

Research on Match Trends in Tennis Competitions Based on Bayesian Online Change Point Detection and Random Forest Prediction Model

Online Random Feature Forests for Learning in Varying Feature Spaces

In this paper, we propose a new online learning algorithm tailored for data streams described by varying feature spaces (VFS), wherein new features constantly emerge and old features may stop to be observed over various time spans. Our proposed algorithm, named Online Random Feature Forests for Feature space Variabilities (ORF3V), provides a strategy to respect such feature dynamics by generating, updating, pruning, as well as online re-weighing an ensemble of what we call feature forests, which are generated and updated based on a compressed and storage efficient representation for each observed feature. We benchmark our algorithm on 12 datasets, including one novel real-world dataset of government COVID-19 responses collected through a crowd-sensing program in Spain. The empirical results substantiate the viability and effectiveness of our ORF3V algorithm and its superior accuracy performance over the state-of-the-art rival models.

Discovering spatial-temporal patterns via complex networks in investigating COVID-19 pandemic in the United States.

A novel approach combining time series analysis and complex network theory is proposed to deeply explore characteristics of the COVID-19 pandemic in some parts of the United States (US). It merges as a new way to provide a systematic view and complementary information of COVID-19 progression in the US, enabling evidence-based responses towards pandemic intervention and prevention. To begin with, the Principal Component Analysis (PCA) varimax is adopted to fuse observed time-series data about the pandemic evolution in each state across the US. Then, relationships between the pandemic progress of two individual states are measured by different synchrony metrics, which can then be mapped into networks under unique topological characteristics. Lastly, the hidden knowledge in the established networks can be revealed from different perspectives by network structure measurement, community detection, and online random forest, which helps to inform data-driven decisions for battling the pandemic. It has been found that states gathered in the same community by diffusion entropy reducer (DER) are prone to be geographically close and share a similar pattern and tendency of COVID-19 evolution. Social factors regarding the political party, Gross Domestic Product (GDP), and population density are possible to be significantly associated with the two detected communities within a constructed network. Moreover, the cluster-specific predictor based on online random forest and sliding window is proven useful in dynamically capturing and predicting the epidemiological trends for each community, which can reach the highest.

Long short-term memory self-adapting online random forests for evolving data stream regression

Evolving data stream, especially with concept drift is generally accepted as a challenging data type for regression task, because it usually makes machine learning models trained on old data not adapting to new data, and leads to dramatic performance degradation as a result. Moreover, the behavior of a data stream may change in different modes and therefore introduces various concept drifts, e.g., abrupt, incremental, gradual, recurring, even more, complex concept drifts. Although there are some algorithms that can adapt to stationary data streams or a specific type of concept drift in non-stationary data streams, a wide range of practical applications call for machine learning regression models to handle multi-type of data streams. In this work, we propose an online learning strategy called adaptive long and short-term memories online Random Forests regression(ALSM-RFR), where an adaptive memory activation mechanism is designed to make the model switch adaptively between long-term and hybrid (long-term plus short-term) memory modes in the face of stationary data streams or non-stationary data streams with different types of concept drift. In particular, leaf and tree weights in random forests are used to learn information at different timescales, namely, long-term and short-term memories. Moreover, we devise an adaptive memory activation mechanism to formulate the switch decision of memory modes as a classification problem. Numerical experiments show remarkable improvements of the proposed method in the adaptability of stream types and predictive accuracy in data streams across several real datasets and synthetic datasets, compared to the state-of-the-art online approaches. Besides, the convergence and the influence of the parameters involved in our method are evaluated.

Online Rebuilding Regression Random Forests

Continuous data streams mining is particularly challenging for machine learning. Many efforts have been devoted to propose online learning algorithms that can train iteratively from new coming data and provide evolutionary predictions. Compared to off-line approaches, these algorithms have shown better predictive performance and certain adaptation to high volume continuous data stream. However, a wide range of practical applications calls for regression models that can make adequate use of the large volume of pre-collected training data, meanwhile, handle continuous data stream with multi-type concept drifts, such as abrupt, gradual, incremental, recurring concept drifts. Random Forests(RFs) are an effective ensemble learning model for regression tasks. However, the fixed structure of RFs by off-line training has restricted its applicability for real-world tasks with dynamic data streams. To address these issues, we propose an online rebuilding strategy for the pre-trained Random Forests model, which is called Online Rebuilding Regression Random Forests(ORB-RRF). Specifically, a leaf-pruning technique and online reconstruction of subtrees based on the change of feature space on certain nodes are designed to adjust the local structure of regression tree to adapt to dynamic inputs. Numerical experiments with ORB-RRF show remarkable improvements in the adaptability in data stream and the predictive accuracy in several benchmark real datasets and synthetic datasets, compared to several state-of-art methods. Moreover, we show the convergence and stability of the proposed method.

Daily Routine Recognition for Hearing Aid Personalization

This work focuses on daily routine recognition to personalize the hearing aid (HA) configuration for each user. So far, there is only one public data set containing the data of two acceleration sensors taken under unconstrained real-life conditions of one person. Therefore, we create a realistic and extensive data set with seven subjects and a total length of 63449 min. For the recordings, the HA streams the acceleration and audio data to a mobile phone, where the user simultaneously annotates it. This builds the grounds for our comprehensive simulations, where we train a set of classifiers in an offline and online manner to analyze the model generalization abilities across subjects for high-level activities. To achieve this, we build a feature representation, which describes the recurring daily situations and environments well. For the offline classification, the deep neural network, multi-layer perceptron (MLP), and random forest (RF) trained in a person-dependent manner show the significantly best F-measure performance of 86.6%, 87.1%, and 87.3%, respectively. We confirm that for high-level activities the person-dependent model outperforms the independent one. In our online experiments, we personalize a model that was pretrained in a person-independent manner by daily updates. Thereby, multiple incremental learners and an online RF are tested. We demonstrate that MLP and RF improve the F-measure compared to the offline baselines.

Online random forests regression with memories

In recent years, the online schema of the conventional Random Forests(RFs) have attracted much attention because of its ability to handle sequential data or data whose distribution changes during the prediction process. However, most research on online RFs focuses on structural modification during the training stage, overlooking critical aspects of the sequential dataset, such as autocorrelation. In this paper, we demonstrate how to improve the predictive accuracy of the regression model by exploiting data correlation. Instead of modifying the structure of the off-line trained RFs, we endow RFs with memory during regression prediction through an online weight learning approach, which is called Online Weight Learning Random Forest Regression(OWL-RFR). Specifically, the weights of leaves are updated based on a novel adaptive stochastic gradient descent method, in which the adaptive learning rate considers the current and historical prediction bias ratios, compared with the static learning rate. Thus, leaf-level weight stores the learned information from the past data points for future correlated prediction. Compared with tree-level weight which only has immediate memory for current prediction, the leaf-level weight can provide long-term memory. Numerical experiments with OWL-RFR show remarkable improvements in predictive accuracy across several common machine learning datasets, compared to traditional RFs and other online approaches. Moreover, our results verify that the weight approach using the long-term memory of leaf-level weight is more effective than immediate dependency on tree-level weight. We show the improved effectiveness of the proposed adaptive learning rate in comparison to the static rate for most datasets, we also show the convergence and stability of our method.

Reducing False-Positive Results in Newborn Screening Using Machine Learning.

Newborn screening (NBS) for inborn metabolic disorders is a highly successful public health program that by design is accompanied by false-positive results. Here we trained a Random Forest machine learning classifier on screening data to improve prediction of true and false positives. Data included 39 metabolic analytes detected by tandem mass spectrometry and clinical variables such as gestational age and birth weight. Analytical performance was evaluated for a cohort of 2777 screen positives reported by the California NBS program, which consisted of 235 confirmed cases and 2542 false positives for one of four disorders: glutaric acidemia type 1 (GA-1), methylmalonic acidemia (MMA), ornithine transcarbamylase deficiency (OTCD), and very long-chain acyl-CoA dehydrogenase deficiency (VLCADD). Without changing the sensitivity to detect these disorders in screening, Random Forest-based analysis of all metabolites reduced the number of false positives for GA-1 by 89%, for MMA by 45%, for OTCD by 98%, and for VLCADD by 2%. All primary disease markers and previously reported analytes such as methionine for MMA and OTCD were among the top-ranked analytes. Random Forest’s ability to classify GA-1 false positives was found similar to results obtained using Clinical Laboratory Integrated Reports (CLIR). We developed an online Random Forest tool for interpretive analysis of increasingly complex data from newborn screening.

FORF-S: A Novel Classification Technique for Class Imbalance Problem

In recent years, the class imbalance problem that aims to correctly classify imbalanced data sets and improve the classification performance of minority instances has received attention. Such problem can be roughly described as one of the class(es) termed as minority class(es) contains much smaller instances than the others, also referred as majority class(es). To address this problem, a novel classification method called focused online random forest based on synthetic minority oversampling technique (FORF-SMOTE) is proposed in this paper and simply expressed as FORF-S, which constructs two online random forests respectively trained by original training dataset and new generated dataset, then further jointly consititute the model. Instead of oversampling the minority instances in data level, the algorithm in this paper is motivated by making the sampling strategies integrated into algorithm level to create classifiers, which can better identify the minority class. Moreover, the method is also compared with other state-of-the-art methods and the results have demonstrated that the proposed algorithm takes advantages of the aforementioned methods.

Significant wave height forecasting via an extreme learning machine model integrated with improved complete ensemble empirical mode decomposition

Data-intelligent algorithms designed for forecasting significant height of coastal waves over the relatively short time period in coastal zones can generate crucial information about enhancing the renewable energy production. In this study, a machine learning model is designed and evaluated for forecasting significant wave height (Hs) within the eastern coastal zones of Australia. The extreme learning machine (ELM) model is coupled with an improved complete ensemble empirical mode decomposition method with adaptive noise (ICEEMDAN) to design the proposed ICEEMDAN-ELM model. This model incorporates the historical lagged series of Hs as the model's predictor to forecast future Hs. The ICEEMDAN algorithm demarcates the original Hs data from January-2000 to March-2018, recorded at 30-min intervals, into decomposed signals i.e., intrinsic mode functions (IMFs) and a residual component. After decomposition, the partial autocorrelation function is determined for each IMF and the residual sub-series to determine the statistically significant lagged input dataset. The ELM model is applied for forecasting of each IMF by incorporating the significant antecedent Hs sub-series as inputs. Finally, all the forecasted IMFs are summed up to obtain the final forecasted Hs. The results are benchmarked with those from an online sequential extreme learning machine (OSELM) and random forest (RF) integrated with ICEEMDAN, i.e., the ICEEMDAN-OSELM and ICEEMDAN-RF models. The proposed ICEEMDAN-ELM model is tested geographically at two coastal sites of the Queensland state, Australia. The testing performance of all the standalone (ELM, OSELM, RF) and integrated models (ICEEMDAN-ELM, ICEEMDAN-OSELM, ICEEMDAN-RF), according to robust statistical error metrics, is satisfactory; however, the hybrid ICEEMDAN-ELM model is found to be a beneficial Hs forecasting tool in accordance to high performance accuracy. The proposed ICEEMDAN-ELM model can be considered as a pertinent decision-support framework and is vital for designing of reliable ocean energy converters.

Robust Target Tracking by Online Random Forests and Superpixels

This paper presents a robust joint discriminative appearance model-based tracking method using online random forests and mid-level feature (superpixels). To achieve superpixel-wise discriminative ability, we propose a joint appearance model that consists of two random forest-based models, i.e. , the background-target discriminative model (BTM) and the distractor-target discriminative model (DTM). More specifically, the BTM effectively learns discriminative information between the target object and the background. In contrast, the DTM is used to suppress distracting superpixels, which significantly improves the tracker’s robustness and alleviates the drifting problem. A novel online random forest regression algorithm is proposed to build the two models. The BTM and DTM are linearly combined into a joint model to compute a confidence map. Tracking results are estimated using the confidence map, in which the position and scale of the target are estimated orderly. Furthermore, we design a model updating strategy to adapt the appearance changes over time by discarding degraded trees of the BTM and DTM and initializing new trees as replacements. We test the proposed tracking method on two large tracking benchmarks, the CVPR2013 tracking benchmark and VOT2014 tracking challenge. Experimental results show that the tracker runs at real-time speed and achieves favorable tracking performance compared with the state-of-the-art methods. The results also suggest that the DTM improves tracking performance significantly and plays an important role in robust tracking.

Video Tracking Algorithm Based on Particle Filter and Online Random Forest

Because Particle Filter algorithm in video tracking has some problems, such as weight degradation, poor samples after resampling, being difficult to select the optimal important probability density, being difficult to accurately follow to the target again after tracking failure, the random forest learning algorithm is introduced into the Particle Filter tracking algorithm. The current estimated target state image is transmitted to the random forest detection module, the algorithm adopts reliable sample update strategy, the random forest model is used to detect the target area, and the decision tree detection module rapidly detects target state adjacent areas. In the case of target temporarily lost, two methods of collaboration eventually accurately allocate to the tracking target. The algorithm ensures the robustness of tracking, effectively avoids the drift problem.

Online classification for object tracking based on superpixel

Abstract Treating object tracking as a binary classification problem has been greatly explored in recent years. State-of-the-art classification based trackers perform better robustness than many of the other existing trackers. In this paper, we propose a collaborative model by incorporating the local and holistic models together which is corresponding to discriminative and generative models in tracking-classification-framework. At the local level, an on-line Random Forest (RF) classifier is trained to distinguish the superpixels of the object from the background. A series of local superpixels are used to represent the target, so as to adapt the appearance variances. The discriminative model is used to classify superpixels in the next frame as either belonging to the object or background. A confidence map consisting of dependability and stability is formed to measure the probabilities of superpxiels pertaining to the target from the classifier. A modified mean-shift is proposed to work on the confidence map to find the peak, where is the position of the target. Meanwhile, a separate component for managing the training set dynamically is employed to control the updating of the RF model. At the global level, the target is represented by covariance matrix of multi-scale bounding boxes. The generative model is applied for protection measure, which can effectively reduce the drifts during tracking. Experimental results demonstrate that our method is effective and performs favorably in comparison to the state-of-the-art trackers.

Video Tracking Algorithm Based on Kalman Filter and Online Random Forest

Video Tracking Algorithm Based on Kalman Filter and Online Random Forest

Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views.

Segmentation of the placenta from fetal MRI is challenging due to sparse acquisition, inter-slice motion, and the widely varying position and shape of the placenta between pregnant women. We propose a minimally interactive framework that combines multiple volumes acquired in different views to obtain accurate segmentation of the placenta. In the first phase, a minimally interactive slice-by-slice propagation method called Slic-Seg is used to obtain an initial segmentation from a single motion-corrupted sparse volume image. It combines high-level features, online Random Forests and Conditional Random Fields, and only needs user interactions in a single slice. In the second phase, to take advantage of the complementary resolution in multiple volumes acquired in different views, we further propose a probability-based 4D Graph Cuts method to refine the initial segmentations using inter-slice and inter-image consistency. We used our minimally interactive framework to examine the placentas of 16 mid-gestation patients from MRI acquired in axial and sagittal views respectively. The results show the proposed method has 1) a good performance even in cases where sparse scribbles provided by the user lead to poor results with the competitive propagation approaches; 2) a good interactivity with low intra- and inter-operator variability; 3) higher accuracy than state-of-the-art interactive segmentation methods; and 4) an improved accuracy due to the co-segmentation based refinement, which outperforms single volume or intensity-based Graph Cuts.

An Improved Fast Compressive Tracking Algorithm Based on Online Random Forest Classifier

The fast compressive tracking (FCT) algorithm is a simple and efficient algorithm, which is proposed in recent years. But, it is difficult to deal with the factors such as occlusion, appearance changes, pose variation, etc in processing. The reasons are that, Firstly, even if the naive Bayes classifier is fast in training, it is not robust concerning the noise. Secondly, the parameters are required to vary with the unique environment for accurate tracking. In this paper, we propose an improved fast compressive tracking algorithm based on online random forest (FCT-ORF) for robust visual tracking. Firstly, we combine ideas with the adaptive compressive sensing theory regarding the weighted random projection to exploit both local and discriminative information of the object. The second reason is the online random forest classifier for online tracking which is demonstrated with more robust to the noise adaptively and high computational efficiency. The experimental results show that the algorithm we have proposed has a better performance in the field of occlusion, appearance changes, and pose variation than the fast compressive tracking algorithm’s contribution.

Real-time WiFi localization of heterogeneous robot teams using an online random forest

In this paper we present a WiFi-based solution to the localization and mapping problem for teams of heterogeneous robots operating in unknown environments. By exploiting wireless signal strengths broadcast from access points, a robot with a large sensor payload creates a WiFi signal map that can then be shared and utilized for localization by sensor-deprived robots. In our approach, WiFi localization is cast as a classification problem. An online clustering algorithm processes incoming WiFi signals that are then incorporated into an online random forest (ORF). The algorithm's robustness is increased by a Monte Carlo localization algorithm whose sensor model exploits the results of the ORF classification. The proposed algorithm is shown to run in real-time, allowing the robots to operate in completely unknown environments, where a priori information such as a blue-print or the access points' location is unavailable. A comprehensive set of experiments not only compares our approach with other algorithms, but also validates the results across different scenarios covering both indoor and outdoor environments.

Online tree-based ensembles and option trees for regression on evolving data streams

The emergence of ubiquitous sources of streaming data has given rise to the popularity of algorithms for online machine learning. In that context, Hoeffding trees represent the state-of-the-art algorithms for online classification. Their popularity stems in large part from their ability to process large quantities of data with a speed that goes beyond the processing power of any other streaming or batch learning algorithm. As a consequence, Hoeffding trees have often been used as base models of many ensemble learning algorithms for online classification. However, despite the existence of many algorithms for online classification, ensemble learning algorithms for online regression do not exist. In particular, the field of online any-time regression analysis seems to have experienced a serious lack of attention. In this paper, we address this issue through a study and an empirical evaluation of a set of online algorithms for regression, which includes the baseline Hoeffding-based regression trees, online option trees, and an online least mean squares filter. We also design, implement and evaluate two novel ensemble learning methods for online regression: online bagging with Hoeffding-based model trees, and an online RandomForest method in which we have used a randomized version of the online model tree learning algorithm as a basic building block. Within the study presented in this paper, we evaluate the proposed algorithms along several dimensions: predictive accuracy and quality of models, time and memory requirements, bias–variance and bias–variance–covariance decomposition of the error, and responsiveness to concept drift.

Progressive 3D shape segmentation using online learning

In this article, we propose a progressive 3D shape segmentation method, which allows users to guide the segmentation with their interactions, and does segmentation gradually driven by their intents. More precisely, we establish an online framework for interactive 3D shape segmentation, without any boring collection preparation or training stages. That is, users can collect the 3D shapes while segment them, and the segmentation will become more and more precise as the accumulation of the shapes.Our framework uses Online Multi-Class LPBoost (OMCLP) to train/update a segmentation model progressively, which includes several Online Random forests (ORFs) as the weak learners. Then, it performs graph cuts optimization to segment the 3D shape by using the trained/updated segmentation model as the optimal data term. There exist three features of our framework. Firstly, the segmentation model can be trained gradually during the collection of the shapes. Secondly, the segmentation results can be refined progressively until users’ requirements are met. Thirdly, the segmentation model can be updated incrementally without retraining all shapes when users add new shapes. Experimental results demonstrate the effectiveness of our approach.

Robust Tracking with Discriminative Ranking Middle-Level Patches

The appearance model has been shown to be essential for robust visual tracking since it is the basic criterion to locating targets in video sequences. Though existing tracking-by-detection algorithms have shown to be greatly promising, they still suffer from the drift problem, which is caused by updating appearance models. In this paper, we propose a new appearance model composed of ranking middle-level patches to capture more object distinctiveness than traditional tracking-by-detection models. Targets and backgrounds are represented by both low-level bottom-up features and high-level top-down patches, which can compensate each other. Bottom-up features are defined at the pixel level, and each feature gets its discrimination score through selective feature attention mechanism. In top-down feature extraction, rectangular patches are ranked according to their bottom-up discrimination scores, by which all of them are clustered into irregular patches, named ranking middle-level patches. In addition, at the stage of classifier training, the online random forests algorithm is specially refined to reduce drifting problems. Experiments on challenging public datasets and our test videos demonstrate that our approach can effectively prevent the tracker drifting problem and obtain competitive performance in visual tracking.