Oblique random forest ensemble via Least Square Estimation for time series forecasting

Abstract

Recent studies in Machine Learning indicates that the classifiers most likely to be the bests are the random forests. As an ensemble classifier, random forest combines multiple decision trees to significant decrease the overall variances. Conventional random forest employs orthogonal decision tree which selects one “optimal” feature to split the data instances within a non-leaf node according to impurity criteria such as Gini impurity, information gain and so on. However, orthogonal decision tree may fail to capture the geometrical structure of the data samples. Motivated by this, we make the first attempt to study the oblique random forest in the context of time series forecasting. In each node of the decision tree, instead of the single “optimal” feature based orthogonal classification algorithms used by standard random forest, a least square classifier is employed to perform partition. The proposed method is advantageous with respect to both efficiency and accuracy. We empirically evaluate the proposed method on eight generic time series datasets and five electricity load demand time series datasets from the Australian Energy Market Operator and compare with several other benchmark methods.