Online Model Selection Research Articles

Multi‐armed bandit based online model selection for concept‐drift adaptation

AbstractEnsemble methods are among the most effective concept‐drift adaptation techniques due to their high learning performance and flexibility. However, they are computationally expensive and pose a challenge in applications involving high‐speed data streams. In this paper, we present a computationally efficient heterogeneous classifier ensemble entitled OMS‐MAB which uses online model selection for concept‐drift adaptation by posing it as a non‐stationary multi‐armed bandit (MAB) problem. We use a MAB to select a single adaptive learner within the ensemble for learning and prediction while systematically exploring promising alternatives. Each ensemble member is made drift resistant using explicit drift detection and is represented as an arm of the MAB. An exploration factor controls the trade‐off between predictive performance and computational resource requirements, eliminating the need to continuously train and evaluate all the ensemble members. A rigorous evaluation on 20 benchmark datasets and 9 algorithms indicates that the accuracy of OMS‐MAB is statistically at par with state‐of‐the‐art (SOTA) ensembles. Moreover, it offers a significant reduction in execution time and model size in comparison to several SOTA ensemble methods, making it a promising ensemble for resource constrained stream‐mining problems.

A novel reinforced online model selection using Q-learning technique for wind speed prediction

An accurate wind speed prediction is a fundamental prerequisite for enhanced wind energy integration with grid. Existing forecasting models are trained on the wind speed data. And these models rely on global accuracy without considering local variation of wind. Due to the local variation of wind, performance of each model varies for every time-step. As a result, this paper implements a robust and novel hybrid framework i.e. online model selection using Q-learning (OMS-QL) provided forecasting model pool (FMP). The proposed framework is the first model developed for online selection of best forecasting model dynamically using reinforcement learning (RL) approach. The proposed framework is mainly clustered into two parts: FMP, and Q-learning agent for online model selection. First, FMP is constructed using nine robust approaches, which are trained on the wind speed data. Then, Q-learning agent is developed to dynamically select the best prediction approach at every time-step for improved accuracy. Two experiments are conducted using the real-time wind speed datasets. Experimental results indicate that the proposed OMS-QL framework improved by 47% and 48% in both case studies compared to benchmark models.

An Adaptive Heterogeneous Online Learning Ensemble Classifier for Nonstationary Environments.

In recent years, the prevalence of technological advances has led to an enormous and ever-increasing amount of data that are now commonly available in a streaming fashion. In such nonstationary environments, the underlying process generating the data stream is characterized by an intrinsic nonstationary or evolving or drifting phenomenon known as concept drift. Given the increasingly common applications whose data generation mechanisms are susceptible to change, the need for effective and efficient algorithms for learning from and adapting to evolving or drifting environments can hardly be overstated. In dynamic environments associated with concept drift, learning models are frequently updated to adapt to changes in the underlying probability distribution of the data. A lot of work in the area of learning in nonstationary environments focuses on updating the learning predictive model to optimize recovery from concept drift and convergence to new concepts by adjusting parameters and discarding poorly performing models while little effort has been dedicated to investigate what type of learning model is suitable at any given time for different types of concept drift. In this paper, we investigate the impact of heterogeneous online ensemble learning based on online model selection for predictive modeling in dynamic environments. We propose a novel heterogeneous ensemble approach based on online dynamic ensemble selection that accurately interchanges between different types of base models in an ensemble to enhance its predictive performance in nonstationary environments. The approach is known as Heterogeneous Dynamic Ensemble Selection based on Accuracy and Diversity (HDES-AD) and makes use of models generated by different base learners to increase diversity to circumvent problems associated with existing dynamic ensemble classifiers that may experience loss of diversity due to the exclusion of base learners generated by different base algorithms. The algorithm is evaluated on artificial and real-world datasets with well-known online homogeneous online ensemble approaches such as DDD, AFWE, and OAUE. The results show that HDES-AD performed significantly better than the other three homogeneous online ensemble approaches in nonstationary environments.

A heterogeneous online learning ensemble for non-stationary environments

Learning in non-stationary environments is a challenging task which requires the updating of predictive models to deal with changes in the underlying probability distribution of the problem, i.e., dealing with concept drift. Most work in this area is concerned with updating the learning system so that it can quickly recover from concept drift, while little work has been dedicated to investigating what type of predictive model is most suitable at any given time. This paper aims to investigate the benefits of online model selection for predictive modelling in non-stationary environments. A novel heterogeneous ensemble approach is proposed to intelligently switch between different types of base models in an ensemble to increase the predictive performance of online learning in non-stationary environments. This approach is Heterogeneous Dynamic Weighted Majority (HDWM). It makes use of “seed” learners of different types to maintain ensemble diversity, overcoming problems of existing dynamic ensembles that may undergo loss of diversity due to the exclusion of base learners. The algorithm has been evaluated on artificial and real-world data streams against existing well-known approaches such as a heterogeneous Weighted Majority Algorithm (WMA) and a homogeneous Dynamic Weighted Majority (DWM). The results show that HDWM performed significantly better than WMA in non-stationary environments. Also, when recurring concept drifts were present, the predictive performance of HDWM showed an improvement over DWM.

Model predictive control with active learning for stochastic systems with structural model uncertainty: Online model discrimination

Structural model uncertainty is prevalent in control design and arises from incomplete knowledge of the system or the existence of different modes of dynamic behavior, such as those arising from system faults and malfunctions. This paper addresses control of stochastic nonlinear systems using model predictive control, or mpc, under structural model uncertainty. Inspired by dual control, the mpc strategy with active learning presented here can probe the uncertain system to select, among a set of candidates, the model that best describes the observed closed-loop system data. The proposed controller involves online model selection based on estimation of the model-hypothesis probabilities and minimization of a computationally tractable measure of the predicted Bayes risk of selection error. The performance of the proposed approach is compared to that of nominal mpc with no learning, mpc with passive learning, and a robust mpc approach that systematically accounts for structural model uncertainty but has no learning mechanism. Simulation results on a nonlinear bioreactor demonstrate that active learning can have significant advantages in maintaining adequate control performance in the presence of structural uncertainty. Active learning can be particularly beneficial for improving online model discrimination and active fault diagnosis under closed-loop control.

Adaptive SVM for Data Stream Classification

In this paper, we address the problem of learning an adaptive classifier for the classification of continuous streams of data. We present a solution based on incremental extensions of the Support Vector Machine (SVM) learning paradigm that updates an existing SVM whenever new training data are acquired. To ensure that the SVM effectiveness is guaranteed while exploiting the newly gathered data, we introduce an on-line model selection approach in the incremental learning process. We evaluated the proposed method on real world applications including on-line spam email filtering and human action classification from videos. Experimental results show the effectiveness and the potential of the proposed approach.

Cooperative parallel particle filters for online model selection and applications to urban mobility

We design a sequential Monte Carlo scheme for the dual purpose of Bayesian inference and model selection. We consider the application context of urban mobility, where several modalities of transport and different measurement devices can be employed. Therefore, we address the joint problem of online tracking and detection of the current modality. For this purpose, we use interacting parallel particle filters, each one addressing a different model. They cooperate for providing a global estimator of the variable of interest and, at the same time, an approximation of the posterior density of each model given the data. The interaction occurs by a parsimonious distribution of the computational effort, with online adaptation for the number of particles of each filter according to the posterior probability of the corresponding model. The resulting scheme is simple and flexible. We have tested the novel technique in different numerical experiments with artificial and real data, which confirm the robustness of the proposed scheme.

Probabilistic wind power forecasting with online model selection and warped gaussian process

Based on the online model selection and the warped Gaussian process (WGP), this paper presents an ensemble model for the probabilistic wind power forecasting. This model provides the non-Gaussian predictive distributions, which quantify the non-Gaussian uncertainties associated with wind power. In order to follow the time-varying characteristics of wind generation, multiple time dependent base forecasting models and an online model selection strategy are established, thus adaptively selecting the most probable base model for each prediction. WGP is employed as the base model, which handles the non-Gaussian uncertainties in wind power series. Furthermore, a regime switch strategy is designed to modify the input feature set dynamically, thereby enhancing the adaptiveness of the model. In an online learning framework, the base models should also be time adaptive. To achieve this, a recursive algorithm is introduced, thus permitting the online updating of WGP base models. The proposed model has been tested on the actual data collected from both single and aggregated wind farms.

Online model selection approach based on Unscented Kalman Filtering

Highly predictive mathematical models are of inestimable value in systems biology. Their application ranges from investigations of basic processes in living organisms up to model based drug design in the field of pharmacology. For the development of reliable models suitable model candidates and related model parameters have to be identified by minimising the difference between the model outcome and available measurement data. Due to the complexity of the analysed processes mathematical models capture only the essential features of interest. This approximate representation, which is usually combined with a vague knowledge of basic processes, leads in many cases to a variety of potential model candidates describing the real process almost equally well. To determine the most plausible model candidate is the objective of model selection or model discrimination methods. If under given operation conditions no sufficient discrimination can be achieved, Optimal Experimental Designs (OED) comes into play. OED searches for operation conditions which facilitate the overall selection process. In this work an online model selection framework is presented. Here, the Unscented Kalman Filter (UKF) provides statistical information which is used to assign probability values to every model candidate. These probability values are immediately updated as soon as new measurement data become available. In addition, during the experimental run the process is steered in a fashion which maximises the differences in these candidates. To overcome limitations caused by parameter uncertainties the most sensitive model parameters are simultaneously estimated in the course of the model selection framework. The combined application of the online framework and the joint estimation of sensitive model parameters provides a very efficient usage of measurement data reducing the overall number of experiments. The method is demonstrated for a well known motif in signalling pathways, the mitogen-activated protein (MAP) kinase.

Online Model Selection Based on the Variational Bayes

The Bayesian framework provides a principled way of model selection. This framework estimates a probability distribution over an ensemble of models, and the prediction is done by averaging over the ensemble of models. Accordingly, the uncertainty of the models is taken into account, and complex models with more degrees of freedom are penalized. However, integration over model parameters is often intractable, and some approximation scheme is needed. Recently, a powerful approximation scheme, called the variational bayes (VB) method, has been proposed. This approach defines the free energy for a trial probability distribution, which approximates a joint posterior probability distribution over model parameters and hidden variables. The exact maximization of the free energy gives the true posterior distribution. The VB method uses factorized trial distributions. The integration over model parameters can be done analytically, and an iterative expectation-maximization-like algorithm, whose convergence is guaranteed, is derived. In this article, we derive an online version of the VB algorithm and prove its convergence by showing that it is a stochastic approximation for finding the maximum of the free energy. By combining sequential model selection procedures, the online VB method provides a fully online learning method with a model selection mechanism. In preliminary experiments using synthetic data, the online VB method was able to adapt the model structure to dynamic environments.