Predictive Stochastic Complexity Research Articles

Generalised linear model selection by the predictive least quasi-deviance criterion

We consider the problem of selecting a model with the best predictive ability in a class of generalised linear models. A predictive least quasi-deviance criterion is proposed to measure the predictive ability of a model. This criterion is obtained by applying the idea of the predictive minimum description length principle and the theory of quasi-likelihood functions. The resulting predictive quasi-deviance function is an extension of the predictive stochastic complexity of the model. Under rather weak conditions the predictive least quasi-deviance method is shown to be consistent in the sense that the probability of selecting the right model converges to one as the number of observations goes to infinity. Also we show that the selected model converges to the optimal model in expectation. The method is then modified for finite sample applications. Examples and simulation results are presented.

Forecasting exchange rates using feedforward and recurrent neural networks

AbstractIn this paper we investigate the out‐of‐sample forecasting ability of feedforward and recurrent neural networks based on empirical foreign exchange rate data. A two‐step procedure is proposed to construct suitable networks, in which networks are selected based on the predictive stochastic complexity (PSC) criterion, and the selected networks are estimated using both recursive Newton algorithms and the method of nonlinear least squares. Our results show that PSC is a sensible criterion for selecting networks and for certain exchange rate series, some selected network models have significant market timing ability and/or significantly lower out‐of‐sample mean squared prediction error relative to the random walk model.

On Rissanen's predictive stochastic complexity for stationary ARMA processes

Abstract The aim of this paper is to show that for a wide class of stationary ARMA processes Rissanen's predictive stochastic complexity is asymptotically equal to the lower bound provided by the Rissanen-Shannon inequality almost surely. An analogous theorem is proved for a more easily computable version of the predictive stochastic complexity based on the recursive estimation of the ARMA parameters.

Predictive stochastic complexity and model estimation for finite-state processes

Abstract It is shown that the predictive and nonpredictive stochastic complexities relative to the class of finite-state models are asymptotically equivalent in a probabilistic sense. To this end, a universal, sequential, noiseless coding scheme attaining the minimum description length (MDL) of the data with respect to this class is presented and investigated. It relies on an MDL-based estimator of the model structure, which is proved to be strongly consistent. An interpretation of this result is that a process ‘close’ to every process in the class, regardless of the model structure, can be constructed. This universal process can be employed in the solution of sequential decision problems like coding, prediction, and gambling, in an asymptotically optimal manner.

Stochastic Complexity in System Identification and Adaptive Control

We present a general result describing the effect of parameter uncertainty due to estimation error on system performance. For a class of estimation and control problems that we call quasi-Newton methods the cumulative effect of parameter uncertainty is described by giving the almost sure asymptotics for a practically computable predictive stochastic complexity. Interestingly the number of estimated parameter has to be replaced by what we call the effective dimension of the problem. Finally we consider the above problems in a nonparametric setting and among others prove a nonparametric version of Davisson's formula.