A distribution-free technique to estimate the order of markovian dependence using entropy

Abstract

A model-order determination procedure for time series is described which is based on computing the differential entropy of the time series' multivariate amplitude probability density. The procedure, termed the differential entropy method, applies to linear and nonlinear models for the time series. Simulations were performed to determine the performance characteristics of the differential entropy method on first-order autoregressive models with Gaussian and non-Gaussian inputs and on a first-order nonlinear model. It produces accurate model-order estimates for the linear model once the correlation coefficient becomes large enough. The threshold for accurate estimates is dependent on the amplitude distribution of the model's input. This technique estimates the model order of data having nonlinear dependence structure more accurately than thead hoc application of the AIC and MDL methods. Extension of the method to higher-order models is described, but, because of computation complexity issues, the differential entropy method is best used for estimating small model orders.