Finite sample properties of system identification of ARX models under mixing conditions

Abstract

The asymptotic convergence properties of system identification methods are well known, but comparatively little is known about the practical situation where only a finite number of data points are available. In this paper we consider the finite sample properties of prediction error methods for system identification. We consider ARX models and uniformly bounded criterion functions. The problem we pose is: how many data points are required in order to guarantee with high probability that the expected value of the identification criterion is close to its empirical mean value. The sample sizes are obtained using generalisations of risk minimisation theory to weakly dependent processes. We obtain uniform probabilistic bounds on the difference between the expected value of the identification criterion and the empirical value evaluated on the observed data points. The bounds are very general, in particular no assumption is made about the true system belonging to the model class. Further analysis shows that in order to maintain a given bound on the difference, the number of data points required grows at most at a polynomial rate in the model order and in many cases no faster than quadratically. The results obtained here generalise previous results derived for the case where the observed data was independent and identically distributed.