The lower tail of random quadratic forms with applications to ordinary least squares

Abstract

Finite sample properties of random covariance-type matrices have been the subject of much research. In this paper we focus on the “lower tail” of such a matrix, and prove that it is sub-Gaussian under a simple fourth moment assumption on the one-dimensional marginals of the random vectors. A similar result holds for more general sums of random positive semidefinite matrices, and our (relatively simple) proof uses a variant of the so-called PAC-Bayesian method for bounding empirical processes. Using this bound, we obtain a nearly optimal finite-sample result for the ordinary least squares estimator under random design.