A Regression Residual Process

Abstract

A residual process obtained from regression model fitting is studied. The process is a partial sum of reordered residuals in such a way that residuals near small order statistics are added in first. In the case of fitting a correct model by the method of least squares, the partial sum process converges to a Brownian bridge. If an incorrect model is fit, the partial sum process adds the residuals in a manner to accumulate locally negative residuals first, thereby giving a possibly useful detection tool. Finally a simulation study of a Kolmogorov–Smirnov supremum statistic of the partial sum process is made, with a comparison of other competing test statistics. The main result shows that one can produce a natural graphical plot and its associated distribution as a diagnostic for linear regression of a variable against time. The weak convergence also enables the study of continuous functionals of the process. The process, called the random reordering residual process, accumulates partial sums of residuals to pick up local trend discrepancies of regression model fitting.