Robust directed tests of normality against heavy-tailed alternatives

Abstract

Many statistical procedures rely on the assumption that the observed data are normally distributed. Consequently, there exists a vast literature on tests of normality and their statistical properties. Today the most commonly used omnibus test for general use is the Shapiro–Wilk method while the Jarque–Bera test is the most popular omnibus test in economics and related applications. In a recent securities law case concerning an allegation of profit sharing, an inference is drawn on a relationship between the hypothetical profits, which customers made on days with special stock offerings, and the ratio of their commission business to those profits became an issue. Two common statistical tests are based on the Pearson or Spearman correlation coefficients. The results provided by the two tests are dramatically different with contradictory implications for the existence of a “profit sharing arrangement”. In particular, the Pearson correlation coefficient is of - 0.13 with a p -value of 0.35 while the Spearman coefficient is of - 0.68 with a p -value less than 0.0001 . However, if the data do not come from a bivariate normal distribution, the usual sampling theory for the Pearson correlation would not be applicable. As stock market data are typically heavy-tailed, a simple test of normality directed against heavy-tailed alternatives complemented by a graphical method is desired for presentation in a courtroom. In this paper we utilize the idea of the Shapiro–Wilk test whose test statistic W is the ratio of the classical standard deviation s n to the optimal L-estimator of the population spread. In particular, we replace the mean and standard deviation by the median and a robust estimator of spread in the Shapiro–Wilk statistic W . This eventually leads to a new more powerful directed test and a clearer QQ plot for assessing departures from normality in the tails. Somewhat surprisingly, the robust estimator of spread that yields the more powerful test of normality utilizes the average absolute deviation from the median (MAAD) while the statistic based on the median absolute deviation from the median (MAD) usually provides a clearer graphical display. The asymptotic distribution of the proposed test statistic is derived, and the use of the new test is illustrated by simulations and by application to data sets from the securities law, finance and meteorology.