Comments on using alternatives to normal theory statistics in social and behavioural science.

Abstract

Some Myths Concerning Parametric and Nonparametric Tests by Hunter and May Hunter and May offer a paper on myths and misconceptions (M and M's) that is an excellent companion article to Brewer (1985), who wrote on M and M's in statistical textbooks. Brewer addressed hypothesis testing, confidence intervals, and sampling distributions and the Central Limit Theorem. Hunter and May extend the M and M motif to parametric and nonparametric statistics, particularly with reference to power, robustness, scale of measurement, the null hypothesis, and generality of application.In the section on power, Hunter and May point out that when underlying assumptions of the parametric test are violated nonparametric tests may be more powerful. They call this a knee - jerk argument because this fact is usually ignored in selecting tests. In considering alternatives to normal theory statistics, they offer what they consider to be the definitive argument: ... the reason some nonparametric tests are less powerful than parametric tests is not because they are nonparametric tests per se, but because they are rank or nominal - scale tests and therefore are based on less information.In contradistinction to their reasoning, consider the following analogy: both an accomplished opera singer sings and an off - key beginning tuba player plays dots and dashes of the International Morse Code. While some may consider the opera singer's notes to be sounds of music, there is, in fact, no more information in those dots and dashes than in the off - key notes of the beginning tuba player, with respect to the code. If the complexity and subtlety of what is often imagined to be included in interval scales is noise and not signal, parametric tests will have no more information available than a rank test, and will be less efficient by trying to discriminate a signal from noise when in fact there isn't any. This is my interpretation of Hemelrijk (1961): the cost of being robust with respect to both Type I and Type II error under nonnormality precludes the t test from remaining the Uniformly Most Powerful Unbiased test under nonnormality.In the M and M section on the robustness of parametric tests, they cite Micceri (1989) as evidence of the widespread problem of nonnormality in psychology and education data. Yet, there are many, many Monte Carlo studies that demonstrate that normal theory tests such as the F and t test are robust to departures from normality. These studies used well known mathematical functions (e.g., cauchy, chi - square, exponetional, uniform) to model real data and showed that so long as sample sizes are about equal, sample sizes are at least 20 - 25 per group, and the tests are two - tailed, rather than one - tailed, the t test is robust.Micceri's (1989) argument, echoed by Hunter and May, was that those mathematical functions are poor models of psychology and education data, and consequently Monte Carlo studies based on them are not convincing. His study pointed out how radical real distributions may be, such as the so - called multi - modal lumpy, extreme bimodal, extreme asymmetric, digit preference, and discrete mass at zero with gap distributions. Nevertheless, a Monte Carlo study by Sawilowsky and Blair (1992) demonstrated by sampling with replacement from Micceri's data sets, that so long as sample sizes were equal, about 20 - 25, and tests were two tailed, the independent and dependent samples t tests were robust by any definition.The real issue of the effects of nonnormality, as indicated by Sawilowsky and Blair (1992), is on the comparative power, not robustness, of the t test. For example, a Monte Carlo comparison (10,000 repetitions) of the power for the t test and Wilcoxon test with a sample size of (5,15) drawn from an extreme asymmetric distribution identified by Micceri (1989) indicated that at the .05 alpha level and effect size of .20Greek not transcribed, the power of the Wilcoxon test was . …