A Monte Carlo investigation of the robustness of the Wald, likelihood ratio, and median tests with specified symmetric and asymmetric marginal distributions

Abstract

The effect of nonnormality on the Type I (τ) error when comparing two independent binomial proportions (P) or the nonparametric alternatives, the Median (Me), Wald (W), and Likelihood Ratio (LR), has not been investigated. If these selected tests are overly conservative the implied loss of power would moderate their practical use. The purpose of the present study was to investigate the impact of nonnormality on small to moderate sample sizes on the estimated τ for α = 0.10, 0.05 and 0.01 for the P, Me, W, and LR tests. Samples were generated from nine long-tailed symmetric and asymmetric distributions using a multiplicative congruential generator. For each marginal distribution and for a variety of sample sizes, the proportion of samples for which the test statistic exceeded the 10, 5, and 1 percentage points was tabulated. For data that mimic a symmetric distribution, the median test uniformly yields an empirical α considerably less than τ, while the likelihood ratio consistently overestimates τ for small samples ( n ≤ 15) over all symmetric distributions and empirical α levels. For asymmetric distributions, the median test again yields an empirical α significantly less than τ. Similar underestimates of τ were found for the chi-square (2 df), chi-square (4 df) log normal, and gamma (2, 1) distributions. The likehood ratio test consistently overestimates τ for small samples ( n ≤ 15) over all asymmetric distributions and empirical α levels. The independent proportions test produces an empirical α closest to τ for n = 10 for all asymmetric distributions. The most consistent test procedure, disregarding sample size or distribution characteristics, is the independent proportions test procedure.