Some Properties of a Bayesian Adaptive Ability Testing Strategy

Abstract

Four monte carlo simulation studies of Owen's Bayesian sequential procedure for adaptive mental testing were conducted. In contrast to previous simulation studies of this procedure which have concentrated on evaluating it in terms of the corre lation of its test scores with simulated ability in a normal population, these four studies explored a number of additional properties, both in a normally distributed population and in a distribution-free context. Study 1 replicated previous studies with finite item pools, but examined such properties as the bias of estimate, mean absolute error, and cor relation of test length with ability. Studies 2 and 3 examined the same variables in a number of hypo thetical infinite item pools, investigating the effects of item discriminating power, guessing, and vari able vs. fixed test length. Study 4 investigated some properties of the Bayesian test scores as latent trait estimators. The properties of interest included the conditional bias of the ability estimates, the infor mation curve of the trait estimates, and the rela tionship of test length to ability level. The results of these studies indicated that the ability estimates de rived from the Bayesian testing strategy were highly correlated with ability level. However, the ability es timates were also highly correlated with number of items administered, were non-linearly biased and provided measurements which were not of equal precision at all levels of ability.