Empirical bayes estimates of domain scores under binomial and hypergeometric distributions for test scores

Abstract

We introduce two simple empirical approximate Bayes estimators (EABEs)—\(\widetilde{d}_N (x)\) and\(\widetilde\delta _N (x)\)—for estimating domain scores under binomial and hypergeometric distributions, respectively. Both EABEs (derived from corresponding marginal distributions of observed test scorex without relying on knowledge of prior domain score distributions) have been proven to hold Δ-asymptotic optimality in Robbins' sense of convergence in mean. We found that, where\(\widetilde{d}^* _N\) and\(\widetilde\delta ^* _N\) are the monotonized versions of\(\widetilde{d}_N\) and\(\widetilde\delta _N\) under Van Houwelingen's monotonization method, respectively, the convergence rate of the overall expected loss of Bayes risk in either\(\widetilde{d}^* _N\) or\(\widetilde\delta ^* _N\) depends on test length, sample size, and ratio of test length to size of domain items. In terms of conditional Bayes risk,\(\widetilde{d}^* _N\) and\(\widetilde\delta ^* _N\) outperform their maximum likelihood counterparts over the middle range of domain scales. In terms of mean-squared error, we also found that: (a) given a unimodal prior distribution of domain scores,\(\widetilde\delta ^* _N\) performs better than both\(\widetilde{d}^* _N\) and a linear EBE of the beta-binomial model when domain item size is small or when test items reflect a high degree of heterogeneity; (b)\(\widetilde{d}^* _N\) performs as well as\(\widetilde\delta ^* _N\) when prior distribution is bimodal and test items are homogeneous; and (c) the linear EBE is extremely robust when a large pool of homogeneous items plus a unimodal prior distribution exists.