Parameter Estimation With Small Sample Size A Higher-Order IRT Model Approach

Abstract

Sample size ranks as one of the most important factors that affect the item calibration task. However, due to practical concerns (e.g., item exposure) items are typically calibrated with much smaller samples than what is desired. To address the need for a more flexible framework that can be used in small sample item calibration, this article proposes an approach that accounts for the dimensionality of the assessments in the calibration process. This approach is based on the higher-order item response theory (HO-IRT) model. The HO-IRT model is a multi-unidimensional model that uses in-test collateral information and represents it in the correlational structure of the domains through a higher-order latent trait formulation. Using Markov chain Monte Carlo in a hierarchical Bayesian framework, the item parameters, the overall and domain-specific abilities, and their correlations are estimated simultaneously. The feasibility and effectiveness of the proposed approach are investigated under varied conditions in a simulation study and illustrated using actual assessment data.