An Investigation of the Impact of Misrouting Under Two‐Stage Multistage Testing: A Simulation Study

Abstract

The purpose of this study was to investigate the potential impact of misrouting under a 2‐stage multistage test (MST) design, which includes 1 routing and 3 second‐stage modules. Simulations were used to create a situation in which a large group of examinees took each of the 3 possible MST paths (high, middle, and low). We compared differences in examinees' scores associated with different paths under 2 MST assembly conditions: a small‐difference condition in which 3 modules at Stage 2 overlap in difficulty, and a large‐difference condition in which the 3 modules at Stage 2 are distinct in difficulty. We also compared examinees' score differences associated with the target MST path with the ones obtained when the second‐stage module was 1 level of difficulty above or below the target module. When the second‐stage modules overlapped in difficulty, the score differences (i.e., bias) associated with different MST paths were negligible for practical purposes. Similar trends appeared even when the second‐stage modules were significantly distinct in difficulty. The impact of misrouting was generally minimal under the MST design used in this study.