On the Cover

Abstract

The Alluvial Diagram of MST Paths Taken provides a mechanism for visualizing the sequence of modules seen by examinees in the context of a six-stage multistage adaptive test (with five decision points for routing between stages). Examinees start the test at one of five modules representing different difficulty levels, and are routed to subsequent modules based on IRT proficiency estimates with no constraints on routing. The colors correspond to starting level of examinees, and the relative width of the lines between stages illustrates the frequency of examinees following different sequences of modules throughout the stages of the test. This visualization is useful in that it depicts how the routing of examinees to modules of different difficulty levels stabilizes in terms of test levels as examinees pass through the stages of an MST. In the earlier stages of the test, there is more movement as examinee proficiency is computed based on fewer items. As the test progresses, and additional measurement information is gathered, the iteratively computed proficiency estimates stabilize, and consequently, there is less movement of examinees from level to level across stages in the latter part of the test. By and large, many examinees on this test stay within level across all six stages, or move to one level adjacent (higher or lower in difficulty). We see that much of the movement between stages involves examinees in the middle three difficulty levels, with most movement between levels tapering off by the fourth MST decision point. The white columns illustrate the proportion of examinees routed to modules at each of those stages, so this visualization also shows the relative exposure of modules to examinees within each stage. If you are interested in learning more about this informative data visualization method, contact the authors HyunJoo Jung (hyunjoo.jung2@gmail.com) and April L. Zenisky (azenisky@educ.umass.edu). We want to hear your feedback! Let us know what you think of the visualization by emailing Ally Shay Thomas (allythomas@pitt.edu).