Abstract
Several forced-choice (FC) computerized adaptive tests (CATs) have emerged in the field of organizational psychology, all of them employing ideal-point items. However, despite most items developed historically follow dominance response models, research on FC CAT using dominance items is limited. Existing research is heavily dominated by simulations and lacking in empirical deployment. This empirical study trialed a FC CAT with dominance items described by the Thurstonian Item Response Theory model with research participants. This study investigated important practical issues such as the implications of adaptive item selection and social desirability balancing criteria on score distributions, measurement accuracy and participant perceptions. Moreover, nonadaptive but optimal tests of similar design were trialed alongside the CATs to provide a baseline for comparison, helping to quantify the return on investment when converting an otherwise-optimized static assessment into an adaptive one. Although the benefit of adaptive item selection in improving measurement precision was confirmed, results also indicated that at shorter test lengths CAT had no notable advantage compared with optimal static tests. Taking a holistic view incorporating both psychometric and operational considerations, implications for the design and deployment of FC assessments in research and practice are discussed.
Highlights
It can be seen that, while the item mean utility parameters had no effect on the maximum amount of information gain, they had an impact on the location of where that maximum occurs
As a consequence of the optimal form design, the last parts provided less information in general compared to the first pairs
Information for scales A (left) and C (right) from pair {A235, A117}
Summary
Information Gain and Item Mean Utility Differences This study adopted the item mean utility parameter as a proxy for item social desirability. It can be seen that, while the item mean utility parameters had no effect on the maximum amount of information gain, they had an impact on the location of where that maximum occurs
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have