Effects of Allowing Examinees to Select Questions on a Test of Divergent Thinking

Abstract

Under some circumstances, allowing examinees to choose which test items to respond to may increase test validity. In this study, we explored how choice, that is, allowing examinees to assign themselves to test questions, affected examinee performance and test characteristics for a measure of the ability to generate hypotheses about a given situation. Four forms of this experimental test were randomly spiraled within a large sample of students taking the computer-based Graduate Record Examinations General Test, with 2 of the forms allowing examinee choice. Results suggested that items were in fact differentially attractive to examinees, indicating that in some form choice was operating. Furthermore, performance was significantly higher when a given item was chosen than when it was required. There was some indication that internal consistency was improved by allowing examinee choice. Finally, the difficulties in designing definitive studies of the effects of examinee choice are discussed.