Abstract
BackgroundThis study examined the influence of two Situational Judgement Test (SJT) design features (response instructions and response format) on applicant perceptions. Additionally, we investigated demographic subgroup differences in applicant perceptions of an SJT.MethodsMedical school applicants (N = 372) responded to an online survey on applicant perceptions, including a description and two example items of an SJT. Respondents randomly received one of four SJT versions (should do-rating, should do-pick-one, would do-rating, would do-pick-one). They rated overall favourability and items on four procedural justice factors (face validity, applicant differentiation, study relatedness and chance to perform) and ease-of-cheating. Additionally, applicant perceptions were compared for subgroups based on gender, ethnic background and first-generation university status.ResultsApplicants rated would-do instructions as easier to cheat than should-do instructions. Rating formats received more favourable judgements than pick-one formats on applicant differentiation, study-relatedness, chance to perform and ease of cheating. No significant main effect for demographic subgroup on applicant perceptions was found, but significant interaction effects showed that certain subgroups might have more pronounced preferences for certain SJT design features. Specifically, ethnic minority applicants – but not ethnic majority applicants – showed greater preference for should-do than would-do instructions. Additionally, first-generation university students – but not non-first-generation university students – were more favourable of rating formats than of pick-one formats.ConclusionsFindings indicate that changing SJT design features may positively affect applicant perceptions by promoting procedural justice factors and reducing perceived ease of cheating and that response instructions and response format can increase the attractiveness of SJTs for minority applicants.
Highlights
This study examined the influence of two Situational Judgement Test (SJT) design features on applicant perceptions
The current study examines the influence of two SJT design features, namely response instructions and response format, on applicant perceptions
Setting and procedure This study was conducted at a Dutch medical school, whose admission procedure consisted of three -weighted parts: i) pre-university grade point average, ii) extracurricular activities and iii) performance on three cognitive tests during an on-site testing day
Summary
This study examined the influence of two Situational Judgement Test (SJT) design features (response instructions and response format) on applicant perceptions. We investigated demographic subgroup differences in applicant perceptions of an SJT. The growing popularity of the SJT is a result of the test’s psychometric qualities, in terms of its predictive validity, incremental validity and low adverse impact, from the perspective of medical school admission committees [5]. The quality of an SJT should be investigated from the perspective of medical school applicants, since applicant perceptions may influence test-taking motivation, test performance and applicant withdrawal [6, 7]. The current study examines the influence of two SJT design features, namely response instructions and response format, on applicant perceptions. The perceptions of the SJT are compared for applicants belonging to different demographic subgroups
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
