Abstract
BackgroundMany medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs.MethodsThe analyses included applicants interviewing at ≥1 of the five schools during 2011–2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date.ResultsA total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson’s r 0.07, 0.13, and 0.29, and Cronbach’s α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach’s α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20–0.35) for TIs and 0.47 (95% CI 0.41–0.54) for MMIs.ConclusionsWithin and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.
Highlights
Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because Multiple MiniInterview (MMI) are thought to be more reliable
Using data from the five California Longitudinal Evaluation of Admission Practices (CA-LEAP) consortium medical schools, we examined the within- and betweenschool reliabilities of MMIs and TIs
All applicants who interviewed at one or more TI school contributed to the TI intraclass correlation coefficient (ICC) analyses, and all applicants who interviewed at one or more MMI school contributed to the MMI ICC analyses. For both MMI and TI analyses, we developed mixed linear models [22] with applicants as random effects to derive the ICCs for interview z-scores at TI and MMI schools Both the TI and MMI analyses were conducted with and without adjusting for the following fixed effects: applicant characteristics, number of interviews, number of prior interviews, interview date within interview season, and interview year
Summary
Many medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Prior MMI (and TI) studies have been conducted at single institutions, each employing only one of these interview types. While valuable, such studies have relatively small samples sizes, since at any given school most applicants are not selected for an interview, reducing generalizability. Single-school interview studies have limited utility in comparing the relative reliabilities of MMIs and TIs, due to fundamental differences in designs, analytic approaches, and time frames among studies. No studies have concurrently tested whether inter-rater reliability is higher for MMIs than for TIs by examining a common pool of applicants completing both interview types. As key differences in MMI (and TI) implementation exist among schools, [18] high between-school reliability of the MMI and TI cannot be assumed
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
