Multiple Mini-interview Scores Research Articles

Is a veterinary student's performance on multiple-mini interviews affected by personality preferences?

ObjectivesThe objectives of this study were to evaluate the association between a student's Myers-Briggs Type Indicator (MBTI) preference pairs and resulting types and his or her multiple-mini interview (MMI) scores upon admission, and to determine the proportions of types among veterinary classes over five years. MethodsA cross-sectional study was conducted for data collected from 706 students admitted into the University of California Davis School of Veterinary Medicine (UCDSVM) program beginning in the fall of 2013 and ending in the fall of 2018. Data consisted of a candidate's MBTI preference pairs and types which were collected during the first week of enrollment and multiple-mini interview scores from his or her admission data. ResultsA total of 706 students from 5 classes completed the MBTI survey. Multivariate analysis showed no significant association between the MBTI preference pairs of extroversion and introversion (F(1, 697) = 3.30, p=0.0959), sensing and intuition (F(1, 697) = 0.40, p=0.4395), thinking and feeling (F(1, 697) = 3.59, p=0.0591), or judging and perceiving (F(1, 697) = 0.38, p = 0.5657) and MMI score. Analysis showed no trends (χ2 (60, N=706) =76.51, p=0.074) in the student's MBTI types over the 5-year period. ConclusionsThe MMI score of a candidate admitted into the UCDSVM is unlikely to be affected by personality preferences. Therefore, it is unlikely that multiple-mini interview scores included in the admission process will affect the personality diversity of candidates admitted into a veterinary class.

Multiple Mini Interview Performance and Career Maturity related to Medicine.

This article was migrated. The article was marked as recommended. Introduction: Multiple Mini Interview (MMI) is used to determine the most suitable candidates for the medical profession. It is also important to ensure students have the career maturity necessary to cope with the developmental tasks associated with medical training. This study explored if students stronger in their career decision of medicine perform better on the MMI. Methods: 119 students (72% response rate) completed a career maturity measure. Student MMI scores were matched to career maturity responses. Results: A significant positive correlation (p <.05) existed between MMI score and specify physician as a career preference (r =.177, N =112, p <.031). There was no significant correlation with the MMI score and crystalize a career preference (r =.046, N =119, p =.31) or implement physician as career choice (r =.039, N =114, p =.34). Mean scores for both crystalize career preference ( M = 18.91) and implement physician as career choice ( M = 17.46) indicate that students performing lower on the MMI continue coping with those tasks as compared to specify (M=20.65). Conclusion: Findings suggest a relation between medical student's career maturity and interview performance as measured using the MMI, with students in the intermediate stages of career development having higher MMI scores.

APPE Evaluations are Positively Associated with MMI, Pre-pharmacy GPA and Pharmacy GPA

Objective. To determine factors associated with advanced pharmacy practice experience (APPE) performance in the pre-pharmacy and Doctor of Pharmacy (PharmD) curriculum and establish whether performance on the multiple mini interview (MMI) independently predicts APPE evaluation scores. Methods. A multi-case MMI has been used in the admissions process since 2008. Students are scored anywhere from 1 to 7 (unsatisfactory to outstanding) on each interview. Traditional factors (GPA, PCAT, etc.) are also used in the admissions determination. Pearson product-moment correlation and ordinary least squares regression were used to explore the relationships between admissions data, pharmacy GPA, and APPE evaluation scores for the graduating classes of 2011-2014. These analyses identified which factors (pharmacy GPA, PCAT, MMI score, age, gender, rurality, resident status, degree, and underrepresented minority status) related to APPE performance. Results. Students (n=432) had a mean APPE score of 4.6; a mean MMI score of 5.5; mean pharmacy GPA, PCAT and age of 3.14, 73.2, 22.6 years, respectively. Pre-pharmacy GPA and pharmacy GPA positively correlated with mean APPE scores. MMI score demonstrated positive correlations with overall APPE score; including subcategories patient care, documentation, drug information/EBM, public health, and communication. MMI scores were positively related to overall APPE scores in the multivariable regression. Variables showing negative associations with APPE scores included a pre-pharmacy GPA of <3.0 (ref= GPA >3.5) and pharmacy school GPA of >3.0 - 3.5 and GPA 2.6 - 3.0 when compared to GPAs >3.5. Conclusion. GPA (pre-pharmacy and pharmacy) and MMI positively correlate with preceptor-rated performances in the APPE year.

Multiple mini interviews: revealing similarities across institutions

BackgroundAcross the globe multiple mini interviews (MMIs) have rapidly replaced the use of panel interviews in the selection of medical students and other health professionals. MMIs typically demonstrate better reliability and validity than panel interviews but there is limited research on whether these different types of interview process measure the same or different constructs. Our research aims to ascertain if MMIs are multidimensional or unidimensional, and whether MMIs conducted at different institutions assess the same or different constructs to each other or to panel interviews.MethodsParticipants were applicants to medical degrees who were shortlisted for interviews at three different institutions in 2013 (n = 165) and 2014 (n = 128). Two institutions used a bespoke MMI developed independently from each other and the third used a panel interview. Stations scores and overall (mean) interview scores were examined.ResultsExploratory principal components analysis and confirmatory factor analysis showed similar results in both years’ data, supporting a unidimensional model. The two overall MMI scores were more strongly correlated to each other (r = .56 and .64 in 2013 and 2014 respectively) than either were to the panel interview scores (r = .07 and .15 in 2013; .39 and .48 in 2014).ConclusionsIt appears that both MMIs panel interviews tap a single latent construct, but not the same construct. We suggest that the MMI methodology might allow the measurement of an emergent construct such as adaptability.

Medical School Applicant Characteristics Associated With Performance in Multiple Mini-Interviews Versus Traditional Interviews: A Multi-Institutional Study.

To examine applicant characteristics associated with multiple mini-interview (MMI) or traditional interview (TI) performance at five California medical schools. Of the five California Longitudinal Evaluation of Admission Practices consortium schools, three used TIs and two used MMIs. Schools provided retrospective data on 2011-2013 admissions cycle interviewees: age, gender, race/ethnicity (underrepresented in medicine [UIM] or not), disadvantaged (DA) status, undergraduate GPA, Medical College Admission Test (MCAT) score, and interview score (standardized as z score; mean = 0; SD = 1). Adjusted linear regression analyses, stratified by interview type, examined associations with interview performance. The 4,993 applicants who completed 7,516 interviews included 931 (18.6%) UIM and 962 (19.3%) DA individuals; 3,226 (64.6%) had only 1 interview. Mean age was 24.4 (SD = 2.7); mean GPA and MCAT score were 3.72 (SD = 0.22) and 33.6 (SD = 3.7), respectively. Older age, female gender, and number of prior interviews were associated with better performance on both MMIs and TIs. Higher GPA was associated with lower MMI scores (z score, per unit GPA = -0.26; 95% CI = -0.45, -0.06) but unrelated to TI scores. DA applicants had higher TI scores (z score = 0.17; 95% CI = 0.07, 0.28) but lower MMI scores (z score = -0.18; 95% CI = -0.28, -0.08) than non-DA applicants. Neither UIM status nor MCAT score was associated with interview performance. These findings have potentially important workforce implications, particularly regarding MMI performance of DA applicants, and illustrate the need for other multi-institutional studies.

Do Multiple Mini-Interview and Traditional Interview Scores Differ in Their Associations With Acceptance Offers Within and Across Five California Medical Schools?

In single-school studies, multiple mini-interview (MMI) and traditional interview (TI) scores are associated with acceptance offers. Unexamined is whether scores at one school are associated with acceptance at other schools; such analyses would mitigate single-school design biases and better estimate how well interviews capture desired applicant attributes. Using data from the 5 California Longitudinal Evaluation of Admissions Practices (CA-LEAP) medical schools, the authors examined associations of MMI and TI scores with acceptance offers within and across schools. The analyses included applicants who interviewed at ≥1 CA-LEAP school during the 2011-2013 admissions cycles. Three CA-LEAP schools employed TIs and 2 employed MMIs. Interview scores were standardized (z scores: mean = 0, SD = 1), and associations with acceptance offers were examined within and across schools in analyses stratified by school, adjusting for applicant sociodemographics, academic metrics, year, and total number of interviews. Of 4,993 applicants interviewed, 428 (8.6%) interviewed at both MMI schools, 681 (13.6%) at ≥2 TI schools, and 1,327 (26.6%) at ≥1 MMI and ≥1 TI school. For each school, acceptance was associated with interview score at that school and also with interview scores at the other 4 schools. Cross-school associations of MMI versus TI scores with acceptance did not differ statistically. Interview score at a given school was associated with acceptance at the other 4 schools, with no significant differences in associations for MMIs versus TIs. The findings suggest both MMIs and TIs captured attributes valued by admissions teams across CA-LEAP schools.

How good are we at selecting students that meet our mission? Outcomes of the 2011 and 2012 entering classes selected by a locally developed multiple mini interview

Background: Can a locally developed multiple mini interview (MMI) process lead to outcomes reflective of local values and mission?Methods: In 2017, the authors performed a retrospective analysis of the relationship of MMI with multiple-choice-based outcomes and non-multiple-choice-based outcomes, including clerkship competencies, OSCE, scholarship/service/leadership, academic honor society induction, peer and faculty humanism nominations, and overall performance at graduation for two entering classes with acceptance decisions based exclusively on a locally developed MMI.Results: There was no association between MMI and performance on multiple-choice-based examinations. For other outcomes, the effect size of MMI for OSCE was small and leadership/service and scholarship did not correlate with MMI score. For clerkship competencies, there was medium effect size for patient care, practice-based learning and improvement, interpersonal and communication skills, and cultural competence. Highest and lowest quartile MMI scorers were no different in academic honor society induction; however, top quartile MMI scorers received more humanism votes versus last quartile and were more likely rated outstanding or excellent graduates.Conclusions: Local development of MMI and of admissions processes with sole reliance on MMI for final acceptance decisions will not affect academic preparation/medical school performance in multiple-choice-based assessments but can lead to locally desired attributes in students.

Medical School Performance of Socioeconomically Disadvantaged and Underrepresented Minority Students Matriculating after a Multiple Mini-Interview.

Multiple Mini-Interviews (MMIs) are increasingly used in medical school admissions. We previously reported that while under-represented minority (URM) status was not associated with MMI scores, self-designated disadvantaged applicants had lower MMI scores, possibly affecting their matriculation prospects. No studies have examined how URM status or socioeconomic disadvantage (SED) are associated with academic performance following admission through an MMI. We examined the adjusted associations of MMI scores, SED, and URM status with U.S. Medical Licensing Examination Steps 1 and 2 performance and third-year clerkship Honors, measures affecting residency matching. While URM status was not associated with the measures, students with greater SED had lower Step 1 scores and fewer Honors. Students with higher MMI scores had more Step 1 failures, but more Honors. The findings identify areas to address in medical school admissions, student support, and evaluation processes, which is important given the need for a more representative physician workforce.

Reliability of Multiple Mini-Interviews and traditional interviews within and between institutions: a study of five California medical schools

BackgroundMany medical schools use admissions Multiple Mini-Interviews (MMIs) rather than traditional interviews (TIs), partly because MMIs are thought to be more reliable. Yet prior studies examined single-school samples of candidates completing either an MMI or TI (not both). Using data from five California public medical schools, the authors examined the within- and between-school reliabilities of TIs and MMIs.MethodsThe analyses included applicants interviewing at ≥1 of the five schools during 2011–2013. Three schools employed TIs (TI1, TI2, TI3) and two employed MMIs (MMI1, MMI2). Mixed linear models accounting for nesting of observations within applicants examined standardized TI and MMI scores (mean = 0, SD = 1), adjusting for applicant socio-demographics, academic metrics, year, number of interviews, and interview date.ResultsA total of 4993 individuals (completing 7516 interviews [TI = 4137, MMI = 3379]) interviewed at ≥1 school; 428 (14.5%) interviewed at both MMI schools and 687 (20.2%) at more than one TI school. Within schools, inter-interviewer consistency was generally qualitatively lower for TI1, TI2, and TI3 (Pearson’s r 0.07, 0.13, and 0.29, and Cronbach’s α, 0.40, 0.44, and 0.61, respectively) than for MMI1 and MMI 2 (Cronbach’s α 0.68 and 0.60, respectively). Between schools, the adjusted intraclass correlation coefficient was 0.27 (95% CI 0.20–0.35) for TIs and 0.47 (95% CI 0.41–0.54) for MMIs.ConclusionsWithin and between-school reliability was qualitatively higher for MMIs than for TIs. Nonetheless, TI reliabilities were higher than anticipated from prior literature, suggesting TIs may not need to be abandoned on reliability grounds if other factors favor their use.

Validation of the 5-item doctor-patient communication competency instrument for medical students (DPCC-MS) using two years of assessment data

BackgroundMedical students on clinical rotations have to be assessed on several competencies at the end of each clinical rotation, pointing to the need for short, reliable, and valid assessment instruments of each competency. Doctor patient communication is a central competency targeted by medical schools however, there are no published short (i.e. less than 10 items), reliable and valid instruments to assess doctor-patient communication competency. The Faculty of Medicine of Laval University recently developed a 5-item Doctor-Patient Communication Competency instrument for Medical Students (DPCC-MS), based on the Patient Centered Clinical Method conceptual framework, which provides a global summative end-of-rotation assessment of doctor-patient communication. We conducted a psychometric validation of this instrument and present validity evidence based on the response process, internal structure and relation to other variables using two years of assessment data.MethodsWe conducted the study in two phases. In phase 1, we drew on 4991 student DPCC-MS assessments (two years). We conducted descriptive statistics, a confirmatory factor analysis (CFA), and tested the correlation between the DPCC-MS and the Multiple Mini Interviews (MMI) scores. In phase 2, eleven clinical teachers assessed the performance of 35 medical students in an objective structured clinical examination station using the DPCC-MS, a 15-item instrument developed by Côté et al. (published in 2001), and a 2-item global assessment. We compared the DPCC-MS to the longer Côté et al. instrument based on internal consistency, coefficient of variation, convergent validity, and inter-rater reliability.ResultsPhase 1: Cronbach’s alpha was acceptable (.75 and .83). Inter-item correlations were positive and the discrimination index was above .30 for all items. CFA supported a unidimensional structure. DPCC-MS and MMI scores were correlated. Phase 2: The DPCC-MS and the Côté et al. instrument had similar internal consistency and convergent validity, but the DPCC-MS had better inter-rater reliability (mean ICC = .61).ConclusionsThe DPCC-MS provides an internally consistent and valid assessment of medical students’ communication with patients.

The reliability and validity of multiple mini interviews (MMIs) in values based recruitment to nursing, midwifery and paramedic practice programmes: Findings from an evaluation study

BackgroundUniversities in the United Kingdom (UK) are required to incorporate values based recruitment (VBR) into their healthcare student selection processes. This reflects an international drive to strengthen the quality of healthcare service provision. This paper presents novel findings in relation to the reliability and predictive validity of multiple mini interviews (MMIs); one approach to VBR widely being employed by universities. ObjectivesTo examine the reliability (internal consistency) and predictive validity of MMIs using end of Year One practice outcomes of under-graduate pre-registration adult, child, mental health nursing, midwifery and paramedic practice students. DesignCross-discipline evaluation study. Setting: One university in the United Kingdom. ParticipantsData were collected in two streams: applicants to A) The September 2014 and 2015 Midwifery Studies programmes; B) September 2015 adult; Child and Mental Health Nursing and Paramedic Practice programmes. Fifty-seven midwifery students commenced their programme in 2014 and 69 in 2015; 47 and 54 agreed to participate and completed Year One respectively. 333 healthcare students commenced their programmes in September 2015. Of these, 281 agreed to participate and completed their first year (180 adult, 33 child and 34 mental health nursing and 34 paramedic practice students). MethodsStream A featured a seven station four-minute model with one interviewer at each station and in Stream B a six station model was employed. Cronbach’s alpha was used to assess MMI station internal consistency and Pearson’s moment correlation co-efficient to explore associations between participants’ admission MMI score and end of Year one clinical practice outcomes (OSCE and mentor grading). ResultsStream A: Significant correlations are reported between midwifery applicant’s MMI scores and end of Year One practice outcomes. A multivariate linear regression model demonstrated that MMI score significantly predicted end of Year One practice outcomes controlling for age and academic entry level: coefficients 0.195 (p=0.002) and 0.116 (p=0.002) for OSCE and mentor grading respectively. In Stream B no significant correlations were found between MMI score and practice outcomes measured by mentor grading.Internal consistency for each MMI station was ‘excellent’ with values ranging from 0.966–0.974 across Streams A and B. ConclusionThis novel, cross-discipline study shows that MMIs are reliable VBR tools which have predictive validity when a seven station model is used. These data are important given the current international use of different MMI models in healthcare student selection processes.

Reliability and validity of the multiple mini interview (MMI) for admissions to an occupational therapy professional program

Introduction Admissions decisions are among the most consequential assessment points in educating occupational therapists. Suboptimal measures of academic achievement and personal characteristics are often used to select candidates. The purpose of the study is to investigate reliability and validity evidence supporting the use of the multiple mini interview for selecting candidates to an occupational therapy master’s level program. Method The study used a prospective cohort design. Data sources included both archival data and prospective candidate-generated data (multiple mini interview scores, applicant file review scores and academic performance), analyzed using psychometric and correlational approaches. Results Seventy-seven of 80 candidates participated in the prospective component of the study. Overall reliability of a 10-station multiple mini interview was .72. Inter-rater reliability was reasonable. Weak interrelationships were found between multiple mini interview scores and other application components. Conclusion Multiple mini interview scores may be probing different attributes than the other components of the application package. Our study findings support the use of the multiple mini interview for selecting applicants to a professional master’s program in occupational therapy; however, future work should continue to investigate the predictive validity of the multiple mini interview.

Have we got the selection process right? The validity of selection tools for predicting academic performance in the first year of undergraduate medicine.

This article was migrated. The article was marked as recommended. Content: There remains much debate over the 'best' method for selecting students in to medicine. This study aimed to assess the predictive validity of four different selection tools with academic performance outcomes in first-year undergraduate medical students. Methods: Regression analyses were conducted between admission scores on previous academic performance - the Australian Tertiary Admission Rank (ATAR), the Undergraduate Medicine and Health Sciences Admission Test (UMAT), Multiple-Mini Interview (MMI) and the Personal Qualities Assessment (PQA) with student performance in first-year assessments of Multiple Choice Questions, Short Answer Questions, Objective Structured Clinical Examinations (OSCE) and Problem-Based Learning (PBL) Tutor ratings in four cohorts of students (N = 604, 90%). Results: All four selection tools were found to have significant predictive associations with one or more measures of student performance in Year One of undergraduate medicine. UMAT, ATAR and MMI scores consistently predicted first year performance on a number of outcomes. ATAR was the only selection tool to predict the likelihood of making satisfactory progress overall. Conclusions: All four selection tools play a contributing role in predicting academic performance in first year medical students. Further research is required to assess the validity of selection tools in predicting performance in the later years of medicine.

Multiple Mini-Interview for Postgraduate Year Residency Recruitment: First Experience in Taiwan

Background: We investigated the effectiveness of the first hospital-held multiple mini-interview for recruitment into a postgraduate residency program in Taiwan. Methods: We conducted a multiple mini-interview to recruit 37 trainees into the postgraduate general medicine residency in January 2017 of the 37 postgraduate year candidates, 28 had interned at our hospital. Between-group correlations among subgroup analyses using multiple comparisons, Chi square test, and the generalized linear model were conducted to test the validity and discrimination of the multiple mini-interview questions. Correlations between multiple mini-interview scores, objective structured clinical examination scores, and clinical practice examinations were conducted and available for 19 applicants who interned at this hospital to test the effectiveness of the multiple mini-interview as a recruiting tool. Results: The multiple mini-interview process exhibited high test discrimination (P<0.001). Whether an intern studied at our hospital or at a different hospital did not affect the performance of the multiple mini-interview. Significant correlations were observed among performance in the multiple mini-interview, all three objective structured clinical examinations, and in four of six clinical practice departments among the 19 candidates who interned at our hospital (P<0.05). Conclusions: The significant correlation of the multiple mini-interviews with clinical performance scores provides powerful support for multiple mini-interview assessment in recruiting residents suitable for clinical work. Continuing to collect data to study the multiple mini-interview program would improve our understanding of it and provide better support for the postgraduate year recruitment program.

Impact of sociodemographic characteristics of applicants in multiple mini-interviews

Background: Multiple mini-interviews (MMI) are commonly used for medical school admission. This study aimed to assess if sociodemographic characteristics are associated with MMI performance, and how they may act as barriers or enablers to communication in MMI.Methods: This mixed-method study combined data from a sociodemographic questionnaire, MMI scores, semi-structured interviews and focus groups with applicants and assessors. Quantitative and qualitative data were analyzed using multiple linear regression and a thematic framework analysis.Results: 1099 applicants responded to the questionnaire. A regression model (R2 = 0.086) demonstrated that being age 25–29 (β = 0.11, p = 0.001), female and a French-speaker (β = 0.22, p = 0.003) were associated with better MMI scores. Having an Asian-born parent was associated with a lower score (β = −0.12, p < 0.001). Candidates reporting a higher family income had higher MMI scores. In the qualitative data, participants discussed how maturity and financial support improved life experiences, how language could act as a barrier, and how ethnocultural differences could lead to misunderstandings.Conclusion: Age, gender, ethnicity, socioeconomic status and language seem to be associated with applicants’ MMI scores because of perceived differences in communications skills and life experiences. Monitoring this association may provide guidance to improve fairness of MMI stations.

Identifying applicants suitable to a career in nursing: a value-based approach to undergraduate selection.

The aim of this study was to complement existing evidence on the suitability of Multiple Mini Interviews as a potential tool for the selection of nursing candidates on to a BSc (Hons) nursing programme. This study aimed to trial the Multiple Mini Interview approach to recruitment with a group of first year nursing students (already selected using traditional interviews). Cross-sectional validation study. This paper reports on the evaluation of the participants' detailed scores from the Multiple Mini Interview stations; their original interview scores and their end of year results. This study took place in March 2015. Scores from the seven Multiple Mini Interview stations were analysed to show the internal structure, reliability and generalizability of the stations. Original selection scores from interviews and in-course assessment were correlated with the MMI scores and variation by students' age, gender and disability status was explored. Reliability of the Multiple Mini Interview score was moderate (G=0·52). The Multiple Mini Interview score provided better differentiation between more able students than did the original interview score but neither score was correlated with the module results. Multiple Mini Interview scores were positively associated with students' age but not their gender or disability status. The Multiple Mini Interview reported in this study offers a selection process that is based on the values and personal attributes regarded as desirable for a career in nursing and does not necessarily predict academic success. Its moderate reliability indicates the need for further improvement but it is capable of discriminating between candidates and shows little evidence of bias.

Can Multiple Mini‐Interviews Predict Academic Performance of Dental Students? A Two‐Year Follow‐Up

Prior research has shown that students' previous grade point average (GPA) is the best predictor for future academic success. However, it can only partly predict the variability in dental school performance. The aim of this study was to assess the predictive value of multiple mini-interviews (MMI) as an admission criterion by comparing them with the academic performance of dental students over a two-year period. All incoming undergraduate dental students at the King Abdulaziz University Faculty of Dentistry (KAUFD) during academic year 2013-14 were invited to participate in MMI. Students rotated through six objective structured clinical exam (OSCE)-like stations for 30 minutes total and were interviewed by two trained faculty interviewers at each station. The stations were focused on noncognitive skills thought to be essential to academic performance at KAUFD. The academic performance of these students was then followed for two years and linked to their MMI scores. A total of 146 students (71 males and 75 females) participated in an interview (response rate=92.9%). Most students scored in the acceptable range at each MMI station. Students' total MMI score, ambitions, and motives were significant predictors of GPA during the two years of follow-up (p<0.038 and p<0.001, respectively). In this study, MMI was found to be able to predict future academic performance of undergraduate dental students.

Hidden item variance in multiple mini-interview scores.

The extant literature has largely ignored a potentially significant source of variance in multiple mini-interview (MMI) scores by "hiding" the variance attributable to the sample of attributes used on an evaluation form. This potential source of hidden variance can be defined as rating items, which typically comprise an MMI evaluation form. Due to its multi-faceted, repeated measures format, reliability for the MMI has been primarily evaluated using generalizability (G) theory. A key assumption of G theory is that G studies model the most important sources of variance to which a researcher plans to generalize. Because G studies can only attribute variance to the facets that are modeled in a G study, failure to model potentially substantial sources of variation in MMI scores can result in biased estimates of variance components. This study demonstrates the implications of hiding the item facet in MMI studies when true item-level effects exist. An extensive Monte Carlo simulation study was conducted to examine whether a commonly used hidden item, person-by-station (p×s|i) G study design results in biased estimated variance components. Estimates from this hidden item model were compared with estimates from a more complete person-by-station-by-item (p×s×i) model. Results suggest that when true item-level effects exist, the hidden item model (p×s|i) will result in biased variance components which can bias reliability estimates; therefore, researchers should consider using the more complete person-by-station-by-item model (p×s×i) when evaluating generalizability of MMI scores.

An Examination of Correlations between MMI scores and Pharmacy School GPA

Objective. To investigate the correlation of mean admission multiple mini-interview (MMI) scores with cumulative and overall GPA across didactic years 1-3 in the doctor of pharmacy (PharmD) curriculum. Methods. The mean admission MMI score and cumulative and overall GPA for first year (P1), second year (P2), and third year (P3) students in the PharmD curriculum was used to conduct a multiple regression analysis and to calculate a Pearson correlation coefficient. Statistical analysis was performed using SPSS, v20. Results. A negative correlation between mean admission MMI and overall GPA was observed for P1 students and P2 students. A significant positive correlation was observed between mean admission MMI and overall GPA for P3 students. Conclusion. A weak association between mean admission MMI score and GPA was observed, and the direction of the association between MMI and GPA was mixed across cohorts of students in the PharmD curriculum. Further research is needed that includes measurement of noncognitive outcomes and continued validation of the MMI for use in pharmacy school admissions.

Mini Surgical Simulation, Role Play, and Group and Behavioral Interviews in Resident Selection.

A robust selection process is critical to residents' "cultural fit" and success in their program. Traditional selection methods have shortcomings. We describe a novel residency interview process for obstetrics-gynecology residents that incorporates behavioral, group, and surgical simulation multiple mini interviews (MMIs). In 2010, the Cedars-Sinai Medical Center obstetrics-gynecology residency program developed surgical simulation, role play, ethics group interview, and Accreditation Council for Graduate Medical Education competency-based behavioral interview stations. From 2010 to 2012, a total of 199 applicants were interviewed, 62 ranked in the top 20, and 18 matched into the program. The MMI scores for interview stations were used in compiling our rank list and were found to adequately differentiate candidates. The MMI mean scores for role play, ethics interview, surgical simulation, and the behavioral interview for the top 20 ranked candidates were statistically significantly higher than those for other applicants. Standardized tests minimally correlated with various interview modalities. Applicants found the interview process acceptable. Implementing these MMI stations increased the total applicant interview time for the day by 15% (from 5.5 to 6.5 hours) and increased the face-to-face interview time from 2 to 4 hours. Approximately 42 hours of coordinator time was required for the yearly interview cycle. A multifaceted interview process utilizing MMI, group interview, and surgical simulation MMI is feasible and acceptable. The approach may decrease subjectivity and reliance on traditional interview methods and facilitate the selection of "compatible" residents into the program.