Multiple Mini-interview Research Articles

Medical School Admission Interview and Licensing Examination Scores

The study by Dr Eva and colleagues1 highlighted an important issue in evaluating applicants for admission to medical school, but I believe it was inherently flawed. The authors reported that achievement on the multiple mini-interview (MMI) enabled successful selection of applicants as measured by scores on the national licensing examination. Obtaining high scores on national licensing examinations, however, is an inappropriate outcome metric to demonstrate the success of admissions committees in selecting superior candidates

Assessing the Multiple Mini Interview (MMI) for Use in Selecting Applicants to a Neurology Residency Program (P04.253)

Assessing the Multiple Mini Interview (MMI) for Use in Selecting Applicants to a Neurology Residency Program (P04.253)

Improving Student Selection Using Multiple Mini-Interviews With Multifaceted Rasch Modeling

The authors report multiple mini-interview (MMI) selection process data at the University of Dundee Medical School; staff, students, and simulated patients were examiners and investigated how effective this process was in separating candidates for entry into medical school according to the attributes measured, whether the different groups of examiners exhibited systematic differences in their rating patterns, and what effect such differences might have on candidates' scores. The 452 candidates assessed in 2009 rotated through the same 10-station MMI that measured six noncognitive attributes. Each candidate was rated by one examiner in each station. Scores were analyzed using Facets software, with candidates, examiners, and stations as facets. The computer program calculated fair average scores that adjusted for examiner severity/leniency and station difficulty. The MMI reliably (0.89) separated the candidates into four statistically distinct levels of noncognitive ability. The Rasch measures accounted for 31.69% of the total variance in the ratings (candidates 16.01%, examiners 11.32%, and stations 4.36%). Students rated more severely than staff and also had more unexpected ratings. Adjusting scores for examiner severity/leniency and station difficulty would have changed the selection outcomes for 9.6% of the candidates. The analyses highlighted the fact that quality control monitoring is essential to ensure fairness when ranking candidates according to scores obtained in the MMI. The results can be used to identify examiners needing further training, or who should not be included again, as well as stations needing review. "Fair average" scores should be used for ranking the candidates.

The Possible Impact of the MMI and GPA on Diversity

In Reply to Norman et al: Norman et al (two of whom helped to develop the Multiple Mini-Interview [MMI]) make several comments we wish to address. First, they claim that our findings about the association between extroversion and performance on the MMI suggest that the MMI “selects for individuals who are … interested in others, and skilled in the interpersonal domain.” Yet, extroversion is not uniquely associated with interest in people or demonstrated skill at communication.1 High extroverts do, however, tend to receive more favorable subjective communication ratings in brief exposure forums that may disadvantage the less extroverted (e.g., clerkship rounds).2 Even so, whether high extroverts are superior communicators in clinical practice is unstudied. Second, they note that grade point average (GPA) is associated with conscientiousness and imply that we might also be concerned that using GPA in admissions screening may preferentially select highly conscientious students. But we do not view this as a concern, since conscientiousness (unlike extroversion) is strongly predictive of superior medical school academic and clinical performance.2–5 Third, are they implying that the association of MMI score with extroversion that we observed is statistical only and lacks practical significance? If so, we disagree. In our study, applicants in the top extroversion quartile scored almost 6 points higher on the MMI (total possible score = 30) than did the bottom-quartile participants—a sizeable difference. Furthermore, for our prediction model for base medical-school-acceptance offers, which excluded personality, the receiver operating characteristic curve C statistic was 62.6%. After adding personality variables to the model, the C statistic increased substantively to 70.6%. Finally, after adding MMI score to the model, extroversion was no longer associated with acceptance offers, indicating that the MMI “captured” high extroversion effects on acceptance offers. Full model details are provided in our report. Fourth, for unclear reasons, they cite a book recounting the abhorrant practice of using personal interviews to discriminate against Jewish medical school applicants in the early 1900s. The book clearly does not address the utility of traditional interviews when appropriately employed. Finally, they claim that the personal interview “has been repeatedly demonstrated to have no relation to important educational outcomes.” However, one of the two reviews they cite in support (by Albanese et al) concludes that “interview ratings are predictive of subjective clinical assessments, and low interview assessments are predictive of failure or withdrawal from medical school.” The other review, by Salvatori, concludes that “controversy remains regarding the reliability, predictive validity and cost effectiveness of the selection interview. The evidence is clear, however, that the use of a structured format as well as trained and experienced interviewers serves to enhance its psychometric properties.” Anthony Jerant, MD Professor, Department of Family and Community Medicine, University of California Davis School of Medicine, Sacramento, California; [email protected] Joshua Fenton, MD, MPH Associate professor, Department of Family and Community Medicine, University of California Davis School of Medicine, Sacramento, California. Peter Franks, MD Professor, Department of Family and Community Medicine, University of California Davis School of Medicine, Sacramento, California.

Reliability and acceptability of a five-station multiple mini-interview model for residency program recruitment

BackgroundStandard interviews are used by most residency programs in the United States for assessment of aptitude of the non-cognitive competencies, but variability of interviewer skill, interviewer bias, interviewer leniency or stringency, and context specificity limit reliability.AimTo investigate reliability and acceptability of five-station multiple mini-interview (MMI) model for resident selection into an internal medicine residency program in the United States.SettingOne independent academic medical center.ParticipantsTwo hundred and thirty-seven applicants and 17 faculty interviewers.Program descriptionFive, 10-min MMI stations with five different interviewers blinded to the candidate's records and one traditional 20-min interview with the program director. Candidates were rated on two items: interpersonal and communication skills, and overall performance.Program evaluationGeneralizability data showed that the reliability of our process was high (>0.9). The results of anonymous surveys demonstrated that both applicants and interviewers consider the MMI as a fair and more effective tool to evaluate non-cognitive traits, and prefer the MMI to standard interviews.DiscussionThe MMI process for residency interviews can generate reliable interview results using only five stations, and it is acceptable and preferred over standard interview modalities by the applicants and faculty members of one US residency program.

The admissions process in a graduate-entry dental school: can we predict academic performance?

To assess the association between the admissions performance and subsequent academic achievement within a graduate-entry dental school. The study was conducted at the University of Aberdeen Dental School. UCAS forms for course applicants were reviewed and assigned a pre-admission score (PAS) and a tariff given for the UCAS personal statement (UCAS). Individuals ranked highest were invited to attend multiple mini-interviews (MMI), which were scored. Data was correlated with academic performance reported as the University Common Assessment Scale (0-20). Comparisons were also made between the first degree and subsequent educational achievement. Data were analysed by multiple linear regression, Pearson correlation and unstacked ANOVA (IBM SPSS Statistics 19). Data were obtained for 75 students (F: 50; M: 25). A correlation between performance at MMI and CAS scores was identified (r = 0.180, p = 0.001, df = 538). A correlation was also noted between each student's first degree and the CAS scores (F = 4.08, p = 0.001, df = 9). This study suggests that candidate performance at MMI might be a stronger predictor of academic and clinical performance of graduate-entry dental students compared to other pre-interview selection criteria. The first degree for such a programme also appears to be significant.

The selection process in the Faculty of Medicine: the usefulness of a psychological and aptitude assessment

In almost all the universities worldwide, the admission to the Faculty of Medicine is subject to a specific selection process. In most cases tests are used to assess the cognitive abilities of the candidates. However, a growing number of universities (particularly American and Canadian) is using a set of instruments to evaluate non-cognitive skills in addition to these tests. Among the most commonly used tools are included personal interviews, Multiple Mini Interview (MMI), Personal Quality Assessment (PQA). In Italy in the last decade increasing emphasis was placed on assessing students' attitudes through the use of orientation tests (given in classrooms or online fillable), as part of several projects that involve collaboration between secondary school and university. Nevertheless, the admission to the Faculty of Medicine does not yet provide a psychological and aptitude assessment, but relies exclusively on cognitive features. In a similar way to what is already the case with the selection of personnel in law enforcement agency, we propose a selection model that includes an assessment of psychological features and aptitudes of the candidates, as well as cognitive skills.

Transforming Admissions: The Gateway to Medicine

IN THIS ISSUE OF JAMA, A STUDY BY EVA AND COLleagues describes the multiple mini-interview (MMI) process pioneered at McMaster University and its association with students’ national licensing examination scores. The MMI is based on sequential structured interviews, with a series of tasks analogous to the objective structured clinical examination. The authors found that students who were accepted by McMaster based on their MMI results and other application data scored higher on Canadian licensing examinations than students who were not accepted by McMaster but attended other medical schools. The MMI appears to be an effective technique for probing dimensions ranging from applicants’ responses to novel situations to their reactions to an ethical conflict. The adoption of the MMI by more schools also may increase the diversity of medical student personal attributes, which may lead to increased opportunity for student professional growth, increased specialty distribution, and improved patient care. The MMI is one important, relatively new tool that medical schools have developed as part of an admissions process known as holistic review, a “flexible, highly individualized process by which balanced consideration is given to the multiple ways in which applicants may prepare for and succeed as medical students and doctors.” The goal of holistic review is 3-fold: to assess applicants’ academic readiness; to gauge their intrapersonal and interpersonal competencies, such as integrity and compassion; and to promote diversity in the student body. Ultimately, the hope is that medical schools’ use of holistic review and a competency-based admissions process will better enable identification of students who have the greatest potential to be the physicians of the future. Traditional admissions processes have placed considerable weight on measures of academic readiness, such as Medical College Admission Test (MCAT) examination scores and grade point averages, when deciding which applicants to interview. Although MCAT scores are a reliable predictor of academic success in the first 2 years of medical school, they are not strongly correlated with later outcomes in medical school or practice. Nevertheless, MCAT scores appeal to admissions committees because they allow for easy comparison of applicants from different undergraduate schools. Although academic credentials have an important place in medical school admissions, there is increasing recognition that overreliance on measures of academic readiness may eliminate some applicants who could become excellent future physicians. As a result, medical schools are moving toward a broader view of medical school readiness that emphasizes the competencies applicants have demonstrated in addition to their academic credentials. This change is essential to identify future physicians with the skills and knowledge to manage illness in the 21st century. For example, in the last 40 years, the largest contributors to increased life expectancy in the United States are reduced tobacco consumption and increased levels of exercise, both of which are behaviors linked to disease prevention. To prevent and treat disease in the future, physicians must have cognitive and behavioral competencies. The new MCAT examination, beginning in 2015, will shift toward a broader definition of academic readiness. The new MCAT will achieve this by including a new section on behavioral and social science content and by revising the current verbal section to test the way examinees reason through text excerpted from books, journals, and magazines across a variety of disciplines, including ethics and philosophy, cross-cultural studies, population health, and other subjects. Although robust measures of academic readiness will remain important, admissions interviews continue to offer a key opportunity for schools to gauge the second dimension of holistic review: interpersonal and intrapersonal competencies, ie, the personal characteristics required of physicians to practice patient-centered care. In assessing competencies, such as oral communication or resilience and adaptability, a variety of approaches are emerging, including assessment centers to evaluate medical school candidates’ personal and interpersonal attributes, situationbased judgment tests, and hybrids of these interview approaches. The movement toward competency-based ad-

Association Between a Medical School Admission Process Using the Multiple Mini-interview and National Licensing Examination Scores

There has been difficulty designing medical school admissions processes that provide valid measurement of candidates' nonacademic qualities. To determine whether students deemed acceptable through a revised admissions protocol using a 12-station multiple mini-interview (MMI) outperform others on the 2 parts of the Canadian national licensing examinations (Medical Council of Canada Qualifying Examination [MCCQE]). The MMI process requires candidates to rotate through brief sequential interviews with structured tasks and independent assessment within each interview. Cohort study comparing potential medical students who were interviewed at McMaster University using an MMI in 2004 or 2005 and accepted (whether or not they matriculated at McMaster) with those who were interviewed and rejected but gained entry elsewhere. The computer-based MCCQE part I (aimed at assessing medical knowledge and clinical decision making) can be taken on graduation from medical school; MCCQE part II (involving simulated patient interactions testing various aspects of practice) is based on the objective structured clinical examination and typically completed 16 months into postgraduate training. Interviews were granted to 1071 candidates, and those who gained entry could feasibly complete both parts of their licensure examination between May 2007 and March 2011. Scores could be matched on the examinations for 751 (part I) and 623 (part II) interviewees. Admissions decisions were made by combining z score transformations of scores assigned to autobiographical essays, grade point average, and MMI performance. Academic and nonacademic measures contributed equally to the final ranking. Scores on MCCQE part I (standardized cut-score, 390 [SD, 100]) and part II (standardized mean, 500 [SD, 100]). Candidates accepted by the admissions process had higher scores than those who were rejected for part I (mean total score, 531 [95% CI, 524-537] vs 515 [95% CI, 507-522]; P = .003) and for part II (mean total score, 563 [95% CI, 556-570] vs 544 [95% CI, 534-554]; P = .007). Among the accepted group, those who matriculated at McMaster did not outperform those who matriculated elsewhere for part I (mean total score, 524 [95% CI, 515-533] vs 546 [95% CI, 535-557]; P = .004) and for part II (mean total score, 557 [95% CI, 548-566] vs 582 [95% CI, 569-594]; P = .003). Compared with students who were rejected by an admission process that used MMI assessment, students who were accepted scored higher on Canadian national licensing examinations.

421 Comparison of the Multiple-Mini Interview With Traditional Interviewing Techniques For Emergency Medicine Residency Applicants: A Single-Institution Experience

421 Comparison of the Multiple-Mini Interview With Traditional Interviewing Techniques For Emergency Medicine Residency Applicants: A Single-Institution Experience

Does Applicant Personality Influence Multiple Mini-Interview Performance and Medical School Acceptance Offers?

To examine relationships among applicant personality, Multiple Mini-Interview (MMI) performance, and medical school acceptance offers. The authors conducted an observational study of applicants who participated in the MMI at the University of California, Davis, School of Medicine during the 2010-2011 admissions cycle and responded to the Big Five Inventory measuring their personality factors (agreeableness, conscientiousness, extraversion, neuroticism, openness). Individuals' MMI performance at 10 stations was summarized as a total score. Regression analyses examined associations of personality factors with MMI score, and associations of personality factors and MMI score with acceptance offers. Covariates included sociodemographic and academic performance measures. Among the 444 respondents, those with extraversion scores in the top (versus bottom) quartile had significantly higher MMI scores (adjusted parameter estimate = 5.93 higher, 95% CI: 4.27-7.59; P < .01). In a model excluding MMI score, top (versus bottom) quartile agreeableness (AOR = 3.22; 95% CI 1.57-6.58; P < .01) and extraversion (AOR = 3.61; 95% CI 1.91-6.82; P < .01) were associated with acceptance offers. After adding MMI score to the model, high agreeableness (AOR = 4.77; 95% CI 1.95-11.65; P < .01) and MMI score (AOR 1.33; 95% CI 1.26-1.42; P < .01) were associated with acceptance offers. Extraversion was associated with MMI performance, whereas both extraversion and agreeableness were associated with acceptance offers. Adoption of the MMI may affect diversity in medical student personalities, with potential implications for students' professional growth, specialty distribution, and patient care.

Multiple Mini Interview for Selection into OTL Residency

Objective: 1) To determine the reliability of the Multiple Mini Interview (MMI) for resident selection into an otolaryngology–head and neck surgery (OTL-HNS) program. 2) To assess the degree of acceptance by major stakeholders (interviewers and applicants) towards the MMI when compared to traditional interviews. Method: Canadian medical graduates applying to OTL-HNS residency programs underwent MMI in 2011 and 2012. MMI had 7 stations evaluating unique candidate attributes. Stations include surgical skills assessment and 2 simulation scenarios with standardized actors for noncognitive traits. Reliability was determined, and upon completion, stakeholders rated aspects MMI using 7-point Likert scale. Results: Data were collected from a total of 45 applicants and 19 evaluators. Overall interrater reliability of the MMI was good. The majority of applicants (&gt;80%) felt that MMI helped them present their strengths and that it did not have any gender, cultural, or age bias. Assessors (&gt;85%) felt that the MMI evaluated a valid range of competencies and that it tested more aspects of an applicant than traditional interviews. Both applicants and assessors (&gt;70%) agreed that the MMI was a fair process, and both preferred the MMI over the traditional interview. Conclusion: The MMI is a reliable tool for the selection of applicants to an OTL-HNS residency program. It is well accepted by both applicants and assessors, with the majority of stakeholders preferring the MMI over traditional interviews.

Should Efforts in Favor of Medical Student Diversity Be Focused During Admissions or Farther Upstream?

Traditional medical school admissions assessment tools may be limiting diversity. This study investigates whether the Multiple Mini-Interview (MMI) is diversity-neutral and, if so, whether applying it with greater weight would dilute the anticipated negative impact of diversity-limiting admissions measures. Interviewed applicants to six medical schools in 2008 and 2009 underwent MMI. Predictor variables of MMI scores, grade point average (GPA), and Medical College Admission Test (MCAT) scores were correlated with diversity measures of age, gender, size of community of origin, income level, and self-declared aboriginal status. A subset of the data was then combined with variable weight assigned to predictor variables to determine whether weighting during the applicant selection process would affect diversity among chosen applicants. MMI scores were unrelated to gender, size of community of origin, and income level. They correlated positively with age and negatively with aboriginal status. GPA and MCAT correlated negatively with age and aboriginal status, GPA correlated positively with income level, and MCAT correlated positively with size of community of origin. Even extreme combinations of MMI and GPA weightings failed to increase diversity among applicants who would be selected on the basis of weighted criteria. MMI could not neutralize the diversity-limiting properties of academic scores as selection criteria to interview. Using academic scores in this way causes range restriction, counteracting attempts to enhance diversity using downstream admissions selection measures such as MMI. Diversity efforts should instead be focused upstream. These results lend further support for the development of pipeline programs.

An evaluation of the multiple mini-interview as a selection tool for dental students

This research evaluated the mutliple mini-interview (MMI) process as part of the admissions procedure for a dental school. The thematic analysis of a paper-based questionnaire. Following screening an MMI was arranged for 190 candidates applying to study dentistry with written feedback obtained from candidates and interviewers. A 100% response rate to the paper-based questionnaire was achieved for candidates and a 90% response rate was obtained from interviewers. With specific reference to how candidates perceived their performance, 127 (67.9%) felt they had performed well with 24 (19.3%) unsure and 36 (12.8%) saying their performance was not ideal. Candidates generally thought that they had prepared as well as they could have done for the MMIs and 146 (77.2%) thought that they had done enough at interview to merit being offered a place to study dentistry. The MMI experience was considered favourable and from 137 written comments received by candidates 39 were most positive with reference to the MMIs. A thematic analysis of the comments identified a number of emergent themes including lack of control, anxiety and nervousness, preparedness and comparisons with conventional interviews. The MMI appears to be useful in the selection of dental students.

The multiple mini-interview in the UK context: 3 years of experience at Dundee

Background: The Multiple Mini-Interview (MMI) is a new interview process that Dundee Medical School has recently adopted to assess entrants into its undergraduate medicine course. This involves an ‘Objective Structured Clinical Examination’ like rotational approach in which candidates are assessed on specific attributes at a number of stations.Aims: To present methodological, questionnaire and psychometric data on the transitional process from traditional interviews to MMIs over a 3-year period and discuss the implications for those considering making this transition.Methods: To facilitate the transition, a four-station MMI was piloted in 2007. Success encouraged consideration of desirable attributes which were used to develop a full 10-station process which was implemented in 2009 with assessors being recruited from staff, students and simulated patients. A questionnaire was administered to all assessors and candidates who participated in the 2009 MMIs. Cronbach's alpha and Pearson's r and analysis of variances were used to determine the MMI's psychometric properties. Multi-faceted Rasch modelling (MFRM) was modelled to control for assessor leniency/stringency and the impact of using ‘fair scores’ determined. Analysis was conducted using SPSS 17 and FACETS 3.65.0.Results: The questionnaire confirmed that the process was acceptable to all parties. Cronbach's alpha reliability was satisfactory and consistent. Graduates/mature candidates outperformed UK school-leavers and overseas candidates. Using MFRM fair scores would change the selection outcome of 6.2% and 9.6% of candidates in 2009 and 2010, respectively. Students were less lenient, made more use of the full range of the rating scales and were just as reliable as staff.Conclusions: The strategy of generating institutional support through staged introduction proved effective. The MMI in Dundee was shown to be feasible and displayed sound psychometric properties. Student assessors appeared to perform at least as well as staff. Despite a considerable intellectual and logistical challenge MMIs were successfully introduced and deemed worthwhile.

Development and Pilot Testing of a Multiple Mini-Interview for Admission to a Pharmacy Degree Program

To develop and pilot test a multiple mini-interview (MMI) to select students for admission to a pharmacy degree program. A nominal group process was used to identify 8 important nonacademic attributes of pharmacists, with relative importance determined by means of a paired-comparison survey of pharmacy stakeholders (ie, university-affiliated individuals with a vested interest in the quality of student admitted to the pharmacy program, such as faculty members, students, admissions staff members, and practitioners). A 10-station MMI based on the weighted-attribute blueprint was pilot tested with 30 incoming pharmacy students. MMI score reliability (intraclass correlation coefficient [ICC]) and correlation with other admissions tool scores were determined. Station scores provided by student interviewers were slightly higher than those of faculty member or practitioner interviewers. While most interviewers judged a 6-minute interview as "just right" and an 8-minute interview "a bit long," candidates had the opposite opinion. Station scenarios had face validity for candidates and interviewers. The ICC for the MMI was 0.77 and correlations with prepharmacy average (PPA) and Pharmacy College Admission Test (PCAT) composite were negligible. MMI feasibility was confirmed, based on the finding that interview scores were reliable and that this admissions tool measures different attributes than do the PCAT and PPA.

A Generalizability Analysis of a Veterinary School Multiple Mini Interview: Effect of Number of Interviewers, Type of Interviewers, and Number of Stations

Background: The number of Multiple Mini Interview (MMI) stations and the type and number of interviewers required for an acceptable level of reliability for veterinary admissions requires investigation. Purpose: The goal is to investigate the reliability of the 2009 MMI admission process at the University of Calgary. Methods: Each applicant (n = 103; female = 80.6%; M age = 23.05 years, SD = 3.96) participated in a 7-station MMI. Applicants were rated independently by 2 interviewers, a faculty member, and a community veterinarian, within each station (total interviewers/applicant N = 14). Interviewers scored applicants on 3 items, each on a 5-point anchored scale. Results: Generalizability analysis resulted in a reliability coefficient of G = 0.79. A Decision study (D-study) indicated that 10 stations with 1 interviewer would produce a G = 0.79 and 8 stations with 2 interviewers would produce a G = 0.81; however, these have different resource requirements. A two-way analysis of variance showed that there was a nonsignificant main effect of interviewer type (between faculty member and community veterinarian) on interview scores, F(1, 1428) = 3.18, p = .075; a significant main effect of station on interview scores, F(6, 1428) = 4.34, p < .001; and a nonsignificant interaction effect between interviewer-type and station on interview scores, F(6, 1428) = 0.74, p = .62. Conclusions: Overall reliability was adequate for the MMI. Results from the D-study suggest that the current format with 7 stations provides adequate reliability given that there are enough interviewers; to achieve the same G-coefficient 1 interviewer per station with 10 stations would suffice and reduce the resource requirements. Community veterinarians and faculty members demonstrated an adequate level of agreement in their assessments of applicants.

Enhancing the Reliability of the Multiple Mini-Interview for Selecting Prospective Health Care Leaders

The David Geffen School of Medicine at UCLA Program in Medical Education (UCLA-PRIME) used a 12-station multiple mini-interview (MMI) circuit to assess applicants. The authors sought to determine the reliability of the MMI, potential bias in scores, and the degree of acceptance by interviewers and applicants. In 2009, 28 interviewers interviewed a cohort of 76 applicants. An anonymous survey assessed interviewers' and applicants' satisfaction with the MMI process and perceived bias. Psychometric properties were determined with generalizability and decision theory. The process was repeated the following year with a new cohort of 78 applicants and minor modifications aimed at improving reliability. The MMI format was well received by both applicants and interviewers. No bias based on gender or disadvantaged status was found. The preliminary reliability of the MMI in 2009 was 0.58-lower than reported in previous studies-but improved in 2010 to 0.71 after an easy station was replaced with a more challenging one and a new scoring rubric was introduced. This interview technique proved to be reliable and was seen as transparent, uniform, and fair. The predictive validity of this process remains to be determined.

Associations between the big five personality factors and multiple mini-interviews

Given the accumulating evidence that performance in medical school and beyond is related to personality, it is important for research to consider how personality assessment can be included as part of the process of selecting medical students. Interviews are one way of measuring personality and this study extends prior research investigating whether the multiple mini interview (MMI) is related to the five factor model of personality. In contrast to prior results (Kulasegaram et al. in Adv Health Sci Edu 15:415-423, 2010), examination of MMI scores for 868 applicants to an Australian medical school over 3 years showed significant uncorrected correlations every year with extraversion (.19, .19, .30) and conscientiousness (.20, .22, .25) and with agreeableness in 2 years (.17, .19). Investigation of personality at a facet-level revealed differing relationships with the MMI within the five factors of personality. MMI scores were also correlated in 2 years (.17, .22) with a situational judgment test of interpersonal understanding (UMAT Section 2) but were unrelated to tests of logical reasoning ability (UMAT Section 1), non-verbal reasoning (Section 3), or past academic performance (Higher School Certificate results).

Resident Selection for a Physical Medicine and Rehabilitation Program

The development of a process to select the best residents for training programs is challenging. There is a paucity of literature to support the implementation of an evidence-based approach or even best practice for program directors and selection committees. Although assessment of traditional academic markers such as clerkship grades and licensing examination scores can be helpful, these measures typically fail to capture performance in the noncognitive domains of medicine. In the specialty of physical medicine and rehabilitation, physician competencies such as communication, health advocacy, and managerial and collaborative skills are of particular importance, but these are often difficult to evaluate in admission interviews. Recent research on admission processes for medical schools has demonstrated reliability and validity of the "multiple mini-interview." The objective of our project was to develop and evaluate the multiple mini-interview for a physical medicine and rehabilitation residency training program, with a focus on assessment of the noncognitive physician competencies. We found that the process was feasible, time efficient, and cost-efficient and that there was good interrater reliability. The multiple mini-interview may be applied to other physical medicine and rehabilitation residency programs. Further research is needed to confirm reliability and determine validity.