Validity Of Multiple Mini-interview Research Articles

Exploring the reliability and validity of multiple mini-interviews in admission to teacher education

ABSTRACT This study addressed the need to explore the reliability and validity of the recently renewed aptitude test based on the multiple mini-interview (MMI) format in Finnish teacher education programmes. Utilising a nationwide dataset (N = 3306), we explored the reliability of the format using intraclass correlations and multilevel modelling. Furthermore, we validated the intended structure of the MMI with confirmatory factor analysis and contrasted the scores of the MMI with other admission scores, i.e. cognitive measures. The results showed mostly small effects of clustering of the applicants according to the different interviewers and circuits. The largest variance component for the MMI total score was the applicant (73.3%), followed by measurement error (15.09%). The intended structure was validated, and the MMI seemed to measure different attributes from other admission measures. Thus, the study demonstrated satisfactory reliability and validity of MMI in admission to teacher education, yet the stations need further development.

Reliability and validity of multiple mini interviews in selection to a dual degree midwifery program in Germany: a cross-sectional cohort study

Reliability and validity of multiple mini interviews in selection to a dual degree midwifery program in Germany: a cross-sectional cohort study

Predictive validity of multiple mini interview scores for future academic and clinical placement performance in physiotherapy, occupational therapy and speech and language therapy programmes

Background/aims Limited research exists on the predictive validity of multiple mini interviews when used during the selection process for physiotherapy, occupational therapy and speech and language therapy programmes. Traditional interviews were replaced by multiple mini interviews in the selection process for these three undergraduate programmes in one UK university. The purpose of secondary data analysis was to determine whether multiple mini interviews can predict academic and clinical placement performance during the first and second year of study. Methods Secondary data analysis was performed using the admissions data from 169 students. Data analysed comprised predictors (entry tariff, age and multiple mini interview scores) and outcomes (academic and placement achievement in years 1 and 2 over a period of 2 years from 2014 to 2016). Results Multiple standard regression analyses revealed that, overall, multiple mini interview score was not a significant predictor of academic achievement or clinical placement performance in years 1 and 2 for any of the three professions. Pearson's correlations indicated that age and entry tariff were frequently and significantly correlated with academic achievements for all cohorts. Conclusions Performance in the multiple mini interview is not a predictor of academic achievement or clinical placement performance in this specific cohort of students. Entry tariff showed a significant correlation with academic achievement. Additional studies with larger cohorts are recommended.

The predictive validity of Multiple Mini Interviews (MMIs) in nursing and midwifery programmes: Year three findings from a cross-discipline cohort study

BackgroundEducation literature worldwide is replete with studies evaluating the effectiveness of Multiple Mini Interviews (MMIs) in admissions to medicine but <1% of published studies have been conducted in selection to nursing and midwifery programmes. ObjectivesTo examine the predictive validity of MMIs using end of programme clinical and academic performance indicators of pre-registration adult, child, and mental health nursing and midwifery students. Design and settingA cross-sectional cohort study at one university in the United Kingdom. ParticipantsA non-probability consecutive sampling strategy whereby all applicants to the September 2015 pre-registration adult, child, mental health nursing and midwifery programmes were invited to participate. Of the 354 students who commenced year one, 225 (64%) completed their three-year programme and agreed to take part (adult 120, child 32, mental health nursing 30 and midwifery 43). MethodsAll applicants were interviewed using MMIs with six and seven station, four-minute models deployed in nursing and midwifery student selection respectively. Associations between MMI scores and the cross-discipline programme performance indicators available for each student at this university at the end of year three: clinical practice (assessed by mentors) and academic attainment (dissertation mark) were explored using multiple linear regression adjusting for applicant age, academic entry level, discipline and number of MMI stations. ResultsIn the adjusted models, students with higher admissions MMI score (at six and seven stations) performed better in clinical practice (p < 0.001) but not in academic attainment (p = 0.122) at the end of their three-year programme. ConclusionThese findings provide the first report of the predictive validity of MMIs for performance in clinical practice using six and seven station models in nursing and midwifery programmes. Further evidence is required from both clinical and academic perspectives from larger, multi-site evaluations.

Construct validity of multiple mini interviews – Investigating the role of stations, skills, and raters using Bayesian G-theory

Background: One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that do not depend on specific social skills. Here, we provide a detailed investigation into the construct validity of MMIs, including the identification and quantification of performance facets (social skill-specific performance, station-specific performance, general performance) and their relations with other selection measures. Methods: We used data from three MMI samples (N = 376 applicants, 144 raters) that included six interview and role-play stations and multiple assessed social skills. Results: Bayesian generalizability analyses show that, the largest amount of reliable MMI variance was accounted for by station-specific and general performance differences between applicants. Furthermore, there were low or no correlations with other selection measures. Discussion: Our findings suggest that MMI ratings are less social skill-specific than originally conceptualized and are due more to general performance differences (across and within-stations). Future research should focus on the development of skill-specific MMI stations and on behavioral analyses on the extents to which performance differences are based on desirable skills versus undesired aspects.

The reliability and validity of multiple mini interviews (MMIs) in values based recruitment to nursing, midwifery and paramedic practice programmes: Findings from an evaluation study

BackgroundUniversities in the United Kingdom (UK) are required to incorporate values based recruitment (VBR) into their healthcare student selection processes. This reflects an international drive to strengthen the quality of healthcare service provision. This paper presents novel findings in relation to the reliability and predictive validity of multiple mini interviews (MMIs); one approach to VBR widely being employed by universities. ObjectivesTo examine the reliability (internal consistency) and predictive validity of MMIs using end of Year One practice outcomes of under-graduate pre-registration adult, child, mental health nursing, midwifery and paramedic practice students. DesignCross-discipline evaluation study. Setting: One university in the United Kingdom. ParticipantsData were collected in two streams: applicants to A) The September 2014 and 2015 Midwifery Studies programmes; B) September 2015 adult; Child and Mental Health Nursing and Paramedic Practice programmes. Fifty-seven midwifery students commenced their programme in 2014 and 69 in 2015; 47 and 54 agreed to participate and completed Year One respectively. 333 healthcare students commenced their programmes in September 2015. Of these, 281 agreed to participate and completed their first year (180 adult, 33 child and 34 mental health nursing and 34 paramedic practice students). MethodsStream A featured a seven station four-minute model with one interviewer at each station and in Stream B a six station model was employed. Cronbach’s alpha was used to assess MMI station internal consistency and Pearson’s moment correlation co-efficient to explore associations between participants’ admission MMI score and end of Year one clinical practice outcomes (OSCE and mentor grading). ResultsStream A: Significant correlations are reported between midwifery applicant’s MMI scores and end of Year One practice outcomes. A multivariate linear regression model demonstrated that MMI score significantly predicted end of Year One practice outcomes controlling for age and academic entry level: coefficients 0.195 (p=0.002) and 0.116 (p=0.002) for OSCE and mentor grading respectively. In Stream B no significant correlations were found between MMI score and practice outcomes measured by mentor grading.Internal consistency for each MMI station was ‘excellent’ with values ranging from 0.966–0.974 across Streams A and B. ConclusionThis novel, cross-discipline study shows that MMIs are reliable VBR tools which have predictive validity when a seven station model is used. These data are important given the current international use of different MMI models in healthcare student selection processes.

A comparison of multiple mini-interviews and structured interviews in a UK setting

Introduction: The multiple mini-interview (MMI) has been shown to be a valuable tool to aid the selection of medical students. The increasing body of research data so far has not evaluated this approach in the UK, where selection for medical training has traditionally included a single unstructured or structured interview.Methods: MMI stations were developed and tested on volunteer candidates for admission to the MBBS 4 and MBBS 5 courses. Volunteers undertook their admissions interview before taking part in the MMI trial. Scores were compared between the two interview formats and any relationship with demographic details and the aptitude tests (UK Clinical Aptitude Test and Graduate Medical School Admissions Test) established.Results: MBBS 4 applicants performed just as well on the MMI as they did on the traditional interview, with the MBBS 5 applicants performing better on the MMI. MBBS 4 and MBBS 5 candidates did equally well on the MMI. There was no difference in performance related to sex or age.Discussion: MMIs are reliable, feasible and acceptable to both applicants and interviewers. Longitudinal research will shed more light on the validity of MMI as a way of measuring applicants’ potential to become professional, successful doctors.