OSCE Examination Research Articles

Effect of the COVID-19 Pandemic on Medical Student Performance and Evaluation Scores.

In March 2020, the University of Hawaii John A. Burns School of Medicine suspended in person clinical teaching due to the SARS-CoV-2 (COVID) pandemic. During this period, virtual cases, telehealth participation, and online cases were incorporated into medical education. We have examined the effects of educational outcomes of third and fourth year students throughout clerkship performance, national standardized test scores, and our local fourth year OSCE examination. We found that USMLE step 2 scores were higher in the COVID-affected group. Patient logs in the COVID-affected group were lower for internal medicine, family medicine, OBGYN, and psychiatry clerkships. Clerkship performance grades in the COVID-affected group were lower for OBGYN and higher for surgery and psychiatry, but not different in other clerkships. USNBME subject specific examination scores in the COVID-affected group were higher for internal medicine, surgery, family medicine and psychiatry, but not different in all other specialties. For the fourth year OSCE, students in the COVID-affected group performed better on note taking and worse on physical examination. Future investigations will be needed to explore how our COVID-affected medical students perform in residency and beyond.

Assessing the utility of virtual OSCE sessions as an educational tool: a national pilot study

BackgroundObjective Structured Clinical Examinations (OSCEs) are a common form of assessment used across medical schools in the UK to assess clinical competence and practical skills and are traditionally held in an in-person format. In the past, medical students have often prepared for such exams through in-person peer-assisted learning (PAL), however, due to the recent Covid-19 pandemic, many in-person teaching sessions transitioned to online-based formats. There is currently a paucity of research on the utility of virtual PAL OSCE sessions and thus, we carried out a national pilot study to determine the feasibility of virtual OSCE teaching via feedback from participants and examiners.MethodsA total of 85 students from 19 UK-based medical schools with eight students based internationally attended the series of online OSCE workshops delivered via Zoom®. All students and examiners completed a feedback questionnaire at the end of each session regarding parameters, which included questions on pre-and post-workshop confidence in three OSCE domains: history-taking, communication and data interpretation. A Likert scale using 5 Likert items was used to self-report confidence, and the results were analysed using the Mann-Whitney U test after assessing for normality using the Shapiro-Wilk test.ResultsResults from student feedback showed an increase in confidence for all three OSCE domains after each event (p < 0.001) with 69.4% agreeing or strongly agreeing that online OSCE sessions could sufficiently prepare them for in-person exams. Questionnaire feedback revealed that 97.6% of students and 86.7% of examiners agreed that virtual OSCE teaching would be useful for preparing for in-person OSCE examinations after the pandemic.ConclusionMost participants in the virtual OSCE sessions reported an improvement in their confidence in history-taking, communication and data interpretation skills. Of the participants and examiners that had also experienced in-person OSCE examinations, the majority also reported that they found virtual OSCE sessions to be as engaging and as interactive as in-person teaching. This study has demonstrated that virtual OSCE workshops are a feasible option with the potential to be beneficial beyond the pandemic. However, more studies are required to assess the overall impact on student learning and to determine the value of virtual OSCE workshops on exam performance.

OSCE: An Effective Tool of Assessment for Medical Students

Background: To assess the competency of a student different tools are used. Since its introduction in 1975 by Dr. Harden and his team, OSCE has gained tremendous strides to assess the clinical competencies. Since 1975 onward OSCE has been very successfully used to assess the clinical competencies of medical student globally. OSCE is an assessment tool in which student is observed for performance of different tasks at specified stations. In the current study perception of medical students about OSCE examination was done which shall give room for positive criticism and further improvement of the system where ever required. Objective: To expedite view of final year MBBS students of Azra Naheed College about OSCE Material and Method Study design: Quantitative, cross sectional study. Settings: Azra Naheed College, Lahore. Duration: Six months i.e. 1st July2020 to 31st December 2020 Data Collection procedure: After an informed consent and appropriate briefing, the questionnaire was distributed among the final year medical students of Azra Naheed Medical College. Questionnaire developed by Russell et al was used. Results: Out of 148 students who participated in the study, 66(45%) students were females and 82(55%) were male. Majority of the students were satisfied with the quality of the exam. Consensus about the quality of exam was that, 29.7% were aware about the nature of the exam, 52.7% were satisfied that the syllabus taught was asked in the exam, 58.1% were satisfied about the time allocation for each station. Majority i.e. 60% considered OSCE an exam of practical nature which is not biased by gender or ethnicity. More than 50% of the students were satisfied with the standard of the exam. At the same time more than 50% students considered essay exam the easiest format of assessment. However, OSCE was considered to be fairest form of assessment 73%. 68.9% perceived that learning is enhanced by MCQs rather than other formats of assessment. Conclusion: To conclude this study, it is very much clear that the perception of students about OSCE as an assessment tool was very encouraging, as it not only provided them the opportunity to highlight their weaknesses but also helped them to perform well in the exam, manage time during exam and to overcome them stress which influenced their results. Key words: OSCE, Objective, Examinations, Clinical skills, qualitative analysis

Objective structured clinical examination in fundamentals of nursing and obstetric care as method of verification and assessing the degree of achievement of learning outcomes

Abstract Aim. The aim of the study was (1) to assess the degree of preparedness of nursing and midwifery students for OSCE-type examinations (Objective Structured Clinical Examination), (2) to learn the opinions of students regarding examination in the form of the OSCE and (3) to learn about the emotions which accompany the examination. Material and methods. The number of 160 (53.33%) nursing students and 140 (46.67%) midwifery students from 5 medical universities were studied. The diagnostic survey method and survey techniques were applied. An own questionnaire was used. Results. Students were the best prepared to perform hygienic and nursing procedures (M=4.22, SD=0.85) and the least prepared to perform gastrointestinal procedures (M=3.03, SD=1.24). The majority (237/79%) claimed that the OSCE examiners should be lecturers from their home university. High marks were given to the quantity and quality of equipment prepared for the examination (232/77.34%) and knowledge of the rules and procedure of conducting the examination (226/75.33%). The OSCE examination was accompanied by high levels of stress (M=4.64, SD=0.78) and fear of making a mistake (M=4.54, SD=0.75). One of the lowest rated aspects of the OSCE exam by students was the time allocated to completing tasks (M=2.96, SD=1.21). Conclusions. Because students rated low the time allocated to the tasks, the lecturers organizing the examination in the future should pay more attention to this aspect. From the very first semester, it is worth considering the introduction of classes in nursing and midwifery aimed at teaching effective ways of coping with stress.

Practical exams

As for practical exams, you can learn in the same way as written exams. If you have studied to become a Veterinary Nurse you will have experienced the dreaded OSCE examination. It took me two attem...

Medical student assessments-frequency of radiological images used: a national study.

Objectives:Assessments are a key part of life for medical students at University. We know there is variation in these assessments across Universities. The aims of this study were to expatiate summative assessments in Scottish Medical Schools and to examine how frequently radiological images feature in them.Methods:All Scottish medical schools were invited to participate in the study. Data on objective structured clinical examinations (OSCEs; 5 years) and written assessments (3 years) were retrospectively collected for each university and results were collated. Each University was randomly assigned a letter from A to E and anonymised for data presentation.Results:10,534 multiple choice questions (MCQ) and 1083 OSCE stations were included in this study. There was wide variation in the number, type and timing of assessments across Scottish medical schools. There were highly significant differences in the number of OSCE stations and the number of MCQs set over the study period (p < 0.0001). Radiological images were used on average 0.6 times (range 0–1.1) in each OSCE examination and 2.4 times (range 0.1–3.7) for written assessments.Conclusion:In this detailed study, we demonstrated significant differences in medical student assessments across Scottish Universities. Given the importance of Radiology in modern medicine, the frequency and differences in which radiological images were used in assessments across Universities should be addressed.Advances in knowledge:This is the first national longitudinal study to quantify the role of radiological images in summative Medical Student Assessments. Great variability exists in the extent and how (clinical versus written assessments) radiological images are used to assess Scottish medical students. Radiological images are used infrequently in clinical assessments, but are present in every written assessment. These findings could help inform medical schools and academic radiologists as they prepare medical students for the imminent unified medical licensing examination, where Clinical Imaging is a subject with one of the highest number of associated conditions examinable.

A randomised controlled pilot trial of the influence of non-native English accents on examiners\u2019 scores in OSCEs

BackgroundObjective structured clinical examinations (OSCEs) are important aspects of assessment in medical education. There is anecdotal evidence suggesting that students with non-native English accents (NNEA) may be subjected to unconscious bias. It is imperative to minimise the examiners’ bias so that the difference in the scores reflects students’ clinical competence. Research shows NNEAs can cause stereotyping, often leading to the speaker being negatively judged. However, no medical education study has looked at the influence of NNEAs in assessment.MethodsThis is a randomized, single-blinded controlled trial. Four videos of one mock OSCE station were produced. A professional actor played a medical student. Two near identical scripts were prepared. Two videos showed the actor speaking with an Indian accent and two videos showed the actor speaking without the accent in either script. Forty-two OSCE examiners in the United Kingdom (UK) were recruited and randomly assigned to two groups. They watched two videos online, each with either script, each with a different script. One video with a NNEA and one video was without. Checklist item scores were analysed with descriptive statistics and non-parametric independent samples median test. Global scores were analysed with descriptive statistics and Mann-Whitney test.ResultsThirty-two examiners completed the study. The average scores for the checklist items (41.6 points) did not change when the accent variable was changed. Independent samples median test showed no statistically significant relationship between the accent and the scores (p = 0.787). For the global scores received by the videos with the NNEA, there were one less ‘Good’ grade and one more ‘Fail’ grade compared to those without the NNEA. Mann-Whitney test on global score showed lower scores for videos with NNEA (p = 0.661).ConclusionsExaminers were not biased either positively or negatively towards NNEAs when providing checklist or global scores. Further study is required to validate the findings of this study. More discussion is warranted to consider how the accent should be considered in current medical education assessment.RegistrationTrial registration completed trial, ID: ISRCTN17360102, Retrospectively registered on 15/04/2020.

Implementing choosing wisley in undergraduate medical education

The principles of evidence-based medicine and the "choosing wisley" initiatives create a foundation for patient-centred medicine, which can prevent an overuse or undertreatment for patients. In order to achieve nationwide implementation in Germany, it makes sense to anchor the necessary skills, in particular scientific skills, more firmly in medical studies and to test them in the state exams. With the "Masterplan Medizinstudium 2020", undergraduate medical education should focus more on the teaching and learning of skills and abilities, instead of on the strong tradition of imparting facts and knowledge. Innovative examination formats which are based on the "choosing wisley" recommendations of the DGVS, such as key feature cases and OSCE examinations, as well as examinations on patients in the future, are integral to the M4state examination included in the current draft of Medical Licensing Regulations (ÄApprO) and will help with the nationwide implementation of these recommendations.

Comparison of performance on written and OSCE assessment during end semester pediatric examination.

Objective:To compare the scores obtained on written and OSCE examination during pediatrics end semester examination and to find out the reasons for the discrepancies among the scores obtained.Methods:This co-relational study was carried out in pediatric department, Liaquat University of Medical and Health Sciences (LUMHS). The performance of medical students who were posted in the pediatric department for one semester (November 2016 to October 2017) was compared on the basis of scores obtained in their written and OSCE during end semester examination. To find out the reasons for discrepancies qualitative data was collected by using likert scale on Pre-designed questionnaire distributed among the students at the end of assessment. Frequencies of their responses were calculated.Results:Data of 160 students who participated in study was analyzed by SPSS version 22. The mean and standard deviation of participant’s score was 51.25 ± 12.19. Females performed better in written as well as in OSCE. Moderate correlation was seen between MCQ and SAQ scores (r=0.5, p <0.01). Around 60% considered OSCE as their preferred mode of assessment. Deep learning approach and group discussion was found in female students (65%).Conclusion:Our study concluded that students performed better in OSCE than in written assessment. However, in OSCE they had shown unsatisfactory performance for some important competencies like clinical examination methods and procedural skills. Female students performed better in both segments of assessment.

Exploring the relationship between examiners’ memories for performances, domain separation and score variability

Background: OSCE examiners’ scores are variable and may discriminate domains of performance poorly. Examiners must hold their observations of OSCE performances in “episodic memory” until performances end. We investigated whether examiners vary in their recollection of performances; and whether this relates to their score variability or ability to separate disparate performance domains.Methods: Secondary analysis was performed on data where examiners had: 1/scored videos of OSCE performances showing disparate student ability in different domains; and 2/performed a measure of recollection for an OSCE performance. We calculated measures of “overall-score variance” (the degree individual examiners’ overall scores varied from the group mean) and “domain separation” (the degree to which examiners separated different performance domains). We related these variables to the measure of examiners’ recollection.Results: Examiners varied considerably in their recollection accuracy (recognition beyond chance −5% to +75% for different examiners). Examiners’ recollection accuracy was weakly inversely related to their overall score accuracy (R = −0.17, p < 0.001) and related to their ability to separate domains of performance (R = 0.25, p < 0.001).Conclusions: Examiners vary substantially in their memories for students’ performances which may offer a useful point of difference to study processing and integration phases of judgment. Findings could have implication for the utility of feedback.

Managing extremes of assessor judgment within the OSCE

Context: There is a growing body of research investigating assessor judgments in complex performance environments such as OSCE examinations. Post hoc analysis can be employed to identify some elements of “unwanted” assessor variance. However, the impact of individual, apparently “extreme” assessors on OSCE quality, assessment outcomes and pass/fail decisions has not been previously explored. This paper uses a range of “case studies” as examples to illustrate the impact that “extreme” examiners can have in OSCEs, and gives pragmatic suggestions to successfully alleviating problems.Method and results: We used real OSCE assessment data from a number of examinations where at station level, a single examiner assesses student performance using a global grade and a key features checklist. Three exemplar case studies where initial post hoc analysis has indicated problematic individual assessor behavior are considered and discussed in detail, highlighting both the impact of individual examiner behavior and station design on subsequent judgments.Conclusions: In complex assessment environments, institutions have a duty to maximize the defensibility, quality and validity of the assessment process. A key element of this involves critical analysis, through a range of approaches, of assessor judgments. However, care must be taken when assuming that apparent aberrant examiner behavior is automatically just that.

Do role-players affect the outcome of a high-stakes postgraduate OSCE, in terms of candidate sex or ethnicity? Results from an analysis of the 52,702 anonymised case scores from one year of the MRCGP clinical skills assessment

The MRCGP clinical skills assessment (CSA), a high-stakes OSCE examination whereby GPs exit their training, uses trained role-players to depict patients (though not to mark candidates). Just as subgroups of examiners can potentially affect the results of subgroups of candidates through biases expressed in discriminatory marking, so possibly could subgroups of role-players alter the difficulty for different candidate groups. This study looks at the contribution of role-players to such possible systematic unfairness in the assessment. Using multiple linear regression, we examined the data from all 52,702 case scores from the MRCGP CSA for the academic year 2012–2013. Candidates were dichotomised by sex, by ethnicity and by source of primary medical qualification (PMQ); role-players were dichotomised by sex and binary ethnicity; and the transaction of candidate/role-player encounters were classified as ‘same’ or ‘different’ in terms of the two parties’ sex and of their ethnicity. Neither examiner nor role-player characteristics were found to predict any statistically significant portion of case score variance, where the significant (p < .001) predictors were source of PMQ (UK or elsewhere: 11% of case score variance), candidates’ ethnicity (1%), candidates’ sex (0.6%) and ‘transactional’ sex (0.1%). We did not therefore find any substantial degree of support for the proposition that role-player subgroups systematically influence candidate subgroups’ scores.

100 Medical Emergencies for Finals

The quest of all discerning final-year medical students is to make some clinical sense of the volumes of facts that they have methodically memorised over the preceding years of study. This book has been written with this aim in mind. Its objective is to fill in ‘the gaps in the learner's knowledge about the practicalities of treatment’ through the provision of ‘easily digested and clear advice on emergencies’. The authors have drawn the content from their own experience in medical, surgical and general practice. The target population for this book includes medical students, foundation trainees, emergency nurse practitioners and pre-hospital personnel. The layout is very logical with each chapter focusing on a particular specialty. The topics included are those most likely to be encountered during a routine week in the accident and emergency department. Within each chapter each subsection is divided into clinical features, aetiology and management with an accompanying list of references for further reading. This format is virtually uniform throughout the book, which makes it very helpful to the pre-examination candidate in search of a structured system of learning. However, this format has been deviated from in the few surgical sections and, at an initial glance, appears a little wordy. Nevertheless, closer reading reveals valuable and relevant information conforming to the objectives of the book. In the context of aetiology of the conditions included there is a focus on the most common factors rather than comprehensive lists and this is appropriate for the target reader. A very pleasing component in most chapters is the inclusion of algorithmic strategies of management based on the current evidence base. This is particularly applicable to the foundation trainee and junior accident and emergency doctor. The use of clear and relevant diagrams, mnemonics and validated scoring systems also makes this book a useful aid to candidates preparing for higher examinations. In the context of practical treatment, it would have been helpful to see more details on the precise techniques of skills such as catheterisation and chest-drain insertion, which medical students are expected to have an understanding of in the OSCE examination setting. Description of medical management and stabilisation of patients is excellent in this book, with helpful suggestions of appropriate medications to use. The bullet-pointed format of the treatment sections is delineated in a logical fashion, making them directly applicable to the clinical scenario. Comparing this book to other revision aids for final-year students, the strengths include its logical structure, clarity of presentation and breadth of specialties covered. With its content taking account of only the most common conditions, it helps to focus the final-year student on the reality of day-to-day practice instead of the frequent fascination with the rare and bizarre. The main weakness in comparison to other similar-style books such as the Sprigings and Chambers book on Acute Medicine and the Oxford Handbook of Acute Medicine is that it lacks the essential knowledge of detailed practical procedures that both final-year medical students and foundation trainees need to manage the emergency scenario effectively. I do, however, feel that it is a very useful adjunct to revision for the final-year student preparing for a viva voce style OSCE examination.

Preliminary application of the revised objective structure clinical examination stations

Objective To explore the feasibility and effectiveness of the revised OSCE stations. Methods 106 sstudents of grade 2008 and 2009 from Xinhua clinical medical school of Shanghai Jiaotong University School of Medicine were involved, 32 of whom (experimental group) tool modified OSCE examination and 74 of whom (control group) tool traditional OSCE examination. The revised OSCE exam was the one which on the basis of original OSCE test station setup, improved the operating part of the test stand, adopted clinical cases as stem, asked the students to make possible diagnosis, and perform the corresponding operation. We evaluated the students based on the OSCE scores from the school exam and the overall performances in the national clinical skills competition. Comparison between the two groups was performed by using t test. Results All students who have received the new training methods agreed that it stood closer to the clinical setting. Revised-OSCE trained group of students test scores than traditional OSCE trained group of students, though it didn't have statistical significance. Students from the revised-OSCE group had the score of (80.50±15.45), while the traditional group had the score of (78.30±17.68). By taking the revised-OSCE trainings, all the students have improved their clinical skills and two years in a row to win the Shanghai Jiaotong University School of Medicine Clinical Skills Competition. Conclusions Case-based OSCE stations are well accepted. The new OSCE stations are much closer to clinical teaching and can make better objective assessment of students, so it plays a good role in improving medical students' overall performances. Key words: Case-based learning; Objective structure clinical examination; Medical education

Difficulty levels of OSCE items related to examination and measurement skills.

[Purpose] The difficulty levels of level-2 OSCE (examination and measurement skills) items were examined, with a view to providing reference data for the determination of students’ skills. [Subjects] A total of 284 graduates of physical (PT) and occupational (OT) therapy classes of 2011 (59 and 40), 2012 (46 and 36), and 2013 (61 and 42, respectively) were studied, with PT or OT faculty members as OSCE examiners and a simulated patient. [Methods] Scores for 11 level-2 OSCE items were compared between before and after clinical training. [Results] Scores markedly increased after clinical training. On comparison among the items, scores for sensory examination were the highest, and those for interviews were the lowest. [Conclusion] The results of this study indicate the necessity of considering an appropriate combination of different difficulty levels when adopting OSCE-based educational approaches.

The effectiveness of webcast compared to live lectures as a teaching tool in medical school

Objective: The purpose of this study is to investigate whether webcast lectures are comparable to live lectures as a teaching tool in medical school.Methods: Three Otolaryngology-Head&Neck Surgery (OTO-HNS) lectures were given to third year medical students through their regular academic curriculum with one group receiving lectures in a live lecture format and the other group in a webcast format. All lectures (live or webcast) were given by the same lecturer and contained identical material. Three outcome measures were used: a student satisfaction survey, performance on the OTO-HNS component of their written examination, and performance on an OTO-HNS OSCE station in the general end of year OSCE examination session.Results: Students performance on the written examination was equal between the two groups. The webcast group outperformed the live lecture group in the OSCE station. The majority of students in the webcast group felt it was an effective learning tool for them. Most viewed the lectures more than once, and felt that this was beneficial to their learning.Conclusion: Webcasts appear equally effective to live lectures as a teaching tool.

Reliability of the OSCE for Physical and Occupational Therapists.

[Purpose] To examine agreement rates between faculty members and clinical supervisors as OSCE examiners. [Subjects] The study subjects were involved physical and occupational therapists working in clinical environments for 1 to 5 years after graduating from training schools as OSCE examinees, and a physical or occupational therapy faculty member and a clinical supervisor as examiners. Another clinical supervisor acted as a simulated patient. [Methods] The agreement rate between the examiners for each OSCE item was calculated based on Cohen’s kappa coefficient to confirm inter-rater reliability. [Results] The agreement rates for the behavioral aspects of the items were higher in the second than in the first examination. Similar increases were also observed in the agreement rates for the technical aspects until the initiation of each activity; however, the rates decreased during the middle to terminal stages of continuous movements. [Conclusion] The results may reflect the recent implementation of measures for the integration of therapist education in training schools and clinical training facilities.

Dental Students Perception And Attitude Towards Their First Exit Objective Structured Clinical Examination

Aim was to assess students’ perceptions and attitudes towards having their first summative (OSCE) at the dental school of the Libyan International Medical University, Libya. A 14 item questionnaire used in previous publications was adapted to assess 5th year students’ responses. Twenty four students filled the questionnaire immediately after sitting OSCE examination. The questionnaire was administered to the same group of students on 3 study blocks. A 5 point Likert scale was used for rating. The OSCE consisted of total 72 stations. The responses were analyzed using descriptive statistics. The Cronbach’s alpha index of the questionnaire was 0.92 and the mean score of all items was 3.03 (SD 0.09) with a range from 1 to 4. The mean of scores assessed by students regarding the exam easiness was low (2.5). Students score for the adequacy of the stations number and the stations time was 3.3. Based on the evaluations of questionnaires, it could be concluded that our students neutrally evaluated their first experience with OSCE. However, most students perceived it as a difficult method of assessment. Writing stations induced less anxiety than immediately scored ones.

Constructing the Training Curriculum of Standardized Patients for OSCE Examination for Undergraduate Nursing Students in Shanxi Medical University

Objective: The objective of this study is to construct the simulated patient training curriculum for OSCE examination for undergraduate nursing students and to explore the theoretical and practical foundation. Methods: To establish TSP training curriculum and SSP training curriculum, 30 experts were invited to finish the questionnaires which were proved by Delphi Method. Findings: We established the training curriculum of TSP and SSP, and set the weight of various curricula and teaching contents. Conclusions: The experts considered that the degree and importance of these two training curricula were comparable. This conclusion lays the foundation for applying these curricula to teaching practice and clinical practice, and enhancing the teaching outcome of undergraduate nursing students. Implications: This study provided a new way of assessing the clinic ability of nursing students.

Impact of student choice on academic performance: cross-sectional and longitudinal observations of a student cohort

BackgroundStudent choice plays a prominent role in the undergraduate curriculum in many contemporary medical schools. A key unanswered question relates to its impact on academic performance.MethodsWe studied 301 students who were in years 2 and 3 of their medical studies in 2005/06. We investigated the relationship between SSC grade and allocated preference. Separately, we examined the impact of ‘self-proposing’ (students designing and completing their own SSC) on academic performance in other, standard-set, summative assessments throughout the curriculum. The chi-squared test was used to compare academic performance in SSC according to allocated preference. Generalised estimating equations were used to investigate the effect of self-proposing on performance in standard-set examinations.Results(1) Performance in staff-designed SSC was not related to allocated preference. (2) Performance in year 1 main examination was one of the key predictors of performance in written and OSCE examinations in years 2, 3 and 4 (p<0.001). (3) The higher the score in the year 1 examination, the more likely a student was to self-propose in subsequent years (OR [CI] 1.07 [1.03-1.11], p<0.001). (4) Academic performance of students who self-proposed at least once in years 2 and/or 3 varied according to gender and year of course.ConclusionIn this study, no association was observed between allocated preference and SSC grade. The effect of self-proposing on academic performance in standard-set examinations was small. Our findings suggest instead that academically brighter students are more likely to design their own modules. Although student choice may have educational benefits, this report does not provide convincing evidence that it improves academic performance.