Total Objective Structured Clinical Examination Score Research Articles

To what extent does the Health Professions Admission Test-Ireland predict performance in early undergraduate tests of communication and clinical skills? An observational cohort study.

BackgroundInternationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills.MethodStudents enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models.ResultsData were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest.ConclusionWe found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course.

Development and Implementation of an Objective Structured Clinical Examination (OSCE) in Neuromuscular Physical Therapy

Background and Purpose. Designing evaluative experiences that can objectively assess clinical performance in didactic settings to ensure adequate skills preparation allows students to be “practice ready” to perform in clinical settings. Objective structured clinical examinations (OSCEs), commonly used by health professions, effectively assess clinical performance of students; however, little literature exists about using OSCEs to assess clinical skills with primarily neuromuscular populations. This project investigated the ability to assess clinical performance in neuromuscular physical therapy using OSCEs. Method/Model Description and Evaluation. Examination stations were developed, validated for content by experts, field tested on recent graduates, pilot tested, and analyzed for reliability. An 8-station OSCE tested performance of examination and intervention procedures, interaction with patients, interpretation of observations, and analysis of clinical situations via 4 interactive, standardized patient stations and 4 scenario-based, written/video stations. Third-year doctor of physical therapy students (n = 66) in their last didactic semester before a 6-month final clinical rotation participated in the pilot testing over a 1-year period. Outcomes. Test stations were valid and reliable (Cronbach's alpha = 0.72). Factor analysis confirmed 2 components to the OSCE, isolating domains representing standardized patient stations and written/video stations. All stations significantly correlated with the total OSCE score, and significant correlations were identified among patient interaction stations (r = .533 to .884). OSCE scores were highly correlated to the final grade in the course and overall grade point average (GPA). Discussion and Conclusion. Aside from higher than normal anxiety prior to and after the OSCE, students' perception of ability to perform examinations, interventions, or interpret results did not change before and after the OSCE. Successful use of OSCEs in physical therapy programs to assess knowledge and skills related to the management of patients with neuromuscular dysfunction is feasible. More studies investigating the relationship between OSCE performance and clinical performance need to occur.

Peer teaching: a randomised controlled trial using student-teachers to teach musculoskeletal ultrasound

This study (a post-intervention assessment) was designed to assess the effectiveness of peer-assisted learning (PAL) using student-teachers (STs) with limited training to teach complicated technical skills for interpreting ultrasound images of the shoulder. Students in Years 3 and 4 of medical school were randomly assigned to two groups. In the PAL group (PG), teaching was delivered by a group of nine STs from Years 3 and 4, who undertook a 30-minute general training and 1 week of self-teaching. In the staff-led group (SG), students were taught by a group of three ultrasound-experienced doctors. Exposure took place in two separate lessons (each of 120 minutes) and introduced eight standard sectional planes (EULAR) using a 10-MHz Nemio XG system (Toshiba Medical Systems GmbH). The theoretical and practical learning outcomes were tested using a multiple-choice question (MCQ) test and an objective structured clinical examination (OSCE). Qualitative differences were evaluated using Likert scale-based items. Evaluation of differences between the PG (n = 75) and SG (n = 76) in the theoretical (MCQ score; P = 0.644) and practical (total OSCE score; P = 0.133) outcomes showed no difference between the two groups. However, the STs themselves showed significantly better results overall (P < 0.05). Staff members were rated more highly than STs, especially on items relating to competence (P < 0.05). Complicated technical skills can be adequately taught to students using the PAL system by STs with limited training. Self-teaching learning strategies are successful in contexts of limited teacher training. However, despite positive objective results, STs still face prejudice from students with regard to competency.

The relationship between second‐year medical students’ OSCE scores and USMLE Step 2 scores

A valid tool to measure clinical competency early in medical school could identify students who may require special educational attention. The overall aim is to assess the relationship between students' scores on an objective structured clinical examination (OSCE) given in the second year of medical school and their subsequent performance on Step 2 of the United States Medical Licensing Examination (USMLE Step 2). Participants were 390 second-year medical students participating in a required OSCE; complete data (Medical College Admission Test, OSCE, USMLE Step 1 and Step 2 scores) were available for 340 students (87%). Univariate correlations and linear regression analyses were performed. Total OSCE score was moderately correlated with USMLE Step 2 score (r = 0.395, P < 0.001), as were two skills subscores of the OSCE, differential diagnosis (r = 0.343, P < 0.001) and identification of abnormality (r = 0.322, P < 0.001). In linear regression analysis, neither OSCE total score nor any of the subscores independently predicted Step 2 scores; only Step 1 score (beta = 0.687, P < 0.001) and female sex (beta = 0.152, P < 0.001) remained independent correlates of Step 2 score. OSCEs early in medical school can be useful in the early assessment of clinical competence.

Patients' global ratings of student competence. Unreliable contamination or gold standard?

To determine whether global ratings by patients are valid and reliable enough to be used within a major summative assessment of medical students' clinical skills. In 11 stations of an 18-station objective structured clinical examination (OSCE), where a student was asked to educate or take a history from a patient, the patient was asked, 'How likely would you be to come back and discuss your concerns with this student again?' These 11 opinions were aggregated into a single patient opinion mark and correlated with other measures of student competence. The patients were not experienced in student assessment. A total of 204 students undertook the OSCE. Reliability of patient opinion across all 11 stations revealed a Cronbach alpha of 0.65. The correlation coefficient between the patient ratings and the total OSCE score was good (r = 0.74; P < 0.001) and was better than the correlation between any single OSCE station and the total OSCE score. It was also better than the correlation between the aggregated patient opinion and tests of student knowledge (r = 0.47). It is known that patients can reliably complete checklists of clinical skills and that doctors can reliably provide global ratings of students. We have now shown that, by controlling the context, asking the right question and aggregating several opinions, untrained patients can provide a reliable and valid global opinion that contributes to the assessment of a student's clinical skills.

How well do second-year students learn physical diagnosis? Observational study of an Objective Structured Clinical Examination (OSCE).

BackgroundLittle is known about using the Objective Structured Clinical Examination (OSCE) in physical diagnosis courses. The purpose of this study was to describe student performance on an OSCE in a physical diagnosis course.MethodsCross-sectional study at Harvard Medical School, 1997–1999, for 489 second-year students.ResultsAverage total OSCE score was 57% (range 39–75%). Among clinical skills, students scored highest on patient interaction (72%), followed by examination technique (65%), abnormality identification (62%), history-taking (60%), patient presentation (60%), physical examination knowledge (47%), and differential diagnosis (40%) (p < .0001). Among 16 OSCE stations, scores ranged from 70% for arthritis to 29% for calf pain (p < .0001). Teaching sites accounted for larger adjusted differences in station scores, up to 28%, than in skill scores (9%) (p < .0001).ConclusionsStudents scored higher on interpersonal and technical skills than on interpretive or integrative skills. Station scores identified specific content that needs improved teaching.