Objective Structured Clinical Examination Format Research Articles

Investigating the Effectiveness of Formative OSCE Combined with Visual Feedback in Improving Clinical Competence among Iranian Nurse Anesthesia Students: A Quasi-experimental study.

Lack of clinical competence can endanger the patient's safety and reduce the quality of providing health care services. The aim of this study was to examine the impact of formative Objective Structured Clinical Examinations (OSCEs) with immediate verbal and visual feedback on the clinical competence of fourth-year nurse anesthesia students. This was a single blind quasi-experimental study with a pre-test/post-test design in compliance with the CONSORT (Consolidated Standards of Reporting Trials) statement. Forty-eight students were allocated to intervention (n=24) and control (n=24) groups. During the academic semester, the students of the intervention group attended 3 sessions of formative OSCE (5 stations) with immediate verbal and visual feedback. In the control group, however, the students received in-person feedback according to curricular routine. The data collection tool included two sections. The first section included a questionnaire to collect demographic information such as age, sex, grade point average and marital status. In the second section, clinical competence of students was measured by Common Clinical Assessment Tool (CCAT). The collected data were analyzed by Analysis of Covariance, paired T-test, Chi-square, and Fischer's exact test in SPSS, version 16. Comparing post-test scores by ANCOVA showed a significant difference between groups (p=0.001) because there was a significant positive change in the overall clinical competence score in the intervention group after receiving formative OSCE. This study showed that regular implementation of formative OSCEs in nurse anesthesia education fosters learning and has a positive effect on improving students' educational behaviors and helps them learn more efficiently. However, it is recommended to conduct more studies with a larger number of participants to confirm this conclusion.

Development and Validation of a Formative Objective Structured Clinical Examination (OSCE) Assessing Home Hemodialysis (HHD) Clinical Skills

Development and Validation of a Formative Objective Structured Clinical Examination (OSCE) Assessing Home Hemodialysis (HHD) Clinical Skills

Early formative objective structured clinical examinations for students in the pre-clinical years of medical education: A non-randomized controlled prospective pilot study

Background The value of formative objective structured clinical examinations (OSCEs) during the pre-clinical years of medical education remains unclear. We aimed to assess the effectiveness of a formative OSCE program for medical students in their pre-clinical years on subsequent performance in summative OSCE. Methods We conducted a non-randomized controlled prospective pilot study that included all medical students from the last year of the pre-clinical cycle of the Université Paris-Cité Medical School, France, in 2021. The intervention group received the formative OSCE program, which consisted of four OSCE sessions, followed by debriefing and feedback, whereas the control group received the standard teaching program. The main objective of this formative OSCE program was to develop skills in taking a structured medical history and communication. All participants took a final summative OSCE. The primary endpoint was the summative OSCE mark in each group. A questionnaire was also administered to the intervention-group students to collect their feedback. A qualitative analysis, using a convenience sample, was conducted by gathering data pertaining to the process through on-site participative observation of the formative OSCE program. Results Twenty students were included in the intervention group; 776 in the control group. We observed a significant improvement with each successive formative OSCE session in communication skills and in taking a structured medical history (p<0.0001 for both skills). Students from the intervention group performed better in a summative OSCE that assessed the structuring of a medical history (median mark 16/20, IQR [15; 17] versus 14/20, [13; 16], respectively, p = 0.012). Adjusted analyses gave similar results. The students from the intervention group reported a feeling of improved competence and a reduced level of stress at the time of the evaluation, supported by the qualitative data showing the benefits of the formative sessions. Conclusion Our findings suggest that an early formative OSCE program is suitable for the pre-clinical years of medical education and is associated with improved student performance in domains targeted by the program.

Impact of Formative Objective Structured Clinical Examination (OSCE) on Clinical Performance of Undergraduate Medical Students in Summative OSCE

Impact of Formative Objective Structured Clinical Examination (OSCE) on Clinical Performance of Undergraduate Medical Students in Summative OSCE

Formative Objective Structured Clinical Examinations (OSCEs) as an Assessment Tool in UK Undergraduate Medical Education: A Review of Its Utility.

The Objective Structured Clinical Examination (OSCE) is a globally established clinical examination; it is often considered the gold standard in evaluating clinical competence within medicine and other healthcare professionals' educations alike. The OSCE consists of a circuit of multiple stations testing a multitude of clinical competencies expected of undergraduate students at certain levels throughout training. Despite its widespread use, the evidence regarding formative renditions of the examination in medical training is highly variable; thus, its suitability as an assessment has been challenged for various reasons. Classically, Van Der Vleuten's formula of utility has been adopted in the appraisal of assessment methods as means of testing, including the OSCE. This review aims to provide a comprehensive overview of the literature surrounding the formative use of OSCEs in undergraduate medical training, whilst specifically focusing on the constituents of the equation and means of mitigating factors that compromise its objectivity.

Formative objective structured clinical examination with immediate feedback improves surgical clerks’ self-confidence and clinical competence

Objective Clerkship is crucial for fourth-year medical students before entering the clinical environment. However, lack of confidence impairs clerks’ performance during the clinical rotation. We assess the impact of formative Objective Structured Clinical Examination (OSCE) with immediate feedback on surgical clerks’ self-confidence and clinical competence. Methods This is a prospective randomized controlled study. Thirty-eight fourth-year medical students starting their surgical clerkship were randomly divided into the control group (n = 19) and the OSCE group (n = 19), where an extra 6-station formative OSCE was given prior to the surgical rotation with immediate feedback on the participant’s performance. Self-confidence assessment (SCA) was collected from each participant before, right after the formative OSCE and one month later. Clinical competence was assessed using a mini-clinical evaluation exercise (mini-CEX) with a case of acute abdominal pain and direct observation of procedural skills (DOPS) with incision and suture one month later. Results The SCAs were significantly improved in the OSCE group right after the training, and a month later, compared to the control group. The mini-CEX score was significantly higher in the OSCE group compared to the control group, but not the DOPS score of incision and suture. Conclusion The formative OSCE with immediate feedback could significantly enhance surgical clerks’ self-confidence and their clinical competence when taking the history, performing the physical examination, and in clinical reasoning; however, the formative OSCE did not improve their dexterity in performing the procedural skills.

Psychometric testing and cost of a five-station OSCE for newly graduated nurses

BackgroundThe Objective Structured Clinical Examination (OSCE) is widely used in nursing education, but its implementation is costly and resource intensive, potentially limiting its feasibility. A five-station OSCE was designed to replace a previously validated 12-station OSCE but was not evaluated for its psychometric properties. PurposeThis study aimed to evaluate the validity, reliability, acceptability, educational impact, and cost of a five-station OSCE using Van der Vleuten's utility formula. MethodsThis study was based on a cross-sectional, nonexperimental design involving psychometric testing, a survey, and a cost analysis. ResultsThe five-station OSCE showed high content and predictive validity and inter-rater reliability among examiners. The OSCE format was well accepted by participants and achieved a cost saving of 45.2% compared with the 12-station format. ConclusionsThe five-station OSCE provides an authentic and objective assessment of competence among newly graduated nurses.

The use of a formative OSCE to prepare emergency medicine residents for summative OSCEs: a mixed-methods cohort study

BackgroundThe objective structured clinical examination (OSCE) is a part of emergency medicine (EM) examinations such as the Masters of Medicine in Emergency Medicine (MMed) examination and the equivalent Member of the Royal College of Emergency Medicine (MRCEM) examination. The use of formative OSCEs to prepare EM residents for summative OSCEs has not been investigated. This study aimed to evaluate the role of formative OSCEs in preparing EM residents for the MMed and MRCEM OSCE.MethodsThis was an observational, retrospective, mixed-methods cohort study. We analysed data from formative OSCEs conducted by the National Healthcare Group EM residency programme from 2013 to 2019, and from a questionnaire distributed to all residents during the study period. Residents’ formative OSCE participation and scores were compared with first-attempt summative OSCE success. Qualitative analysis of residents’ opinions on the usefulness of the formative OSCE was performed.ResultsForty-three of the 50 (86.0%) residents attended at least one formative OSCE. Of the 46 who responded to the questionnaire, 40 (87.0%) had attempted and succeeded in the MMed or MRCEM OSCE, of whom 35 (87.5%) had succeeded on the first attempt. Residents who succeeded in the summative OSCE on the first attempt tended to have higher proximate (mean = 70.6, SD = 8.9 vs mean = 64.3, SD = 10.8) and mean (mean = 67.4, SD = 7.1 vs mean = 62.8, SD = 7.3) formative OSCE scores. All 8/40 (20.0%) residents who attended more than three formative OSCEs succeeded in the summative OSCE on their first attempt. Residents’ formative OSCE scores tended to improve with successive formative OSCEs, demonstrating a positive training effect. All residents felt that the formative OSCE was useful in preparing them for the summative OSCE.ConclusionsParticipation in multiple formative OSCEs was beneficial in preparing residents for the summative OSCE. The formative OSCE was useful in familiarising residents with the examination, giving them an opportunity to perform in near-examination conditions, and providing feedback to residents and faculty about their progress. Our findings may support the implementation of formative OSCEs in other training programmes to prepare learners for high-stake summative OSCEs.

The 3-D Skills Model: a Randomised Controlled Pilot Study Comparing a Novel 1\u20131 Near-Peer Teaching Model to a Formative OSCE with Self-regulated Practice

IntroductionNear-peer teaching is a popular pedagogical teaching tool, with well-recognised benefits for students and tutors. There are multiple existing models to structure these interventions, but it is often unclear how they translate to academic attainment. We designed a novel near-peer teaching model that expands on previous research.MethodsOur model was piloted in a formative Objective Structured Clinical Examination (OSCE) setting, trialled on 22 pre-clinical medical students to establish feasibility, acceptability and descriptive outcomes that could inform the design of a larger study. Students were randomly assigned to intervention or control cohorts. Each cohort undertook 5 min formative OSCE assessments with either 3 additional minutes of structured teaching or 3 min of self-regulated practice before reattempting the first OSCE station. Checklist marking sheets for 1st and 2nd sittings were collected by independent external markers, in addition to a global assessment rating in which we used the Borderline Regression Method to establish the station pass mark.ResultsA quantitative and qualitative result analysis was performed, demonstrating that students gained on average 3 additional marks after teaching with this model. Students and student-tutors reported increased confidence, high course satisfaction and evidence of reflective practice.DiscussionWe established acceptability and feasibility outcomes. The descriptive outcomes will support the design of a larger, adequately powered study required to demonstrate translation to summative exam performance.

Implementation of a capstone course with Formative OSCEs to evaluate preparedness for Advanced Pharmacy Practice Experiences (APPE)

Background: Capstone courses are well documented in pharmacy programmes, but vary in content, methods, and assessment. Aim: To describe the development and implementation of a capstone course for pharmacy students. Description: Components of the capstone course included clinical reviews, pre-tests, calculations, cases, and formative and summative objective structured clinical examinations (OSCEs). Formative OSCEs were a unique aspect of this capstone course, and were used to help students identify clinical strengths and recognise areas of weakness prior to advanced pharmacy practice experiences (APPE). Evaluation: A total of 72 students completed the capstone course in spring 2018. Student survey data indicated that the format of the course was conducive to learning, particularly the use of formative OSCEs. Conclusion: A capstone course was successfully designed and implemented that assessed a variety of pharmacy knowledge and skills prior to APPE. Student feedback and performance in the course provided insight that led to revisions in the pre-APPE curriculum.

Peer marking of OSCEs within a UK pharmacy undergraduate programme - student views

Objective: Objective Structured Clinical Examinations (OSCEs) assess competencies in undergraduate pharmacy students. Students in senior years marked those in the year below during a formative OSCE. Method: The study explored the views of students who marked and had been marked by peers via an online survey. Ethical approval was obtained. Results: 115 (36%) of markers and 114 (35%) of those who were marked responded. Most students who were marked agreed that it was a positive experience and were comfortable receiving feedback from their peers; there were varied views on student marker consistency and how they were not equivalent to staff markers. Student markers felt prepared and confident giving feedback and marks. Markers felt it was a beneficial experience. Working collaboratively with another marker and assessing more than one student was valuable to allow discussion and inform their own OSCE. Conclusion: Overall, students supported this initiative, which has now been embedded into the undergraduate programme.

Medical students' perception and satisfaction on peer-assisted learning in formative OSCE and its effectiveness in improving clinical competencies

PurposePeer-assisted learning (PAL) in formative objective structured clinical examination (OSCE) is an approach used to evaluate the students' clinical competence. This study aims to determine the PAL effectiveness in the formative OSCE in improving clinical competencies and the students' perception and satisfaction of PAL.Design/methodology/approachSixty-seven Year 2 medical students experienced PAL in a formative OSCE setting. A self-administered questionnaire was distributed to evaluate their perception and satisfaction on PAL. The effectiveness of the sessions was determined by comparing their OSCE scores between the pre- and post-intervention.FindingsForty-three students completed the intervention and had the experience to assess their peers as well as being assessed during formative OSCE. However, only 26 students managed to attend both pre- and post-intervention OSCE. It was found that there was a significant improvement of OSCE score after the PAL sessions. Over 97% of these students perceived positively and were satisfied with the sessions and thought that formative OSCE gave an opportunity to improve their learning and develop self-confidence in clinical skills.Originality/valueThe findings suggested that PAL in the formative OSCE is a good opportunity to help in students' learning and improve their clinical competencies apart from learning with experts.

The AaLplus near-peer teaching program in Family Medicine strengthens basic medical skills—A five-year retrospective study

Basic medical skills such as history taking and physical examination are essential components of clinical work profiles, but nevertheless have been neglected by conventional preclinical curricula. The near-peer-teaching program AaLplus [living anatomy plus] teaches basic medical skills, especially history taking, physical examination, and venepuncture, to preclinical students. It is a highly popular compulsory course in the first four semesters (320 students/year, 9h/semester) at Heidelberg University and ends with a formative Objective Structured Clinical Examination (OSCE) during which students receive structured in-depth feedback on their performance. AaLplus is part of the Department of General Practice's longitudinal curriculum for Family Medicine. This study aims to assess whether the AaLplus program has positive effects on students' clinical skill development and subjective confidence in history taking, physical examination and venepuncture. From 2015 to 2019, we asked all AaLplus participants to rate the program and self-assess their medical skills on 5-point Likert scales (min 1, max 5). In 4-station OSCEs, trained tutors rated the students' performance in all taught skills using standardized checklists. From 2015 to 2019 n = 1534 questionnaires returned (response rate = 98.6%, 52.7% females). After course completion, students felt able to take a patient's history (mean 3.97, SD = 0.75) and perform physical examinations (means range 3.82-4.36, SDs range 0.74-0.89) as well as venepuncture (mean 4.12, SD = 0.88). A large majority of students claimed they acquired these skills in the AaLplus program. During OSCE, 81.9% passed anamnesis, 93.1% passed physical examination, and 95.4% passed venepuncture (of n = 1556). Students mostly rated the feedback they received during the OSCE as "helpful" or "very helpful" (means for different stations 4.69-4.76, SDs 0.50-0.70). AaLplus is a positive example of a peer teaching program in the preclinical stage of medical studies. It successfully trains junior students in essential medical abilities and increases their confidence in their skills. A high percentage of students pass the formative OSCE and evaluate it positively. Consistently high ratings indicate the program's routine viability. Further studies are needed to analyze if programs like AaLplus could have an impact on the number of graduates choosing career in Family Medicine.

Blended learning: Improving the diabetes mellitus counseling skills of German pharmacy students

IntroductionPharmacists have the responsibility to conduct medication therapy management and appropriately counsel patients and physicians on medicines. Blended learning and objective structured clinical examinations (OSCEs) may have the potential to address these skills. This study aimed to assess whether a blended learning program consisting of e-learning and OSCEs could enhance pharmacy students' abilities to conduct pharmaceutical consultations with type 2 diabetes mellitus patients. MethodsThe blended learning program consisted of e-learning, formative OSCEs, and summative OSCEs. The e-learning aimed to bring the students to the same level of diabetes knowledge. The students' baseline consultation skills were measured by a summative pre-training OSCE. After the formative OSCEs, a summative post-training OSCE quantified the change in performance. During the pre- and post-training OSCEs, analytical checklists were used to assess consultation skills and a global rating scale assessed communication skills. Surveys were used to measure students' self-assessment and satisfaction. ResultsThe study demonstrated significant improvement in students' pharmaceutical knowledge of diabetes mellitus, consultation performances on diabetes mellitus, and self-assessment scores. Most students responded positively on the satisfaction survey. ConclusionsThis study demonstrated that a blended learning approach with e-learning and OSCEs can improve students' diabetes consultations and communication skills. These results support the use of this instructional technique for teaching patient counseling. Further studies should consider a control group to assess whether the improvement of the consultation performance is a result of the training or solely a learning effect due to the repetition of summative OSCEs.

Assessing Advanced Communication Skills via Objective Structured Clinical Examination: A Comparison of Faculty Versus Self, Peer, and Standardized Patient Assessors

Construct: The construct addressed in this study is assessment of advanced communication skills among senior medical students. Background: The question of who should assess participants during objective structured clinical examinations (OSCEs) has been debated, and options discussed in the literature have included peer, self, standardized patient, and faculty assessment models. What is not known is whether same-level peer assisted learning can be utilized for formative assessment of advanced communication skills when no faculty, standardized patients, or other trained assessors are involved in providing feedback. If successful, such an educational model would optimize resource utilization and broaden the scope of topics that could be covered in formative OSCEs. Approach: The investigators developed a 4-station formative OSCE focused on advanced communication skills for senior medical students, and evaluated the concordance of assessment done by same-level peers, self, standardized patients, and faculty for 45 students. After each station, examinees completed a self-assessment checklist and received checklist-based assessment and verbal feedback from same-level peers only. Standardized patients completed checklist-based assessments outside the room, and faculty did so after the OSCE via video review; neither group provided direct feedback to examinees. The investigators assessed inter-rater agreement and mean difference scores on the checklists using faculty score as the gold standard. Findings: There was fair to good overall agreement among self, same-level peer, standardized patient, and faculty-assessment of advanced communication skills. Relative to faculty, peer and standardized patient assessors overestimated advanced communication skills, while self-assessments underestimated skills. Conclusions: Self and same-level peer-assessment may be a viable alternative to faculty assessment for a formative OSCE on advanced communication skills for senior medical students.

Effect of a Formative Objective Structured Clinical Examination on the Clinical Performance of Undergraduate Medical Students in a Summative Examination: A Randomized Controlled Trial

To study the effect of formative Objective structured clinical examination (OSCE) on the undergraduate medical students' performance in a subsequent summative-OSCE assessment. In a randomized single-blind trial, 130 fifth year medical students at Raparin hospital, Erbil were assigned to intervention (n=61) and control group (n=69). Formative-OSCE was performed for the intervention group in pediatric module with feedback on their performance versus standard pediatric module for the control group. Students' clinical performance was assessed by a summative-OSCE. Multiple regression was used to predict the summative-OSCE score depending on the participation in formative-OSCE along with the other factors. Eleven students were excluded because of early drop-out, leaving 119 students for analysis. The summative-OSCE mean score (out of a total score of 100) in intervention group 64.6 (10.91) was significantly lower as compared to the control group 69.2 (10.45). Single formative-OSCE does not necessarily lead to better performance in subsequent summative-OSCE.

Agreement between simulated patients and faculty: Assessment of communication skills during objective structured clinical examination.

Objective:Ensuring competence in communication skills amongst trainees is essential in health professions education. Involving faculty members for the same is a challenge in Obstetrics and Gynecology (OBGYN) due to their clinical commitments. The present study compares scores of OBGYN faculty, non-OBGYN faculty and simulated patients (SPs) on communication skills of postgraduate trainees during formative Objective Structured Clinical Examination (OSCE).Methods:This is a psychometric study conducted in Feburary 2017 at the Aga Khan University Medical College (AKU-MC). All thirty-two postgraduate trainees of OBGYN gave consent. Each trainee was assessed by OBGYN faculty, non-OBGYN faculty and SP on communication skills at six stations using nine-point itemized rating-scale during formative OBGYN OSCE. The scores were reviewed using descriptive statistics, reliability was calculated using Cronbach’s alpha and inter-rater reliability was analyzed using Pearson correlation and intra-class correlation coefficient.Results:The score reliability of each of the examiners was >0.7. The mean scores showed that OBGYN faculty were most stringent while SPs were lenient examiners, however, non-OBGYN faculty scored in between. The inter-rater reliability among any two of the OBGYN, non-OBGYN and SP examiner was >0.84 using Pearson correlation and >0.9 using intra-class correlation.Conclusion:The SPs and non-OBGYN clinical faculty can also be used to assess communication and counseling skills on OBGYN OSCEs after required training as examiners.

Evaluation of a coaching workshop for the management of veterinary nursing students\u2019 OSCE-associated test anxiety

BackgroundHigh stress levels amongst undergraduates (particularly in relation to assessment) and efforts to improve mental wellbeing have been increasingly reported in the veterinary educational literature. However reports to date have primarily focused on the experiences of students of veterinary medicine, rather than veterinary nursing students.MethodsThe purpose of this mixed method sequential explanatory study was to establish the “Big-five” personality traits and quantify the level of test anxiety associated with objective structured clinical examinations (OSCEs) amongst a cohort of 23 final year veterinary nursing students at an Irish third level college. The 12 item Brief FRIEDBEN Test Anxiety Scale (B-FTAS) and the 20 item mini International Personality Item Pool (mini-IPIP) were used to identify test anxiety levels and personality traits in this cohort. Focus groups were then employed to examine the effectiveness of a coaching intervention in ameliorating this test anxiety.ResultsThe initial, quantitative, phase found these students to have higher levels of test anxiety than previously reported for undergraduates sitting written examinations. No association was found between test anxiety and neurotic personality traits in this student cohort. In the qualitative follow up phase the coaching intervention was reported to have been helpful in equipping the students to better manage test anxiety. The OSCE stressors identified in this study closely resembled those previously reported by nursing and midwifery students.ConclusionsThe shared experience of the coaching intervention and formative OSCE was reported to have been helpful in empowering the students to manage assessment-associated anxiety. Implications and recommendations for educators were identified.

Feedback Credibility in a Formative Postgraduate Objective Structured Clinical Examination: Effects of Examiner Type.

Resident perspectives on feedback are key determinants of its acceptance and effectiveness, and provider credibility is a critical element in perspective formation. It is unclear what factors influence a resident's judgment of feedback credibility. We examined how residents perceive the credibility of feedback providers during a formative objective structured clinical examination (OSCE) in 2 ways: (1) ratings of faculty examiners compared with standardized patient (SP) examiners, and (2) ratings of faculty examiners based on alignment of expertise and station content. During a formative OSCE, internal medicine residents were randomized to receive immediate feedback from either faculty examiners or SP examiners on communication stations, and at least 1 specialty congruent and either 1 specialty incongruent or general internist faculty examiner for clinical stations. Residents rated perceived credibility of feedback providers on a 7-point scale. Results were analyzed with proportional odds models for ordinal credibility ratings. A total of 192 of 203 residents (95%), 72 faculty, and 10 SPs participated. For communication stations, odds of high credibility ratings were significantly lower for SP than for faculty examiners (odds ratio [OR] = 0.28, P < .001). For clinical stations, credibility odds were lower for specialty incongruent faculty (OR = 0.19, P < .001) and female faculty (OR = 0.45, P < .001). Faculty examiners were perceived as being more credible than SP examiners, despite standardizing feedback delivery. Specialty incongruency with station content and female sex were associated with lower credibility ratings for faculty examiners.

How do formative objective structured clinical examinations drive learning? Analysis of residents’ perceptions

Introduction: Although several studies have explored the relationship between learning and written tests, little is understood about how performance-based examinations influence learning. The purpose of this study was to explore how a formative objective structured clinical examination (OSCE) drives learning.Methods: We administered surveys to residents (n = 35) at three time points to determine if and how an OSCE influenced their learning: before and immediately following the OSCE, and after the distribution of their results. Differences in quantitative responses between high- and low-performing residents and across time were compared using repeated-measures ANOVA. Thematic analysis was used to analyze narrative comments.Results: Participants’ goals for the OSCE related to performance, mastery and feedback. Almost all participants reported that they had learned something from the OSCE (94%) and most participants generated learning goals after the OSCE (71%). High performers appeared to recognize the importance of content-related knowledge for scoring well before and after the OSCE, whereas low performers may have under-estimated its importance until after the examination.Discussion: Participants viewed a formative OSCE as both a hurdle to overcome (assessment of learning) and an opportunity to learn (assessment for learning). Understanding how OSCEs influence study behavior can help guide the development of assessments that promote learning.