Traditional Objective Structured Clinical Examination Research Articles

Comparison of Methods of Assessment and Results of Traditional Osce with Electronic-Osce

Introduction: The onset and rise of COVID-19 and its sudden progression to a worldwide pandemic lead medical and dental institutes to change their way of teaching and conducting assessments to distance learning as compared to the previously applied conventional approaches. Teaching and assessment during this era have changed vastly, earlier it was solely traditional/live Objective Structured Clinical Examination (OSCE) but due to the current circumstances electronic/online OSCE (E-OSCE) method was introduced. In Pakistan also, Pakistan Medical Council (PMC) and the College of Physicians and Surgeons of Pakistan (CPSP) conducted online exams so that the scheduled exams do not get delayed. Aim: The main objective of this paper is to measure the reliability of an E-OSCE and to compare it with the traditional OSCE. Methodology: This was a cross-sectional study which got conducted at the Rawal Institute of Health Sciences, Islamabad. Traditional OSCEs and E-OSCEs were conducted with 71 participants including final year students and house officers. Each OSCEs had 10 stations, including one interactive station in traditional OSCE. Students’ scores in both the OSCEs were collected and paired t-test was used to compare the mean scores at p<0.05. Results: Total number of house officers were 27 and final year students were 44. The scores of 71 participants were collected. The difference between mean scores of house officers’ traditional OSCE and E-OSCE was statistically significant (p=0.000). The difference between final year students traditional OSCE and E-OSCE was also statistically significant (p=0.020). Finally, the overall difference between traditional OSCE and E-OSCE was also statistically significant (p=0.000) Practical implication: The main objective of this study was to assess the reliability of an E-OSCE and to compare whether the electronic method of conducting OSCE is more reliable than the traditional method of conducting OSCE. Conclusion: Despite limitations and the biases, the results of E-OSCE proved to be better than traditional OSCE. Further research needs to be conducted on E-OSCE to control the factors causing biases and limitations. Keywords: Traditional OSCE, E-OSCE, COVID-19, methods of assessment, medical education, dental education, distance learning.

Virtual OSCE Delivery and Quality Assurance During a Pandemic: Implications for the Future.

BackgroundDuring 2020, the COVID-19 pandemic caused worldwide disruption to the delivery of clinical assessments, requiring medicals schools to rapidly adjust their design of established tools. Derived from the traditional face-to-face Objective Structured Clinical Examination (OSCE), the virtual OSCE (vOSCE) was delivered online, using a range of school-dependent designs. The quality of these new formats was evaluated remotely through virtual quality assurance (vQA). This study synthesizes the vOSCE and vQA experiences of stakeholders from participating Australian medical schools based on a Quality framework.MethodsThis study utilized a descriptive phenomenological qualitative design. Focus group discussions (FGD) were held with 23 stakeholders, including examiners, academics, simulated patients, professional staff, students and quality assurance examiners. The data was analyzed using a theory-driven conceptual Quality framework.ResultsThe vOSCE was perceived as a relatively fit-for purpose assessment during pandemic physical distancing mandates. Additionally, the vOSCE was identified as being value-for-money and was noted to provide procedural benefits which lead to an enhanced experience for those involved. However, despite being largely delivered fault-free, the current designs are considered limited in the scope of skills they can assess, and thus do not meet the established quality of the traditional OSCE.ConclusionsWhilst virtual clinical assessments are limited in their scope of assessing clinical competency when compared with the traditional OSCE, their integration into programs of assessment does, in fact, have significant potential. Scholarly review of stakeholder experiences has elucidated quality aspects that can inform iterative improvements to the design and implementation of future vOSCEs.

Virtual Evaluation of Clinical Competence in Nurse Practitioner Students.

All in-person nurse practitioner (NP) student clinical activities were suspended in the spring of 2020 due to COVID-19. This posed a unique summer semester challenge, as students needed to complete an objective structured clinical examination (OSCE). The traditional face-to-face OSCE was revised into a virtual format using a teleconferencing platform and included the following learning objectives: obtain the medical history, describe a focused physical examination, formulate differential diagnoses, and develop an appropriate plan of care. Together with the Health Education Center, students met with a standardized patient (SP) virtually. All elements of the virtual OSCE were the same as the traditional OSCE, except students were required to demonstrate their clinical assessment skills by verbalizing to faculty what they would examine if the visit were in-person. When finished, all participants were invited to complete a survey about their experience. Survey findings revealed that most faculty and students considered the interactive virtual OSCE an extremely effective tool for assessing communication and history taking skills, differential diagnosis, and management of patients. All SPs felt comfortable communicating with the students and felt that the virtual OSCE was a very effective way to assess their interpersonal skills of students. The virtual OSCE also served as an opportunity to incorporate telehealth competencies into a simulation experience for students. This innovative distance learning activity facilitated effective virtual evaluation of clinical competence in NP students and all stakeholders expressed satisfaction with the experience. Most faculty and students strongly agreed that they wanted to continue using the virtual OSCE platform.

Application of network information combined with objective structured clinical examination (OSCE) in the evaluation of clinical skills in obstetrics and gynecology

Objective To explore the value of OSCE system combined with intelligent network in-formation platform in clinical skills assessment of obstetrics and gynecology. Methods 112 clinical medical students who participated in the practice of gynecology and obstetrics in Second Military Medical University in 2017 were randomly divided into the experimental group (network information OSCE) and the control group (traditional OSCE). The teaching results were evaluated by the questionnaire survey of teachers and students and the examination results as well. The statistical analysis was made with the Chi-square test and the t test respectively. Results According to the questionnaire survey of two skills assessment methods, the satisfaction index of the experimental group was higher than that of the control group in both teachers and students, and the difference was statistically significant (P 0.05). Conclusion OSCE combined with the network information system has an unparalleled advantage in the assessment of the obstetrics and gynecology department. The system will promote clinical education of obstetrics and gynecology and the evaluation of the clinical ability of the medical students to a new height, which deserves popularization. Key words: Objective structured clinical examination (OSCE); Network information; Obstetrics and gynecology

Done or Almost Done? Improving OSCE Checklists to Better Capture Performance in Progress Tests

ABSTRACTConstruct: The impact of using nonbinary checklists for scoring residents from different levels of training participating in objective structured clinical examination (OSCE) progress tests was explored. Background: OSCE progress tests typically employ similar rating instruments as traditional OSCEs. However, progress tests differ from other assessment modalities because learners from different stages of training participate in the same examination, which can pose challenges when deciding how to assign scores. In an attempt to better capture performance, nonbinary checklists were introduced in two OSCE progress tests. The purposes of this study were (a) to identify differences in the use of checklist options (e.g., done satisfactorily, attempted, or not done) by task type, (b) to analyze the impact of different scoring methods using nonbinary checklists for two OSCE progress tests (nonprocedural and procedural) for Internal Medicine residents, and (c) to determine which scoring method is better suited for a given task. Approach: A retrospective analysis examined differences in scores (n = 119) for two OSCE progress tests (procedural and nonprocedural). Scoring methods (hawk, dove, and hybrid) varied in stringency in how they awarded marks for nonbinary checklist items that were rated as done satisfactorily, attempted, or not done. Difficulty, reliability (internal consistency), item-total correlations and pass rates were compared for each OSCE using the three scoring methods. Results: Mean OSCE scores were highest using the dove method and lowest using the hawk method. The hawk method resulted in higher item-total correlations for most stations, but there were differences by task type. Overall score reliability calculated using the three methods did not differ significantly. Pass–fail status differed as a function of scoring methods and exam type, with the hawk and hybrid methods resulting in higher failure rates for the nonprocedural OSCE and the dove method resulting in a higher failure rate for the procedural OSCE. Conclusion: The use of different scoring methods for nonbinary OSCE checklists resulted in differences in mean scores and pass–fail status. The results varied with procedural and nonprocedural OSCEs.

Validity and Reliability of the Standardized Orthopedic Assessment Tool (SOAT): A Variation of the Traditional Objective Structured Clinical Examination

Health care professions have replaced traditional multiple choice tests or essays with structured and practical, performance-based examinations with the hope of eliminating rater bias and measuring clinical competence. To establish the validity and reliability of the Standardized Orthopedic Assessment Tool (SOAT) as a measure of clinical competence of orthopaedic injury evaluation. Descriptive laboratory study. University. A total of 60 undergraduate students and 11 raters from 3 Canadian universities and 1 standardized patient. Students were required to complete a 30-minute musculoskeletal evaluation in 1 of 2 randomly assigned mock scenarios involving the knee (second-degree medial collateral ligament sprain) or the shoulder (third-degree supraspinatus muscle strain). We measured interreliability with an intraclass correlation coefficient (ICC) (2,k) and stability of the tool with standard error of measurement and confidence intervals. Agreement was measured using Bland-Altman plots. Concurrent validity was measured using a Pearson product moment correlation coefficient whereby the raters' global rating of a student was matched to the cumulative mean grade score. The ICCs were 0.75 and 0.82 for the shoulder and knee cases, respectively. Bland-Altman plots indicated no systematic bias between raters. In addition, Pearson product moment correlation analysis demonstrated a strong relationship between the overall cumulative mean grade score and the global rating score of the examinees' performances. This study demonstrated good interrater reliability of the SOAT with a standard error of measurement that indicated very modest stability, strong agreement between raters, and correlation indicative of concurrent validity.

A Method to Enhance Student Teams in Palliative Care: Piloting the McMaster-Ottawa Team Observed Structured Clinical Encounter

The need for palliative and end-of-life care (PEOLC) education in prelicensure education has been identified. PEOLC requires effective collaborative teamwork. The competencies required for effective collaborative teamwork are only now emerging and methods to evaluate them must be developed. The adaptation of the traditional Objective Structured Clinical Examination (OSCE) for assessment of a student team addressing palliative care issues was undertaken. The McMaster-Ottawa Team Observed Structured Clinical Encounter (TOSCE) is intended as a formative evaluation tool for both competencies in interprofessional collaboration for patient-centered practice and PEOLC. Three stations based on palliative care scenarios were developed. From January 2007 to January 2008, a total of 141 students and 38 observers participated in the evaluation of three stations, with 6-7 students per group and two observers per station. Observers completed checklists for both PEOLC and interprofessional collaborative competencies and, after completing the TOSCEs, students and observers completed questionnaires on their feasibility and acceptability. Eighty-nine percent of the students and 44% of the observers were from medicine. Students and observers found the TOSCE to be an acceptable and feasible assessment tool for both sets of competencies. Reliability and validity data show that the items in both the clinical and interprofessional checklists fit well together, and interrater reliability is readily achieved. The new formative evaluation TOSCE tool, adapted from the traditional OSCE, was acceptable and feasible to students and observers.

Use of Video-Projected Structured Clinical Examination (ViPSCE) instead of the traditional oral (Viva) examination in the assessment of final year medical students.

Assessment of medical students using the traditional oral (viva) system has been marred by being highly subjective, non-structured, and biased. The use of the objective structured clinical examination (OSCE) would circumvent these disadvantages. The OSCE is, however, costly and time-consuming particularly if used for assessment of large numbers of students. The need for another form of examination that enjoys the advantages of the OSCE while avoiding its disadvantages in the face of limited resources has been the inspiration behind this innovative approach. (1) To identify the characteristics of the new Video-Projected Structured Clinical Examination (ViPSCE). (2) To compare the acceptability of ViPSCE and OSCE by students and tutors. (3) To compare the time-effectiveness of ViPSCE and OSCE. We used a slide video projection to assess the surgical knowledge, problem solving and management abilities of 112 final year medical students at Alazhari University, Khartoum, Sudan. Students completed evaluation forms at the end of the examination. The administration of the ViPSCE was smooth and straightforward. Feedback of the students showed that they preferred the ViPSCE to both traditional oral (viva) examination and OSCE. The examination time was 2 hours using video projection compared to the 6 hours that it used to take a class of 112 students to complete a classical OSCE. The ViPSCE is a better replacement for the traditional oral exam. It is much less time- consuming than traditional OSCE.