Objective Structured Research Articles

A study on the impact of open source metaverse immersive teaching method on emergency skills training for medical undergraduate students

BackgroundIn recent years, the traditional simulation-based medical teaching approach has faced challenges in meeting the requirements of practical emergency medicine education. This study utilized open-source tools and software to develop immersive panoramic videos using virtual reality technology for emergency medical teaching. It aims to investigate the efficacy of this novel teaching methodology. This transformation shifted the focus from physical simulation to virtual simulation in medical education, establishing a metaverse for emergency medical teaching.MethodsIn accordance with the curriculum guidelines, the instructors produced panoramic videos demonstrating procedures such as spinal injury management, humeral fracture with abdominal wall intestinal tube prolapse, head and chest composite injuries, cardiopulmonary resuscitation, and tracheal intubation. Using Unity software, a virtual training application for bronchoscopy was developed and integrated into the PICO4 VR all-in-one device to create a metaverse teaching environment. Fourth-year medical undergraduate students were allocated into either an experimental group (n = 26) or a control group (n = 30) based on student IDs. The experimental group received instruction through the metaverse immersive teaching method, while the control group followed the traditional simulation-based medical teaching approach. Both groups participated in theoretical and practical lessons as usual. Subsequently, all students underwent a four-station Objective Structured Clinical Examination (OSCE) to assess the effectiveness of the teaching methods based on their performance. Additionally, students in the experimental group provided subjective evaluations to assess their acceptance of the new teaching approach.ResultsBefore the training commenced, there were no significant statistical differences in the first aid test scores between the experimental and control groups. Following the training, the experimental group outperformed the control group in the four-station OSCE examination, with all P-values being less than 0.05. The satisfaction rate among the experimental group regarding the new teaching method reached 88.46%, reflecting levels of satisfaction and extreme satisfaction.ConclusionThe open-source metaverse immersive teaching method has demonstrated a positive impact on enhancing the emergency skills of medical undergraduate students, with a high level of acceptance among students. In comparison to traditional simulated medical teaching methods, this approach requires less time and space, incurring lower costs, and is deemed worthy of wider adoption.

Investigations into the Geometric Calibration and Systematic Effects of a Micro-CT System.

Micro-Computed Tomography (µCT) systems are used for examining the internal structures of various objects, such as material samples, manufactured parts, and natural objects. Resolving fine details or performing accurate geometric measurements in the voxel data critically depends on the precise calibration of the µCT systems geometry. This paper presents a calibration method for µCT systems using projections of a calibration phantom, where the coordinates of the phantom are initially unknown. The approach involves detecting and tracking steel ball bearings and adjusting the unknown system geometry parameters using non-linear least squares optimization. Multiple geometric models are tested to verify their suitability for a self-calibration approach. The implementation is tested using a calibration phantom captured at different magnifications. The results demonstrate the system's capability to determine the geometry model parameters with a remaining error on the detector between 0.27 px and 0.18 px. Systematic errors that remain after calibration, as well as changing parameters due to system instabilities, are investigated. The source code of this work is published to enable further research.

Video-Based Performance Analysis in Pituitary Surgery - Part 2: Artificial Intelligence Assisted Surgical Coaching

Superior surgical skill improves surgical outcomes in endoscopic pituitary adenoma surgery. Video-based coaching programs, pioneered in professional sports, have shown promise in surgical training. In this study, we developed and assessed a video-based coaching program using artificial intelligence (AI) assistance. An AI-assisted video-based surgical coaching was implemented over 6months with the pituitary surgery team. The program consisted of 1) monthly random video analysis and review; and 2) quarterly 2-hour educational meetings discussing these videos and learning points. Each video was annotated for surgical phases and steps using AI, which improved video interactivity and allowed the calculation of quantitative metrics. Primary outcomes were program feasibility, acceptability, and appropriateness. Surgical performance (via modified Objective Structured Assessment of Technical Skills) and early surgical outcomes were recorded for every case during the 6-month coaching period, and a preceding 6-month control period. Beta and logistic regression were used to assess the change in modified Objective Structured Assessment of Technical Skills scores and surgical outcomes after the coaching program implementation. All participants highly rated the program's feasibility, acceptability, and appropriateness. During the coaching program, 63 endoscopic pituitary adenoma cases were included, with 41 in the control group. Surgical performance across all operative phases improved during the coaching period (P < 0.001), with a reduction in new postoperative anterior pituitary hormone deficit (P= 0.01). We have developed a novel AI-assisted video surgical coaching program for endoscopic pituitary adenoma surgery - demonstrating its viability and impact on surgical performance. Early results also suggest improvement in patient outcomes. Future studies should be multicenter and longer term.

Strengthening the planning and design of Objective Structured Clinical Examinations.

Although Objective Structured Clinical Examinations (OSCEs) offer innovative, objective, and fair methods of clinical assessment, their quality is compromised by poor planning and design. This study aimed to describe the development and present evidence-based recommendations on strengthening the planning and design of OSCEs for a South African public College of Nursing. A South African public College of Nursing. Recommendations were developed based on synthesising two sets of qualitative data. Set 1 included two main themes with each of the four sub-themes related to barriers and facilitators towards quality in OSCE designs from 14 nurse educator interviews and 15 external moderator reports. Set 2 included 12 quality measures to be adopted in the quality design of OSCEs derived from an integrative literature review. The draft recommendations were reviewed by eight experts to be finalised. Seven recommendations were developed for strengthening OSCEs' planning and design, related to: (1) policy framework, standard operating procedures and stakeholder code of conduct; (2) blueprinting and mapping of the OSCE content; (3) developing a bank of OSCE stations; (4) scoring rubric and standard-setting method selection; (5) examiners and standardised patients' recruitment and training; (6) venue selection; and (7) station piloting. The seven developed recommendations can strengthen the quality of OSCEs in the South African public College of Nursing context. The developed recommendations can assist nurse educators in planning and designing to conduct quality OSCEs following piloting and implementation.

Initial surgical performance in robot-assisted radical prostatectomy is associated with clinical outcomes and learning curves.

The association between surgical performance ratings and clinical outcomes in robotic surgery is poorly understood. Additionally, no studies have reported on the relationship between the surgeon's initial case-skill evaluation and the learning curve in robot-assisted surgery. We evaluated whether an objective surgical technique evaluation score for initial robot-assisted radical prostatectomy (RARP) was associated with clinical outcomes and surgeons' learning curves. Six surgeons who were trained in and started to perform RARP at our institution were included. Anonymized, unedited videos of each surgeon's 10th RARP case were evaluated by three reviewers, using modified Objective Structured Assessment of Technical Skill (OSATS) scores. We then divided the surgeons into two groups on the basis of these OSATS scores. We retrospectively compared the clinical outcomes and learning curves of the console time of the two groups for consecutive RARPs, performed from March 2018 to July 2023. We analyzed 258 RARPs (43 cases/surgeon), including 129 cases performed by high-OSATS score surgeons (18.2-19.3 points) and 129 cases performed by low-OSATS score surgeons (11.9-16.0 points). Overall, the high-OSATS score group had significantly shorter operation and console times than the low-OSATS score group did (both P < 0.01) and their patients' rate of continence recovery by 3months post-RARP was significantly higher (P = 0.03). However, complications, blood loss, and positive margins did not differ between the groups (P = 0.08, P = 0.51, and P = 0.90, respectively). The high-OSATS score group had a significantly shorter console time than the low-OSATS score group did after the 11-20 cases. The OSATS score in early RARP cases can predict subsequent surgical outcomes and surgeons' learning curves.

In situ simulation educational intervention in emergency care nursing: from identifying learning needs to developing clinical and transferable skills

Introduction: The nursing team is crucial in emergency care (EC) settings, serving as patients’ first point of contact. However, gaps in their training have been hindering the effectiveness of care and patient survival. In situ simulation (ISS) has been recognized as a valuable tool in EC education, uncovering hidden risks in clinical care areas. This study aimed to analyse how an ISS educational intervention in the EC department impacted participants’ clinical and transferable skills. Methods: To assess the effectiveness of an educational intervention on the 37 EC nursing professionals, a 12-month study quasi-experimental study was conducted. The study consisted of a survey to identify participants’ learning needs and four workshops tailored to meet the team’s specific needs, culminating in an ISS session. Knowledge levels were evaluated using pre- and post-tests administered before (T0) and after (T1) the workshops, along with a formative assessment with an Objective Structured Clinical Examination (OSCE) holistic marking guide to measure the clinical and transferable skills acquired during the ISS. Results: The primary participant’s learning needs centred around cardiopulmonary arrest caused by acute myocardial infarction, including interpreting electrocardiogram and teamwork. These topics were used to develop the workshop and the simulation scenario. A significant difference (p &lt; 0.001) was observed in the levels of learning between T0 and T1, indicating a marked improvement in the participants’ performance. The nursing professionals displayed proficient clinical and transferable skills during the ISS, successfully completing over 80% of the OSCE items. Discussion: These findings suggested that a customized educational intervention incorporating participatory learning and ISS effectively enhances clinical and transferable skills among nursing professionals. The positive outcomes observed in post-tests and OSCE holistic assessment highlighted the effectiveness of this intervention for the EC nursing team. The results of this study demonstrate that active participation in workshops facilitates knowledge acquisition. Furthermore, the OSCE results illustrate the practical application of clinical and transferable skills during the ISS.

Temporal segmentation and "look ahead" simulation: Physical events structure visual perception of intuitive physics.

How we perceive the physical world is not only organized in terms of objects, but also structured in time as sequences of events. This is especially evident in intuitive physics, with temporally bounded dynamics such as falling, occlusion, and bouncing demarcating the continuous flow of sensory inputs. While the spatial structure and attentional consequences of physical objects have been well-studied, much less is known about the temporal structure and attentional consequences of physical events in visual perception. Previous work has recognized physical events as units in the mind, and used presegmented object interactions to explore physical representations. However, these studies did not address whether and how perception imposes the kind of temporal structure that carves these physical events to begin with, and the attentional consequences of such segmentation during intuitive physics. Here, we use performance-based tasks to address this gap. In Experiment 1, we find that perception not only spontaneously separates visual input in time into physical events, but also, this segmentation occurs in a nonlinear manner within a few hundred milliseconds at the moment of the event boundary. In Experiment 2, we find that event representations, once formed, use coarse "look ahead" simulations to selectively prioritize those objects that are predictively part of the unfolding dynamics. This rich temporal and predictive structure of physical event representations, formed during vision, should inform models of intuitive physics. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Effect of lavender aromatherapy on the accuracy, exam anxiety and performance of nursing students in Objective Structured Clinical Examination (OSCE): a randomized clinical trial

Effect of lavender aromatherapy on the accuracy, exam anxiety and performance of nursing students in Objective Structured Clinical Examination (OSCE): a randomized clinical trial

The Impact of Simulator Training of Vascular Anastomosis and Video Assessment for Surgical Residents

IntroductionThe rapid increase of minimally invasive surgery and the shortened training period for surgical residents has resulted in limited opportunities to acquire proficiency in open surgical techniques, such as vascular anastomosis. However, vascular anastomosis remains an essential skill in every surgery for bleeding control. This study aimed to validate the effectiveness of surgical education model for vascular anastomosis and assess the impact on the comprehension, skill, and confidence of surgical residents in performing vascular anastomosis. MethodsA total of 21 surgical residents with 1st to 3rd years of experience at Seoul National University Hospital participated in a four-week vascular anastomosis training program. The program included an educational lecture and the performance of an end-to-side anastomosis on a procedural model, with evaluations being conducted using the Objective Structured Assessment of Technical Skills (OSATS) and the End Product Rating Score (EPRS) in pre- and post-training surveys. ResultsSignificant improvement was observed in the OSATS score (from 9.22 ± 2.4 in week 1 to 12.87 ± 3.1 in week 4; P < 0.001) and the EPRS score (from 12.47 ± 4.1 in week 1 to 17.57 ± 2.2 in week 4; P < 0.001). Additionally, the surgical performance time significantly decreased from 20.99 ± 4.6 minutes to 16.33 ± 4.2 minutes (P = 0.019) ConclusionsSimulator training of in vitro vascular anastomosis, when accompanied by expert-led instruction, can effectively enhance the surgical proficiency, confidence, and overall surgical outcomes of residents, as inferred from the observed improvements in OSATS and EPRS scores. The results suggest that integration of this training model into surgical curricula could be a promising strategy for enhancing vascular surgical training.

AACN competency-based essentials revisit: Evidence-based validation of entrustable professional activities in Asia

AimThis study intended to validate the competency-based approach through an entrustable professional activity in the nursing undergraduate education arena in Taiwan. BackgroundEntrustable professional activity is a recommended strategy to enhance nursing competencies and skills. It has been widely applied to nursing education in Western countries, especially graduate programs. However, its effects in eastern countries and undergraduate programs remain unclear. DesignA quasi-experimental comparison design was used. MethodThe study is conducted at the Department of Nursing at a university in southern Taiwan. A total of 72 Two-Year Nursing Program students participated in the study. After implementing the designated entrustable professional activity in the Wound Care Nursing course, outcomes were measured using the Competency Inventory of Nursing Students, Learning Satisfaction and Objective Structured Clinical Examinations. These outcomes were then compared with post-test results at the end of the semester. About 90 % of students completed the study with the test group (n=31) receiving extra activity and the comparison group (n=34) receiving usual teaching. Data were analyzed through chi-square, paired t and Student’s t-test. ResultsThe test group demonstrated significantly higher scores in perceived competency and clinical examinations than the comparison group. However, both groups experienced an increase in learning satisfaction without reaching a significant difference. ConclusionResults of the study indicate that competency-based pedagogy, e.g., using entrustable professional activities, should be integrated into nursing curricula to meet the new American Association of Colleges of Nursing standards with compelling evidence.

Video-Based Performance Analysis in Pituitary Surgery—Part 1: Surgical Outcomes

Endoscopic pituitary adenoma surgery has a steep learning curve, with varying surgical techniques and outcomes across centers. In other surgeries, superior performance is linked with superior surgical outcomes. This study aimed to explore the prediction of patient-specific outcomes using surgical video analysis in pituitary surgery. Endoscopic pituitary adenoma surgery videos from a single center were annotated by experts for operative workflow (3 surgical phases and 15 surgical steps) and operative skill (using modified Objective Structured Assessment of Technical Skills [mOSATS]). Quantitative workflow metrics were calculated, including phase duration and step transitions. Poisson or logistic regression was used to assess the association of workflow metrics and mOSATS with common inpatient surgical outcomes. 100 videos from 100 patients were included. Nasal phase mean duration was 24minutes and mean mOSATS was 21.2/30. Mean duration was 34minutes and mean mOSATS was 20.9/30 for the sellar phase, and 11minutes and 21.7/30, respectively, for the closure phase. The most common adverse outcomes were new anterior pituitary hormone deficiency (n= 26), dysnatremia (n= 24), and cerebrospinal fluid leak (n= 5). Higher mOSATS for all 3 phases and shorter operation duration were associated with decreased length of stay (P= 0.003 &P < 0.001). Superior closure phase mOSATS were associated with reduced postoperative cerebrospinal fluid leak (P < 0.001), and superior sellar phase mOSATS were associated with reduced postoperative visual deterioration (P= 0.041). Superior surgical skill and shorter surgical time were associated with superior surgical outcomes, at a generic and phase-specific level. Such video-based analysis has promise for integration into data-driven training and service improvement initiatives.

Lost in translation: Unveiling medical students' untold errors of medical history documentation.

The accurate documentation of a medical history interview is an important goal in medical education. As students' documentation of medical history interviews is mostly decentralised on the wards, a systematic assessment of documentation quality is missing. We therefore evaluated the extent of details missed in students' medical history reports in a standardised setting. In this prospective, observational study, 123 of 380 students (32.4%) participated in an Objective Structured Clinical Examination (OSCE) regarding history taking and documentation. Based on the interviews and nine deductively selected main categories, a categorical system was established using a summarising qualitative content analysis. The items in the transcripts (defined as ground truth) and in students' reports were labelled and assigned to the correct subcategory. The ground truth and students' reports were compared to quantify students' documentation completeness. Next to the nine deductively selected main categories, 61 subcategories were defined. A total of 8943 items were labelled in the 123 interview transcripts (ground truth), compared with 5870 items labelled in students' reports (65.6% completeness of students' reports compared with ground truth). The main category personal details overlapped with 94.2% between students' report and ground truth incontrast to the main category with the highest discrepancy, allergy, with 41.1% overlap. Pertinent negative items and non-numerical quantifications were often missed. Medical students show incomplete documentation of medical history interviews. Therefore, accurate documentation should be taught as an important goal in medical education.

Evaluating the Role of Objective Structured Clinical Examination as a Summative Assessment Tool in Undergraduate and Postgraduate Psychiatry Residents.

The Objective Structured Clinical Examination (OSCE) is the gold standard and universal format to assess medical students' clinical competence in a comprehensive, reliable, and valid manner. OSCE is gaining global popularity for assessing medical students in various specialties. Our country uses it in multiple disciplines, but its application in psychiatry remains limited. OSCE is a resource-demanding assessment method that can face numerous challenges. A comprehensive assessment of perceptions regarding OSCE can help identify areas that need improvement. Therefore, this study was conducted to assess the perceptions of students and examiners toward OSCE in psychiatry. This study aims to evaluate the role of Objective Structured Clinical Examination as a summative assessment tool in assessing competency in undergraduate (as part of their ward leaving examinations in psychiatry) and postgraduate psychiatry residents (as part of their six-monthly assessments) and analyze the perceptions of students as well as of the faculty members regarding OSCE. Six MD students and 49 MBBS students took the OSCE as part of their ward-level exams in psychiatry. In the presence of four faculty members of the psychiatry department, students completed their six-month summative exams. The OSCE was held at separate times for UG and PG students. UG and PG students utilized different stations (PG students had a harder level). A 10-item questionnaire was given to examiners and students at the end of the OSCE to get their opinions regarding the OSCE. Furthermore, data gathered from the faculty through an open-ended questionnaire was compiled and displayed thematically. Since the Likert scale survey generated ordinal data, the statistical analysis was conducted using the median, interquartile range (IQR), and chi-square test. The chi-square test was used to compare the variables. A P-value of less than 0.05 was deemed statistically noteworthy. Four faculty members and fifty-five students in all answered the questionnaire. Regarding the OSCE's characteristics, validity, reliability, and transparency, the majority of students expressed positive opinions. In a similar vein, most examiners had positive perceptions pertaining to OSCE's administration, structure, and procedures. Certain areas, such as "improved confidence in teaching clinical skills" and "improved confidence in giving students structured feedback," were also emphasized by thematic analysis of faculty members. In general, both students and examiners had extremely favorable perceptions of and embraced the OSCE. Improved faculty orientation and student preparation for the OSCE may help allay anxiety and overcome hesitation related to the exam.

Implementation of a Mental Health Objective Structured Clinical Examination (OSCE) Using Standards of Best Practice.

The objective structured clinical examination (OSCE) is effective for the evaluation of clinical competence. Studies examining the use of OSCEs in undergraduate mental health nursing education in the United States are limited. A pilot study and a follow-up study were conducted to establish the reliability and validity of a mental health OSCE to evaluate the clinical competence of prelicensure nursing students. International Nursing Association for Clinical Simulation and Learning Standards of Best Practice were used to guide the design and implementation. Results from both studies provide evidence for the use of OSCE in undergraduate mental health nursing education.

Practical object and flow structure segmentation using artificial intelligence

This paper explores integrating artificial intelligence (AI) segmentation models, particularly the Segment Anything Model (SAM), into fluid mechanics experiments. SAM’s architecture, comprising an image encoder, prompt encoder, and mask decoder, is investigated for its application in detecting and segmenting objects and flow structures. Additionally, we explore the integration of natural language prompts, such as BERT, to enhance SAM’s performance in segmenting specific objects. Through case studies, we found that SAM is robust in object detection in fluid experiments. However, segmentations related to flow properties, such as scalar turbulence and bubbly flows, require fine-tuning. To facilitate the application, we have established a repository (https://github.com/AliRKhojasteh/Flow_segmentation) where models and usage examples can be accessed.

The Role of Interprofessional Education in Optimizing Laparoscopic Skills Training: A Randomized Controlled Trial.

Background: This investigation evaluates the utility and benefits of integrating interprofessional education (IPE) into laparoscopic training, aiming to enrich medical education and skill acquisition methodologies. Methods: The study randomly allocated 36 participants of a 2023 laparoscopic training course into experimental and control groups, each comprising 18 individuals. The control group underwent traditional theory and practical training, whereas the experimental group additionally engaged in interdisciplinary instruction with nursing educators and participated in simulated laparoscopic surgery exercises. The effectiveness of this interdisciplinary approach was assessed by comparing laparoscopic theory and simulation performance, Objective Structured Assessment of Technical Skills (OSATS) scores in animal-based training, and course satisfaction between the groups. Moreover, the impact on interdisciplinary collaborative competencies was measured through pre- and post-training self-evaluations using the Interprofessional Collaborative Competency Attainment Survey (ICCAS) in the experimental group. Results: The experimental group demonstrated superior performance in laparoscopic theory and simulation, as well as higher OSATS scores, compared with the control group. Satisfaction ratings regarding the skills practice mode, effects, and instructional quality were also significantly better in the experimental group (P < .05, P < .01). Furthermore, participants in the experimental group reported significant pre-to-post training enhancements in interprofessional communication, team collaboration, role perception, conflict management, and learning and feedback marked by statistically significant differences (P < .05, P < .01). Conclusion: The introduction of an IPE framework significantly boosts laparoscopic training efficiency and promotes team collaboration awareness. This model effectively bridges gaps between disciplines, illustrating substantial applicative value and expansion potential within medical education and skill training arenas.

Conducting an objective structured clinical examination under COVID-restricted conditions

BackgroundThe administration of performance assessments during the coronavirus disease of 2019 (COVID-19) pandemic posed many challenges, especially for examinations employed as part of certification and licensure. The National Assessment Collaboration (NAC) Examination, an Objective Structured Clinical Examination (OSCE), was modified during the pandemic. The purpose of this study was to gather evidence to support the reliability and validity of the modified NAC Examination.MethodsThe modified NAC Examination was delivered to 2,433 candidates in 2020 and 2021. Cronbach’s alpha, decision consistency, and accuracy values were calculated. Validity evidence includes comparisons of scores and sub-scores for demographic groups: gender (male vs. female), type of International Medical Graduate (IMG) (Canadians Studying Abroad (CSA) vs. non-CSA), postgraduate training (PGT) (no PGT vs. PGT), and language of examination (English vs. French). Criterion relationships were summarized using correlations within and between the NAC Examination and the Medical Council of Canada Qualifying Examination (MCCQE) Part I scores.ResultsReliability estimates were consistent with other OSCEs similar in length and previous NAC Examination administrations. Both total score and sub-score differences for gender were statistically significant. Total score differences by type of IMG and PGT were not statistically significant, but sub-score differences were statistically significant. Administration language was not statistically significant for either the total scores or sub-scores. Correlations were all statistically significant with some relationships being small or moderate (0.20 to 0.40) or large (> 0.40).ConclusionsThe NAC Examination yields reliable total scores and pass/fail decisions. Expected differences in total scores and sub-scores for defined groups were consistent with previous literature, and internal relationships amongst NAC Examination sub-scores and their external relationships with the MCCQE Part I supported both discriminant and criterion-related validity arguments. Modifications to OSCEs to address health restrictions can be implemented without compromising the overall quality of the assessment. This study outlines some of the validity and reliability analyses for OSCEs that required modifications due to COVID.

Exploring the viability of Virtual Reality as a teaching method for knee aspiration

Knee arthrocentesis is a simple procedure commonly performed by general practitioners and junior doctors. As such, doctors should be competent and comfortable in performing the technique by themselves; however, they need to be adequately trained. The best method to ensure practitioner proficiency is by optimizing teaching at an institutional level, thus, educating all future doctors in the procedure. However, the Coronavirus Disease 19 (COVID-19) pandemic caused significant disruption to hospital teaching for medical students which necessitated investigating the effectiveness of virtual reality (VR) as a platform to emulate hospital teaching of knee arthrocentesis. A workshop was conducted with 100 fourth year medical students divided into three Groups: A, B and C, each receiving a pre-reading online lecture. Group A was placed in an Objective Structured Clinical Examination (OSCE) station where they were assessed by a blinded orthopaedic surgeon using the OSCE assessment rubric. Group B undertook a hands-on practice station prior to assessment, while Group C received a VR video (courtesy of the University of Adelaide’s Health Simulation) in the form of VR headset or 360° surround immersion room and hands-on station followed by the OSCE. Upon completion of the workshop, students completed a questionnaire on their confidence with the procedure and the practicality of the VR station. OSCE scores were compared between Groups B and C to investigate the educational value of VR teaching. On average, students with VR headsets reported higher confidence with the procedure and were more inclined to undertake it on their own. Students in Group C who used the VR station prior to assessment scored higher than the non-VR Groups (Group A, 56%; Group B, 67%; Group C 83%). Students in Group A had statistically significant results on average compared to those in Group B (t(69) = 3.003, p = 0.003), as do students in Group B compared to Group C (t(62) = 5.400, p < 0.001). Within Group C students who were given VR headsets scored higher than immersion room students. The VR headset was beneficial in providing students with a representation of how knee arthrocentesis may be conducted in the hospital setting. While VR will not replace conventional in-hospital teaching, given current technological limitations, it serves as an effective teaching aid for arthrocentesis and has many other potential applications for a wide scope of medicine and surgical training.

Faculty of Pharmacy Shares Experience Transitioning from Traditional Lab Practicals to Objective Structured Pharmacy Assessment (OSPA)

The Faculty of Pharmacy revamped its curriculum for the Bachelor of Pharmacy (Honours) 2019 cohort due to the Covid-19 pandemic. The Nutraceutical (PHC560) course transitioned its hands-on tablet formulation practical to a virtual format with recorded demonstrations. The third cohort's reassessment led to redesigning the lab component into an Objective Structured Pharmacy Assessment (OSPA), focusing on Ethics, Regulatory Aspects, Quality Assurance, and Case Study with Role Play. Positive outcomes have sparked optimism for continued OSPA implementation and potential extension to other courses.

Development and validation of a simulation-based assessment of operative competence for higher specialist trainees in general surgery

BackgroundSimulation is increasingly being explored as an assessment modality. This study sought to develop and collate validity evidence for a novel simulation-based assessment of operative competence. We describe the approach to assessment design, development, pilot testing, and validity investigation.MethodsEight procedural stations were generated using both virtual reality and bio-hybrid models. Content was identified from a previously conducted Delphi consensus study of trainers. Trainee performance was scored using an equally weighted Objective Structured Assessment of Technical Skills (OSATS) tool and a modified Procedure-Based Assessment (PBA) tool. Validity evidence was analyzed in accordance with Messick’s validity framework. Both ‘junior’ (ST2–ST4) and ‘senior’ trainees (ST 5–ST8) were included to allow for comparative analysis.ResultsThirteen trainees were assessed by ten assessors across eight stations. Inter-station reliability was high (α = 0.81), and inter-rater reliability was acceptable (inter-class correlation coefficient 0.77). A significant difference in mean station score was observed between junior and senior trainees (44.82 vs 58.18, p = .004), while overall mean scores were moderately correlated with increasing training year (rs = .74, p = .004, Kendall’s tau-b .57, p = 0.009). A pass-fail score generated using borderline regression methodology resulted in all ‘senior’ trainees passing and 4/6 of junior trainees failing the assessment.ConclusionThis study reports validity evidence for a novel simulation-based assessment, designed to assess the operative competence of higher specialist trainees in general surgery.Graphical abstract