Performance Assessment Scores Research Articles

Performance in Irish Selection and Future Performance in Surgical Training

Selection processes for surgical training should aim to identify candidates who will become competent independent practitioners and should aspire to high standards of reliability and validity. To determine the association between measured candidate factors at the time of an Irish selection and assessment outcomes in surgical training, examined via rate of progression to Higher Specialist Training (HST), attrition rates, and performance as assessed through a multimodal framework of workplace-based and simulation-based assessments. This retrospective observational cohort study included data from all successful applicants to the Royal College of Surgeons in Ireland (RCSI) national Core Surgical Training (CST) program. Participants included all trainees recruited to dedicated postgraduate surgical training from 2016 to 2020. These data were analyzed from July 11, 2016, through July 10, 2022. Selection decisions were based on a composite score that was derived from technical aptitude assessments, undergraduate academic performance, and a 4-station multiple mini-interview. Assessment data, attrition rates, and rates of progression to HST were recorded for each trainee. CST performance was assessed using workplace-based and simulation-based technical and nontechnical skill assessments. Potential associations between selection and assessment measures were explored using Pearson correlation, logistic regression, and multiple linear-regression analyses. Data were available for 303 trainees. Composite scores were positively associated with progression to HST (odds ratio [OR], 1.09; 95% CI, 1.05-1.13). There was a weak positive correlation, ranging from 0.23 to 0.34, between scores and performance across all CST assessments. Multivariable linear regression analysis showed technical aptitude scores at application were associated with future operative performance assessment scores, both in the workplace (β = 0.31; 95% CI, 0.14-0.48) and simulated environments (β = 0.57; 95% CI, 0.33-0.81). There was evidence that the interpersonal skills interview station was associated with future performance in simulated communication skill assessments (β = 0.55; 95% CI, 0.22-0.87). In this study, performance at the time of Irish national selection, measured across technical and nontechnical domains in a multimodal fashion, was associated with future performance in the workplace and in simulated environments. Future studies will be required to explore the consequential validity of selection, including potential unintended effects of selection and ranking on candidate performance.

Comparison of peer-assisted learning with expert-led learning in medical school ultrasound education: a systematic review and meta-analysis.

Teaching point-of-care ultrasonography (PoCUS) to medical students is resource intensive. Peer-assisted learning, where the teacher can be a medical student, may be a feasible alternative to expert-led learning. The objective of this systematic review and meta-analysis was to compare the PoCUS performance assessments of medical students receiving peer-assisted vs expert-led learning. This study was submitted to PROSPERO (CRD42023383915) and reported with PRISMA guidelines. MEDLINE, Embase, ERIC, Education Source, Scopus, and Web of Science were searched from inception to November 2022. Inclusion criteria were studies comparing peer-assisted vs expert-led PoCUS teaching for undergraduate medical students. The primary outcome was performance assessment of PoCUS skills. Two reviewers independently screened citations and extracted data. The Cochrane risk-of-bias tool for randomized trials was used to assess study quality. Studies were included in the meta-analysis if mean performance assessment scores with standard deviations and sample sizes were available. A random-effects meta-analysis was conducted to estimate the accuracy score of practical knowledge test for each group. A meta-regression evaluated difference in mean scores. The search yielded 2890 citations; 1417 unique citations remained after removing duplicates. Nine randomized-controlled studies conducted in Germany, USA, and Israel, with 593 participants, were included in the meta-analysis. The included studies assessed teaching of abdominal, cardiac, thoracic, musculoskeletal, and ocular PoCUS skills. Most studies had some risk-of-bias concerns. The estimate accuracy score after weighting is 0.56 (95% CI [0.47, 0.65]) for peer-assisted learning and 0.59 (95% CI [0.49, 0.69]) for expert-led learning. The regression coefficient estimate is 0.0281 (95% CI [-0.1121, 0.1683]); P value is 0.69. This meta-analysis found that peer-assisted learning was a reasonable alternative to expert-led learning for teaching PoCUS skills to medical students.

Reliability in performance assessment creates a potential application of artificial intelligence in veterinary education: evaluation of suturing skills at a single institution.

To evaluate suturing skills of veterinary students using 3 common performance assessments (PAs) and to compare findings to data obtained by an electromyographic armband. 16 second-year veterinary students. Students performed 4 suturing tasks on synthetic tissue models 1, 3, and 5 weeks after a surgical skills course. Digital videos were scored by 4 expert surgeons using 3 PAs (an Objective Structured Clinical Examination [OSCE]- style surgical binary checklist, an Objective Structured Assessment of Technical Skill [OSATS] checklist, and a surgical Global Rating Scale [GRS]). Surface electromyography (sEMG) data collected from the dominant forearm were input to machine learning algorithms. Performance assessment scores were compared between experts and correlated to task completion times and sEMG data. Inter-rater reliability was calculated using the intraclass correlation coefficient (ICC). Inter-rater agreement was calculated using percent agreement with varying levels of tolerance. Reliability was moderate for the OSCE and OSATS checklists and poor for the GRS. Agreement was achieved for the checklists when moderate tolerance was applied but remained poor for the GRS. sEMG signals did not correlate well with checklist scores or task times, but features extracted from signals permitted task differentiation by routine statistical comparison and correct task classification using machine learning algorithms. Reliability and agreement of an OSCE-style checklist, OSATS checklist, and surgical GRS assessment were insufficient to characterize suturing skills of veterinary students. To avoid subjectivity associated with PA by raters, further study of kinematics and EMG data is warranted in the surgical skills evaluation of veterinary students.

The Application of Scenario Simulation Teaching Combined with Case Teaching in Medical Education: Taking Nursing Practice Teaching as An Example

Objective: This study aims to investigate the application effect of scenario simulation teaching combined with case teaching method in nursing practice teaching, and compare it with traditional nursing practice teaching. Methods: One hundred nursing students from a university in 2019 were selected as the research subjects, and were randomly divided into an experimental group and a control group, with 50 students in each group. The experimental group received nursing practice teaching using scenario simulation teaching combined with case teaching method, while the control group received traditional nursing practice teaching. The effectiveness of the two teaching methods was evaluated through examination scores, clinical competency assessment, non-technical skill level comparison, and student satisfaction survey. Results: The students in the experimental group performed better than those in the control group in terms of examination scores, clinical competency assessment, non-technical skill level, and student satisfaction. Specifically, the excellent rate and passing rate of the examination scores and clinical competency assessment of the experimental group were significantly higher than those of the control group, with statistically significant differences (P<0.05); after the teaching, the non-technical skill level scores of the experimental group were higher than those of the control group (P<0.05); the satisfaction ratings of the experimental group on teaching content, teaching methods, teaching environment, teacher’s teaching style, and teaching resources were significantly higher than those of the control group (P<0.05). Conclusion: Scenario simulation teaching combined with case teaching method has good application effect in nursing practice teaching, which can effectively improve students’ learning performance and clinical competency assessment scores, enhance students’ understanding and application ability of teaching content, and increase students’ learning interest and participation.

PEMBELAJARAN FISIKA BERBASIS PRAKTIKUM PHYPHOX MENGGUNAKAN MODEL PEMBELAJARAN INKUIRI TERBIMBING

This study aims to determine the ability of students to understand the concept of rolling motion through Phyphox practicum-based learning using the guided inquiry model. This study uses a guided inquiry model which consists of three stages, namely initial knowledge tests, practicum-based learning activities and final knowledge tests. This study uses a percentage analytical descriptive technique, namely by calculating the percentage value of the initial ability test results, the final ability test using the N-Gain and practicum performance assessment scores (roll's experiment) in the learning process. The research is located in the Department of Physics FMIPA-K UNIMA. A sample of active semester 5 students majoring in physics for the 2022/2023 academic year with 15 research subjects. The results of data processing and analysis obtained an average initial test score of 14.33 and an average final test value of 89.2 and performing statistical tests with the N-Gain test obtained an average result of 0.87 so that there was an increase in student physics learning outcomes after applying the treatment and the results of the practicum performance assessment (Roll's Experiment) in the learning process with an average task achievement of 88.75% so that there is an understanding of physics concepts in students.

Dunning-Kruger Effect Between Self-Peer Ratings of Surgical Performance During a MASCAL Event and Pre-Event Assessed Trauma Procedural Capabilities.

The research question asked to what extent do self-rated performance scores of individual surgeons correspond to assessed procedural performance abilities and to peer ratings of procedural performance during a mass casualty (MASCAL) event? Self-assessment using performance rating scales is ubiquitous in surgical education as a proxy for direct measurement of competence. The validity and reliability of self-ratings as competency measures are susceptible to cognitive biases such as Dunning-Kruger effects, which describe how individuals over/underestimate their own performance compared to assessments from independent sources. The ability of surgeons to accurately self-assess their procedural performance remains undetermined. A purposive sample of military surgeons (N = 13) who collectively cared for trauma patients during a MASCAL event participated in the study. Pre-event performance assessment scores for 32 trauma procedures were compared with post-event self and peer performance ratings using F tests (P < 0.05) and effect sizes (Cohen's d). There were no significant differences between peer ratings and performance assessment scores. There were significant differences between self-ratings and both peer ratings (P < 0.001) and performance assessment scores (P < 0.001). Effect sizes were very large for self to peer rating comparison (Cohen's d = 2.34) and self to performance assessment comparison (Cohen's d = 2.77). The outcomes demonstrate that self-ratings were significantly lower than the independently determined assessment scores for each surgeon, revealing a Dunning-Kruger effect for highly skilled individuals underestimating their abilities. These outcomes underscore the limitations of self-assessment for measuring competence.

Identifying Intraoperative Behaviors Associated With Surgical Resident Teachability

The resident-attending dyad influences the intraoperative training of surgery residents. To better understand the role of trainees within the dyad, we hypothesized there is a measurable concept of "teachability," a combination of the trainee's observed skills and behaviors with their performance. This study aims to define teachability and identify discrete intraoperative behaviors that contribute to this concept. We posit that residents who are active learners as demonstrated by asking questions, proposing next steps, and initiating purposeful actions have higher teachability. Previously recorded videos from 26 laparoscopic inguinal hernia repairs performed by two PGY-5 general surgery residents at a Midwest tertiary care center were qualitatively reviewed for intraoperative behaviors. A summative content analysis identified behaviors associated with increased teachability and improved operative performance assessment scores. Average frequencies of intraoperative behaviors for resident 1 and 2 (R1 and R2) were not significantly different, although R2 asked more medical knowledge and technical questions. While the rate of attending feedback was similar for both residents (x=3.82 vs 3.40, p=0.646), R1 consistently incorporated feedback (x=2.27 vs 0.40, p=0.001) whereas R2 needed frequent prompting (x=2.45 vs 1.55, p=0.239). R1 scored higher in all but one operative performance assessment category, including overall performance (x=4.17 vs 2.93, p=0.007), but R2 had a larger magnitude of overall improvement (+1 vs +2). Teachability is a dynamic component of the resident-attending dyad. While intraoperative active learning behaviors do not appear to be associated with teachability, asking questions may increase the magnitude of improvement in performance. Most importantly, the ability to incorporate intraoperative feedback in real time seems to be a critical aspect of teachability and warrants further research.

Does Placement Predict Performance? Associations Between Student Teaching Environments and Candidates’ Performance Assessment Scores

In the present study, we examine whether characteristics of student teaching sites and cooperating teachers predict teacher candidates’ edTPA scores. Using data from North Carolina, we find that candidates earn higher edTPA scores if they student-taught in a high value-added school and with a cooperating teacher earning higher evaluation ratings. Evidence also suggests that candidates earn higher scores when they are mentored by a cooperating teacher who is more familiar with performance assessments. Finally, we find that the relationships between student teaching environments and edTPA scores vary by characteristics of the candidate. This study reinforces prior work on the importance of learning environments to teacher development and the importance of preparation programs securing placements in high-quality learning environments. Our results may also have implications for the use of performance assessments by states and preparation programs.

Association Between Resident Race and Ethnicity and Clinical Performance Assessment Scores in Graduate Medical Education.

To assess the association between internal medicine (IM) residents' race/ethnicity and clinical performance assessments. The authors conducted a cross-sectional analysis of clinical performance assessment scores at 6 U.S. IM residency programs from 2016 to 2017. Residents underrepresented in medicine (URiM) were identified using self-reported race/ethnicity. Standardized scores were calculated for Accreditation Council for Graduate Medical Education core competencies. Cross-classified mixed-effects regression assessed the association between race/ethnicity and competency scores, adjusting for rotation time of year and setting; resident gender, postgraduate year, and IM In-Training Examination percentile rank; and faculty gender, rank, and specialty. Data included 3,600 evaluations by 605 faculty of 703 residents, including 94 (13.4%) URiM residents. Resident race/ethnicity was associated with competency scores, with lower scores for URiM residents (difference in adjusted standardized scores between URiM and non-URiM residents, mean [standard error]) in medical knowledge (-0.123 [0.05], P = .021), systems-based practice (-0.179 [0.05], P = .005), practice-based learning and improvement (-0.112 [0.05], P = .032), professionalism (-0.116 [0.06], P = .036), and interpersonal and communication skills (-0.113 [0.06], P = .044). Translating this to a 1 to 5 scale in 0.5 increments, URiM resident ratings were 0.07 to 0.12 points lower than non-URiM resident ratings in these 5 competencies. The interaction with faculty gender was notable in professionalism (difference between URiM and non-URiM for men faculty -0.199 [0.06] vs women faculty -0.014 [0.07], P = .01) with men more than women faculty rating URiM residents lower than non-URiM residents. Using the 1 to 5 scale, men faculty rated URiM residents 0.13 points lower than non-URiM residents in professionalism. Resident race/ethnicity was associated with assessment scores to the disadvantage of URiM residents. This may reflect bias in faculty assessment, effects of a noninclusive learning environment, or structural inequities in assessment.

Examining the effects of undergraduate STEM teacher recruitment and teacher education programs on preservice secondary science and mathematics teacher readiness and teacher performance assessment (edTPA) scores

AbstractWe examined the effects of teacher education on preservice secondary science and mathematics teacher readiness. The construct of teacher readiness consisted of three dimensions: (a) preservice teachers’ understanding of how to implement current standards, (b) their understanding of how to teach multilingual learners (MLs), and (c) their beliefs about their abilities and skills as educators. To determine teacher readiness, a subset of preservice teachers enrolled in six teacher education programs (TEPs) completed a pre‐ and post‐survey. We found that the undergraduate STEM teacher recruitment program was associated with developing preservice teachers’ understanding of standards‐based instruction but was not associated with the other two dimensions of teacher readiness. We also found that the post‐baccalaureate TEPs were effective in developing preservice teachers’ understanding of ML instruction, whereas the experimental baccalaureate program was not. Further, we found that teacher readiness did not predict participants’ performance assessment (edTPA) scores. Findings suggest that teacher educators and curriculum developers involved in undergraduate STEM teacher recruitment programs should better address language, literacy, and ML instruction. Both teacher recruitment and TEPs should consider ways to strengthen teacher efficacy. Finally, teacher educators should carefully consider how their coursework and field experiences help preservice teachers prepare for the edTPA.

Automated Movement Assessment in Stroke Rehabilitation

We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).

Reliability and Construct Validity of the edTPA for Music Education

The purpose of this study was to examine the psychometric quality of Educative Teacher Performance Assessment (edTPA) scores for 136 preservice music teachers at a Midwest university. I addressed the factor structure of the edTPA for music education, the extent to which the edTPA fits the one- and three-factor a priori models proposed by the test authors, and the reliability of edTPA scores awarded to music education students. Factor analysis did not support the a priori one-factor model around teacher readiness, or the three-factor model based on the edTPA tasks of Planning, Instruction, and Assessment. Internal consistency was acceptable for all rubrics together and for the Instruction task. However, estimates of interrater reliability fell substantially below those reported by test administrators. These findings indicate the need for revision of the edTPA for music education and call into question its continued use among music teacher candidates in its current form.

Examining the relationship between the Educative Teacher Performance Assessment and preservice teachers' pedagogical content knowledge

AbstractThis study examined the relationship between preservice teachers' Educative Teacher Performance Assessment (edTPA) scores, the most widely implemented preservice teacher performance assessment in the United States, and the pedagogical content knowledge (PCK) levels reflected in their edTPA portfolios. PCK is a specialized form of teacher professional knowledge widely considered a paramount construct of teacher quality. Thirty‐six preservice science teachers' PCK levels were determined by analyzing their edTPA portfolios, including videos and written commentaries, with the PCK mapping approach, an enumerative analytical method capable of quantifying and visualizing the integration of PCK components. Data analysis indicates there was a significant positive correlation between participants' edTPA scores and their PCK levels (rs[36] = 0.949, p &lt; 0.0001). Through further analysis, this study also identified several key features of preservice teachers' PCK revealed in their edTPA portfolios: (1) preservice science teachers' Knowledge of Student's Understanding (KSU) and Knowledge of Instructional Strategies and Representations (KISR) were the most developed components of PCK; (2) accounting for over 50% of all connections between PCK components, KSU, KISR, and Knowledge of Assessments were among the most frequently integrated components of PCK; (3) Knowledge of Science Curriculum was the least frequently integrated, and therefore the weakest, component of preservice teachers' PCK; and (4) inquiry was the most frequently mentioned, and typically the only Orientation toward Teaching Science found in participants' edTPA materials. This study contributes to the growing body of scholarship on the characteristics of preservice science teachers' PCK and becomes one of only a handful of empirical studies that examine the edTPA's diagnostic abilities. Finally, potential implications regarding the edTPA in science teacher education, as well as directions for future research surrounding preservice science teacher PCK and PCK development, are discussed.

Military Protein Intake Related to Strength and Fat Mass Independent of Energy Intake.

Kinetic military units operate in austere training environments and deprivation not commonly experienced by competitive athletes. Nutritional strategies to protect against decrements in performance and potential injury risk may differ for these two groups. A cross sectional analysis was conducted to determine energy and macronutrient characteristics associated with performance metrics. 78 male subjects (age: 28.4±6.0years, height: 178.3±6.7cm, mass: 84.3±9.4kg, 8.5±5.8years of service) assigned to Marine Corps Forces Special Operations Command completed a 1-day performance assessment. Body mass, lean body mass, fat mass (FM), aerobic capacity (VO2max), lactate inflection point (LT), anaerobic power, anaerobic capacity, knee flexion strength, knee extension strength, peak knee flexion strength, and peak knee extension strength outcome values were recorded. Dietary intake was collected using automated self-administered 24-hour dietary recall (ASA24). Performance assessment scores were compared with macronutrient intake and controlled for energy intake using analysis of covariance. Differences in knee flexion strength, knee extension strength, peak knee flexion strength, and peak knee extension strength were significant across low (LPRO), medium (MPRO), and high (HPRO) protein intake groups (p<0.05) with LPRO performance metrics significantly lower than both MPRO and HPRO and MPRO significantly lower than HPRO. FM was significantly higher in LPRO than MPRO or HPRO (p<0.05). Low carbohydrate intake (LCHO) was associated with greater body mass and FM compared with high (HCHO) (p<0.05). There was no association between fat intake and any variable. Increases in protein intake may have beneficial performance effects independent of total energy intake, while moderate increases in carbohydrate intake may not be sufficient to enhance physical performance in a special operations population.

Perturbation-based gait training to improve daily life gait stability in older adults at risk of falling: protocol for the REACT randomized controlled trial

BackgroundThe European population is rapidly ageing. There is an urgent need for innovative solutions to reduce fall risk in older adults. Perturbation-based gait training is a promising new method to improve reactive balance responses. Whereas positive effects on task-specific dynamic balance recovery during gait have been shown in clinical or laboratory settings, translation of these effects to daily life gait function and fall risk is limited. We aim to evaluate the effect of a 4-week perturbation-based treadmill training on daily-life dynamic gait stability, assessed with inertial sensor data. Secondary outcomes are balance recovery performance, clinical balance and gait assessment scores, the amount of physical activity in daily life and falls incidence during 6 months follow-up.MethodsThe study is a monocenter assessor-blinded randomized controlled trial. The target study sample consists of 70 older adults of 65 years and older, living in the community and with an elevated risk of falling. A block-randomization to avoid seasonal effects will be used to allocate the participants into two groups. The experimental group receives a 4-week, two times per week perturbation-based gait training programme on a treadmill, with simulated slips and trips, in combination with cognitive dual tasks. The control group receives a 4-week, two times per week treadmill training programme under cognitive dual-task conditions without perturbations. Participants will be assessed at baseline and after the 4-weeks intervention period on their daily-life gait stability by wearing an inertial sensor on the lower back for seven consecutive days. In addition, clinical balance and gait assessments as well as questionnaires on falls- and gait-efficacy will be taken. Daily life falls will be followed up over 6 months by a fall calendar.DiscussionWhereas perturbation-based training has shown positive effects in improving balance recovery strategies and in reducing laboratory falls, this study will contribute to investigate the translation of perturbation-based treadmill training effects in a clinical setting towards improving daily life gait stability and reducing fall risk and falls.Trial registrationNTR7703 / NL66322.028.18, Registered: January 8, 2019; Enrolment of the first participant April 8, 2019.

Concordance Between Expert and Nonexpert Ratings of Condensed Video-Based Trainee Operative Performance Assessment

We examined the impact of video editing and rater expertise in surgical resident evaluation on operative performance ratings of surgical trainees. Randomized independent review of intraoperative video. Operative video was captured at a single, tertiary hospital in Boston, MA. Six common general surgery procedures were video recorded of 6 attending-trainee dyads. Full-length and condensed versions (n = 12 videos) were then reviewed by 13 independent surgeon raters (5 evaluation experts, 8 nonexperts) using a crossed design. Trainee performance was rated using the Operative Performance Rating Scale, System for Improving and Measuring Procedural Learning (SIMPL) Performance scale, the Zwisch scale, and ten Cate scale. These ratings were then standardized before being compared using Bayesian mixed models with raters and videos treated as random effects. Editing had no effect on the Operative Performance Rating Scale Overall Performance (-0.10, p = 0.30), SIMPL Performance (0.13, p = 0.71), Zwisch (-0.12, p = 0.27), and ten Cate scale (-0.13, p = 0.29). Additionally, rater expertise (evaluation expert vs. nonexpert) had no effect on the same scales (-0.16 (p = 0.32), 0.18 (p = 0.74), 0.25 (p = 0.81), and 0.25 (p = 0.17). There is little difference in operative performance assessment scores when raters use condensed videos or when raters who are not experts in surgical resident evaluation are used. Future validation studies of operative performance assessment scales may be facilitated by using nonexpert surgeon raters viewing videos condensed using a standardized protocol.

Does the performance assessment of students during surgery clerkship in a medical faculty jeopardize the objectivity of unbiased structured clinical examination?

Aim: Objective structured clinical examination (OSCE) has become an important assessment method in medicine which is more reliable than traditional exams. Faculty members make an opinion about the students during their clerkship. We aimed to investigate whether their judgments affect the scoring in the OSCE application.Material and Methods: The study was planned prospectively. 4th-year students participating in the OSCE was identified as the working group. At the end of the clerkship, the faculty members gave a performance assessment (PA) score including the professional attitude of the students. PA and OSCE scores were compared. Two other faculty members participated all OSCE stations as external evaluators.Results: There was a difference between OSCE and PA scores. Fewer students were successful at OSCE than PA (p=0.002). While the mean PA scores of three of the five faculty members were statistically similar, other two were different. The scores given by the responsible faculty members and the external evaluators from the faculty of internal medicine were similar (p>0.05). The evaluators from the surgical faculty gave lower scores statistically different from both groups (p0.05). There was a strong relationship between the scores given by the faculty members responsible for the OSCE application and the faculty members from the surgical faculty (r=0.936; p0.001) and those from the internal medicine faculty (r=0.947; p0.001).Conclusion: PA scores of the faculty members did not affect the OSCE scores which were supported by the external observers. The OSCE assessment was not influenced by the PA, and was found to be reliable.

Relationships between Performance Assessment Scores and Student Skills in a Training Class for Basic Nursing Techniques

Relationships between Performance Assessment Scores and Student Skills in a Training Class for Basic Nursing Techniques

Improving Shoulder Dystocia Outcomes With Teamwork and Deliberate Practice [26S

INTRODUCTION: The management of shoulder dystocia requires coordinated time-sensitive teamwork. A complication of performing the posterior arm delivery is neonatal humerus fracture. The study objective was to determine the effectiveness of teaching shoulder dystocia management using simulation aimed at teamwork and communication, with a focus on deliberate practice of posterior arm delivery skills. METHODS: A prospective pre/post educational study design was used to assess effectiveness of a curriculum designed for labor and delivery personnel using simulation for the coordinated team management of shoulder dystocia. A portable task trainer was used to practice safe delivery of the posterior arm including inserting the provider hand on the side of the fetal chest and splinting of the humerus. Outcomes were measured using performance assessment scores, safety culture surveys and the rate of neonatal humerus fractures in shoulder dystocias over one year. Internal review board exemption was obtained. Paired t-tests and repeated measures were used for data analysis. RESULTS: Teamwork drills were held with 166 multi-disciplinary participants in a large obstetrical hospital over one year. Statistically significant improvement occurred in knowledge (pre 64.01%, post 94.22%, p<0.0001), performance (pre 61.83%, post 92.86%, p<0.0001) and safety culture surveys (pre 3.27, post 1.57, p<0.0001, 1=high, 5=low). Compared with the year prior to the study, the rate of neonatal humerus fractures in shoulder dystocias was reduced by 42.86%, (pre 5.51%, post 3.88%, p<0.0001). CONCLUSION: Simulation training improved teamwork performance during shoulder dystocia emergencies. Deliberate practice of posterior arm delivery skills significantly reduced the rate of neonatal humerus fractures.

Clinical Effectiveness of Simulation in Eclampsia and Obstetrical Hypertensive Emergencies [24S

INTRODUCTION: Simulation prepares teams for rapid response to rare obstetrical emergencies like hypertensive crisis and eclampsia. The study objective was to determine the clinical effectiveness of teamwork simulation training in eclampsia and obstetrical hypertensive emergencies. METHODS: A prospective quasi-experimental educational study design was used to assess the effectiveness of a simulation curriculum on eclampsia and hypertensive emergency. A high fidelity manikin was programmed to produce seizures and hypertension. Outcomes were measured using pre/post knowledge tests, performance assessment scores, safety culture surveys and clinical parameters of intensive care unit (ICU) admissions and length of stay. Internal review board exemption was obtained. Paired t-tests and repeated measures were used for data analysis. RESULTS: Over one year, 256 obstetrical staff participated in weekly drills in a large obstetrical hospital with over 10,000 deliveries per year. Statistically significant improvement occurred in pre-post scores of knowledge (59.86%, 93.87%, p<0.0001), performance (56.54%, 93.54%, p<0.0001) and safety culture (3.12, 1.45, 1=high 5=low, p<0.0001). Clinical outcomes for patients diagnosed with eclampsia indicated fewer ICU admissions and shorter lengths of stay (pre 4.58 days, post 3.50 days, p=0.048) as compared with the year prior to initiation of the simulation program. CONCLUSION: Teamwork simulation training demonstrated improved team knowledge, performance and attitudes for responsiveness to hypertensive emergencies and eclampsia. Fewer ICU admissions and shorter lengths of stay for these rare events suggest improved patient care after initiating teamwork training.